`L idZddlZddlZddlmZmZddlmZddlmZddl m Z ddl Z ddl mZmZmZdd lmZmZdd lmZmZdd lmZmZmZmZdd lmZdd lmZm Z m!Z!ddl"m#Z#ddl$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*ddl+m,Z,ddl-m.Z.ddl/m0Z0m1Z1ddl2m3Z3ddl4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:ddl;me j~e jjZBdZCdZDdZE d%dZFdZGdZHdZIGd d!er?r3r;has_check_inputrequires_feature_indexing estimatorsestimators_featuressupport_sample_weightir1 estimator estimator_fitfeaturesr6 fit_params_request_or_routerconsumes_sample_weightcurr_sample_weight sample_countsnot_indices_maskX_y_s" r7_parallel_build_estimatorsrq[sGGIz))L''K""I!44'(;(;]KO 2 Plj6PJ.h.A.A?S   !jnn_&E&Q (  < G- Q; Qq5,(:;<  Qx ,,E ,U #IMM{KM%MMM6        '!oo'    6x7J7J K %6%?%?)& "&; " !!56"df  " Gy I "m3"$3GY$G#G 78"#34+=K (#<1h;!B "a /; / 7+B7+B.qgVK(8 _ "b 0K 0)$""8,OG-R * **r9c |jd}tj||f}t||D]\}} t |dr{|j |dd| ffi|xsi} |t |jk(r|| z }N|dd|jfxx| ddtt |jfz cc<|j|dd| ffi|xsi} t|D]} || | | fxxdz cc<|S)zBPrivate function used to compute (proba-)predictions within a job.r predict_probaNr+) rJrVzerosziphasattrrslenclasses_rPpredict) rbrcrZ n_classespredict_paramspredict_proba_paramsr5probarfrhproba_estimator predictionsres r7_parallel_predict_probars1 I HHi+ ,E":/BC. 8 9o .5i55!X+#1#7ROC 2 233(a+++,uS!3!34551, ,)++!X+#7#=2K9% .aQ'(A-( .).. Lr9c |jd}tj||f}|jtj tj |t }t||D]\}} |j|dd| ffi|} |t|jk(rtj|| }Otj|dd|jf| ddtt|jf|dd|jf<tj||j} tj|dd| ftj |dd| f<|S)z@Private function used to compute log probabilities within a job.r)dtypeN)rJrVemptyfillinfarangeintrupredict_log_probarwrx logaddexprP setdiff1d) rbrcrZrzrIr5 log_proba all_classesrfrhlog_proba_estimatormissings r7_parallel_predict_log_probars> I)Y/0I NNBFF7))IS1K":/BC Q 89i99!AxK.SFS I../ / Y0CDI02||!Y///0#AuS1C1C-D'E$EF0Ia+++ , ll; 0B0BCG$&LL1g:1F$PIaj ! Q r9cDtfdt||DS)z8Private function used to compute decisions within a job.c3XK|]!\}}|jdd|ffi#ywNdecision_function.0rfrhrZrIs r7 z._parallel_decision_function.. s7 Ix $ ##AakN=f='*sumrurbrcrZrIs ``r7_parallel_decision_functionr % #&z3F#G r9cDtfdt||DS)z:Private function used to compute predictions within a job.c3XK|]!\}}|jdd|ffi#ywrryrs r7rz/_parallel_predict_regression..s7 Ix  !AxK.3F3rrrs ``r7_parallel_predict_regressionrrr9c eZdZUdZeddgdgeedddgeedddeeddd geedddeeddd gd gd gd gd gdegd gd gd Ze e d<e ddddddddddd fd Z e dd dZdZ d!dZe dZdZdZedZdZe dZfdZxZS)" BaseBaggingzBase class for Bagging meta-estimator. Warning: This class should not be used directly. Use derived classes instead. rGryNr+left)closedrrightbooleanr1r^ rfrXr?r>r3r; oob_score warm_startn_jobsr1r^_parameter_constraints?TF r?r>r3r;rrrr1r^c t |||||_||_||_||_||_||_| |_| |_ | |_ y)N)rfrX) super__init__r?r>r3r;rrrr1r^ selfrfrXr?r>r3r;rrrr1r^ __class__s r7rzBaseBagging.__init__5sb %  '(""4"$ ( r9)prefer_skip_nested_validationc t||dt|||ddgddd\}}|j||f|j|d|S) aBuild a Bagging ensemble of estimators from the training set (X, y). Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. y : array-like of shape (n_samples,) The target values (class labels in classification, real numbers in regression). sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Note that this is supported only if the base estimator supports sample weighting. **fit_params : dict Parameters to pass to the underlying estimators. .. versionadded:: 1.5 Only available if `enable_metadata_routing=True`, which can be set by using ``sklearn.set_config(enable_metadata_routing=True)``. See :ref:`Metadata Routing User Guide ` for more details. Returns ------- self : object Fitted estimator. rGcsrcscNFT) accept_sparserensure_all_finite multi_output)r?rE)rr*_fitr?)rrZr[rEr_s r7rGzBaseBagging.fitSsrJ *dE2   %.# 1tyy  (('     r9ciSrrs r7_parallel_argszBaseBagging._parallel_argss r9c tj}jd} | _j j j |||d<trtdfi|n;tt|_ d|vr|djjd<||j_ | j}n5t|t j"st%|jdz}|jdkDr t'd|_tj*t j"r j*} nz } | dkr-t'd jBtEj>fz| dk(r tGd StI| jJ\} tMj8rBtEj>dkDr*|jOtPtEj>|jOtP| _)tUd| jVdjYf dt[| D} xj>t]t^j`jcd| Dz c_xj@t]t^j`jcd| Dz c_ j6rjeS)a*Build a Bagging ensemble of estimators from the training set (X, y). Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. y : array-like of shape (n_samples,) The target values (class labels in classification, real numbers in regression). max_samples : int or float, default=None Argument to use instead of self.max_samples. max_depth : int, default=None Override value used when constructing base estimator. Only supported if the base estimator has a max_depth parameter. check_input : bool, default=True Override value used when fitting base estimator. Only supported if the base estimator has a check_input parameter for fit function. If the meta-estimator already checks the input, set this value to False to prevent redundant input validation. sample_weight : array-like of shape (n_samples,), default=None Sample weights. If None, then samples are equally weighted. Note that this is supported only if the base estimator supports sample weighting. **fit_params : dict, default=None Parameters to pass to the :term:`fit` method of the underlying estimator. Returns ------- self : object Fitted estimator. rrErG)rGz max_samples must be <= n_samplesz"max_features must be <= n_featuresr+z6Out of bag estimation only available if bootstrap=Truez6Out of bag estimate only available if warm_start=False oob_score_ estimators_zTn_estimators=%d must be larger or equal to len(estimators_)=%d when warm_start==TruezJWarm-start fitting without increasing n_estimators does not fit new trees.)sizerr^c 3 K|]O}tt| ||dz jjj Qyw)r+)r^rDr_N)r#rqr^rfrG) rrerZrDrX routed_paramsr\rstartsr]r[s r7rz#BaseBagging._fit..%so  0G. /QfQi&Q-0" '(2266   sAAc3&K|] }|d yw)rNrrts r7rz#BaseBagging._fit..6)D1!A$)Dc3&K|] }|d yw)r+Nrrs r7rz#BaseBagging._fit..9rrr)3rr1rJ _n_samples _validate_y_validate_estimator_get_estimatorrrrrfrGrM max_depthr? isinstancenumbersrrrOrLr>floatn_features_in_maxrKr3rrrvrrestimators_features_rXrwrr-rrr2MAX_INT_seedsr"r^rrPlist itertoolschain from_iterable_set_oob_score)rrZr[r?rrDrEr_r1r5r>n_more_estimatorsr all_resultsrXrr\rr]s``` ` @@@@@r7rzBaseBagging._fitsd*$*;*;< GGAJ #   Q    !4!4!67  $*7J '  +D%F:FM!GM&+ &;M #*,?I#@ ''++O<  (1DOO %  **KK)9)9:kAGGAJ67K  #?@ @( d'')9)9 :,,L ))5 1t0043F3FFGL $-- -AB B1c,/0 *~~$..UV V ??t~~UV V 4 &4??gdM&B!D (*D % --D4D4D0EE q <$$c$*:*:&;<=  ! # ! K(= t{{( $ f!. ??s4#3#34q8  s43C3C/D E$$W3D$E  h 4<< 373F3F3H  6]  $ D OO ) ))D )D D   !!T OO ) ))D )D D&  ! >>   1 % r9cy)z+Calculate out of bag predictions and score.Nr)rrZr[s r7rzBaseBagging._set_oob_scoreAr9ctt|jdk(s|jddk(r t|dS|S)Nr+Tr)rwrJrrr[s r7rzBaseBagging._validate_yEs2 qww<1  a- -r9c #K|jD]X}t||j|j|j|j |j |j\}}||fZywr)rrBr;r3rrrKrL)rseedr@rAs r7_get_estimators_indicesz#BaseBagging._get_estimators_indicesJsmKK 2D/H''##""!!/ +O^">1 1 2sA)A+cT|jDcgc]\}}| c}}Scc}}w)a The subset of drawn samples for each base estimator. Returns a dynamically generated list of indices identifying the samples used for fitting each member of the ensemble, i.e., the in-bag samples. Note: the list is re-created at each call to the property in order to reduce the object memory footprint by not storing the sampling data. Thus fetching the property may be slower than expected. )r)r_rAs r7estimators_samples_zBaseBagging.estimators_samples_[s&9=8T8T8VW#41nWWWs $ct|jj}t}|j ddj ddt |j dr%|j ddj ddn$|j ddj ddt |j dr|j ddnAt |j dr|j ddn|j dd|j |j ||S) ajGet metadata routing of this object. Please check :ref:`User Guide ` on how the routing mechanism works. .. versionadded:: 1.5 Returns ------- routing : MetadataRouter A :class:`~sklearn.utils.metadata_routing.MetadataRouter` encapsulating routing information. )ownerrG)callercalleerrsryr)rfmethod_mapping)rr__name__raddrvr)rrouterrs r7get_metadata_routingz BaseBagging.get_metadata_routingjs9 dnn&=&=>&%6::&/B ; 4&&(/ :"")O"LPP*?Q"")I"FJJ*9K 4&&(*= >   &9BU  Vt**,o>""*=o"V""*=i"P T002> R r9cy)z"Resolve which estimator to return.Nrrs r7rzBaseBagging._get_estimatorrr9ct|}t|jjj |j_t|jjj |j_|Sr)r__sklearn_tags__rr input_tagssparse allow_nan)rtagsrs r7rzBaseBagging.__sklearn_tags__saw')!)$*=*=*?!@!K!K!R!R$,T-@-@-B$C$N$N$X$X! r9N r)NNTN)r __module__ __qualname____doc__rrrrrdict__annotations__rrr rGrrrrrpropertyrrrr __classcell__rs@r7rrsv!%!34d;!(AtFCD Xq$v 6 ZAg 6 Xq$v 6 ZAg 6  [(k[ k"'(;#$D(  :&+4 4 lob:: 2" X X8t11r9r) metaclassc eZdZdZ ddddddddddd fd Zd Zd Zd Zd Zd Z dZ e e dddZ xZS)r.aA Bagging classifier. A Bagging classifier is an ensemble meta-estimator that fits base classifiers each on random subsets of the original dataset and then aggregate their individual predictions (either by voting or by averaging) to form a final prediction. Such a meta-estimator can typically be used as a way to reduce the variance of a black-box estimator (e.g., a decision tree), by introducing randomization into its construction procedure and then making an ensemble out of it. This algorithm encompasses several works from the literature. When random subsets of the dataset are drawn as random subsets of the samples, then this algorithm is known as Pasting [1]_. If samples are drawn with replacement, then the method is known as Bagging [2]_. When random subsets of the dataset are drawn as random subsets of the features, then the method is known as Random Subspaces [3]_. Finally, when base estimators are built on subsets of both samples and features, then the method is known as Random Patches [4]_. Read more in the :ref:`User Guide `. .. versionadded:: 0.15 Parameters ---------- estimator : object, default=None The base estimator to fit on random subsets of the dataset. If None, then the base estimator is a :class:`~sklearn.tree.DecisionTreeClassifier`. .. versionadded:: 1.2 `base_estimator` was renamed to `estimator`. n_estimators : int, default=10 The number of base estimators in the ensemble. max_samples : int or float, default=1.0 The number of samples to draw from X to train each base estimator (with replacement by default, see `bootstrap` for more details). - If int, then draw `max_samples` samples. - If float, then draw `max_samples * X.shape[0]` samples. max_features : int or float, default=1.0 The number of features to draw from X to train each base estimator ( without replacement by default, see `bootstrap_features` for more details). - If int, then draw `max_features` features. - If float, then draw `max(1, int(max_features * n_features_in_))` features. bootstrap : bool, default=True Whether samples are drawn with replacement. If False, sampling without replacement is performed. bootstrap_features : bool, default=False Whether features are drawn with replacement. oob_score : bool, default=False Whether to use out-of-bag samples to estimate the generalization error. Only available if bootstrap=True. warm_start : bool, default=False When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble. See :term:`the Glossary `. .. versionadded:: 0.17 *warm_start* constructor parameter. n_jobs : int, default=None The number of jobs to run in parallel for both :meth:`fit` and :meth:`predict`. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls the random resampling of the original dataset (sample wise and feature wise). If the base estimator accepts a `random_state` attribute, a different seed is generated for each instance in the ensemble. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. verbose : int, default=0 Controls the verbosity when fitting and predicting. Attributes ---------- estimator_ : estimator The base estimator from which the ensemble is grown. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 estimators_ : list of estimators The collection of fitted base estimators. estimators_samples_ : list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. estimators_features_ : list of arrays The subset of drawn features for each base estimator. classes_ : ndarray of shape (n_classes,) The classes labels. n_classes_ : int or list The number of classes. oob_score_ : float Score of the training dataset obtained using an out-of-bag estimate. This attribute exists only when ``oob_score`` is True. oob_decision_function_ : ndarray of shape (n_samples, n_classes) Decision function computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, `oob_decision_function_` might contain NaN. This attribute exists only when ``oob_score`` is True. See Also -------- BaggingRegressor : A Bagging regressor. References ---------- .. [1] L. Breiman, "Pasting small votes for classification in large databases and on-line", Machine Learning, 36(1), 85-103, 1999. .. [2] L. Breiman, "Bagging predictors", Machine Learning, 24(2), 123-140, 1996. .. [3] T. Ho, "The random subspace method for constructing decision forests", Pattern Analysis and Machine Intelligence, 20(8), 832-844, 1998. .. [4] G. Louppe and P. Geurts, "Ensembles on Random Patches", Machine Learning and Knowledge Discovery in Databases, 346-361, 2012. Examples -------- >>> from sklearn.svm import SVC >>> from sklearn.ensemble import BaggingClassifier >>> from sklearn.datasets import make_classification >>> X, y = make_classification(n_samples=100, n_features=4, ... n_informative=2, n_redundant=0, ... random_state=0, shuffle=False) >>> clf = BaggingClassifier(estimator=SVC(), ... n_estimators=10, random_state=0).fit(X, y) >>> clf.predict([[0, 0, 0, 0]]) array([1]) NrTFrrc :t |||||||||| | |  yNrrrrs r7rzBaggingClassifier.__init__W8 %#%1!%  r9cF|j tS|jSzEResolve which estimator to return (default is DecisionTreeClassifier))rfrrs r7rz BaggingClassifier._get_estimatorts >> !)+ +~~r9c  |jd}|j}tj||f}t |j |j |jD]\}}}t||} t|dr/|| ddfxx|j|| ddfdd|fz cc<O|j|| ddfdd|f} d} t|D]} | | s || | | fxxdz cc<| dz } !|jddk(jr td||jdddtj fz } t#|tj$|d}| |_||_y)Nrrsr+axis{Some inputs do not have OOB scores. This probably means too few estimators were used to compute any reliable oob estimates.)rJ n_classes_rVrtrurrrrrvrsryrPranyrnewaxisr argmaxoob_decision_function_r)rrZr[r5rrrfsamplesrhmaskpjreoob_decision_functionrs r7rz BaggingClassifier._set_oob_scorezsGGAJ __ hh :67 ,/   d668Q8Q-   (Iw$GY77Dy/2D!G$ (?(?tQwZH-)$ %%qqz1h;&?@y)AAw#AqtG,1,Q ( OOO #q ( - - / 9  !,koo1o.Eam.T T"1bii !&DE &;##r9ct|d}t|tj|d\|_}t |j|_|S)NTr)return_inverse)rr!rVuniquerxrwrrs r7rzBaggingClassifier._validate_ysB  &$Q'99Qt< qdmm,r9c t||d|j|fi|}|jjt j |ddS)aPredict class for X. The predicted class of an input sample is computed as the class with the highest mean predicted probability. If base estimators do not implement a ``predict_proba`` method, then it resorts to voting. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. **params : dict Parameters routed to the `predict_proba` (if available) or the `predict` method (otherwise) of the sub-estimators via the metadata routing API. .. versionadded:: 1.7 Only available if `sklearn.set_config(enable_metadata_routing=True)` is set. See :ref:`Metadata Routing User Guide ` for more details. Returns ------- y : ndarray of shape (n_samples,) The predicted classes. ryr+rr)rrsrxtakerVr)rrZrIpredicted_probabilitiys r7ryzBaggingClassifier.predictsO: &$ 2!3!3!3A!@!@}}!!299-C!#LTU!VVr9c t|dttddgdddtrt dfi|n#t t t _tjj\}}td |jdjfd t|D}t|jz }|S) a]Predict class probabilities for X. The predicted class probabilities of an input sample is computed as the mean predicted class probabilities of the base estimators in the ensemble. If base estimators do not implement a ``predict_proba`` method, then it resorts to voting and the predicted class probabilities of an input sample represents the proportion of estimators predicting each class. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. **params : dict Parameters routed to the `predict_proba` (if available) or the `predict` method (otherwise) of the sub-estimators via the metadata routing API. .. versionadded:: 1.7 Only available if `sklearn.set_config(enable_metadata_routing=True)` is set. See :ref:`Metadata Routing User Guide ` for more details. Returns ------- p : ndarray of shape (n_samples, n_classes) The class probabilities of the input samples. The order of the classes corresponds to that in the attribute :term:`classes_`. rsrrNFrrrreset)rsrc 3(K|]}ttj||dzj||dzjj j ddj j ddyw)r+ryNrs)r{r|)r#rrrrrfrNrrerZrrrs r7rz2BaggingClassifier.predict_proba..s  -G+ ,  VAE];))&)fQUmD,66::9dK%2%<%<%@%@RV%W    sBBr)rr(r*rrrrfr-rXrr"r^rrPr) rrZrIrr all_probar}rrs `` @@r7rszBaggingClassifier.predict_probasB &$8    %.#    +D/LVLM!GM&+%'&BM #2$2C2CT[[Q6 H 4<< 373F3F3H  6]   I!2!22 r9c  t|dttjdrt ddgdddt rt dfi|n#ttt_tjj\}} t|j fd t|D}|d }td t|D]}t!j"|||}|t!j$jz}|St!j$j&fi|}|S) aPredict class log-probabilities for X. The predicted class log-probabilities of an input sample is computed as the log of the mean predicted class probabilities of the base estimators in the ensemble. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. **params : dict Parameters routed to the `predict_log_proba`, the `predict_proba` or the `proba` method of the sub-estimators via the metadata routing API. The routing is tried in the mentioned order depending on whether this method is available on the sub-estimator. .. versionadded:: 1.7 Only available if `sklearn.set_config(enable_metadata_routing=True)` is set. See :ref:`Metadata Routing User Guide ` for more details. Returns ------- p : ndarray of shape (n_samples, n_classes) The class log-probabilities of the input samples. The order of the classes corresponds to that in the attribute :term:`classes_`. rrrNFr$)rrc 3K|]h}ttj||dzj||dzjj j jywr+)rIN)r#rrrrrfrr's r7rz6BaggingClassifier.predict_log_proba..Ms J534$$VAYA?--fQi&Q-HOO(22DD  JsA.A1rr+)rr(rvrMr*rrrrfr-rXrr"r^rPrwrVrlogrs) rrZrIrr all_log_probarrrrs `` @@r7rz#BaggingClassifier.predict_log_probasO@ &$(;< 4??$7 8$en"' A ! /6I TV T % */%'*J '!6d6G6G U FAvIHFDLLI Jv J M&a(I1c-01 FLLM!4DE  F  1 12 2I 1t11!>v>?Ir9r)rrf) delegatesc t|dttddgdddtrt dfi|n#t t t _tjj\}}t|jfd t|D}t|jz }|S) aBAverage of the decision functions of the base classifiers. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. **params : dict Parameters routed to the `decision_function` method of the sub-estimators via the metadata routing API. .. versionadded:: 1.7 Only available if `sklearn.set_config(enable_metadata_routing=True)` is set. See :ref:`Metadata Routing User Guide ` for more details. Returns ------- score : ndarray of shape (n_samples, k) The decision function of the input samples. The columns correspond to the classes in sorted order, as they appear in the attribute ``classes_``. Regression and binary classification are special cases with ``k == 1``, otherwise ``k==n_classes``. rrrNFr$rrc 3K|]]}ttj||dzj||dzjj _ywr+)r#rrrrfrr's r7rz6BaggingClassifier.decision_function..syF  1G/ 0  VAE];))&)fQUmD$..@@   F A#A&rr(r*rrrrfr-rXrr"r^rPr) rrZrIrr all_decisions decisionsrrs `` @@r7rz#BaggingClassifier.decision_functiones> &$(;<    %.#    +D2EPPM!GM&+eg&FM #2$2C2CT[[Q6E EF 6]F   &):):: r9r)rrrrrrrrryrsrr r'rrrs@r7r.r.seR   : %$N WDHTN`*6RS@@r9c PeZdZdZ d dddddddddd fd Zd Zd Zd ZxZS) r/aA Bagging regressor. A Bagging regressor is an ensemble meta-estimator that fits base regressors each on random subsets of the original dataset and then aggregate their individual predictions (either by voting or by averaging) to form a final prediction. Such a meta-estimator can typically be used as a way to reduce the variance of a black-box estimator (e.g., a decision tree), by introducing randomization into its construction procedure and then making an ensemble out of it. This algorithm encompasses several works from the literature. When random subsets of the dataset are drawn as random subsets of the samples, then this algorithm is known as Pasting [1]_. If samples are drawn with replacement, then the method is known as Bagging [2]_. When random subsets of the dataset are drawn as random subsets of the features, then the method is known as Random Subspaces [3]_. Finally, when base estimators are built on subsets of both samples and features, then the method is known as Random Patches [4]_. Read more in the :ref:`User Guide `. .. versionadded:: 0.15 Parameters ---------- estimator : object, default=None The base estimator to fit on random subsets of the dataset. If None, then the base estimator is a :class:`~sklearn.tree.DecisionTreeRegressor`. .. versionadded:: 1.2 `base_estimator` was renamed to `estimator`. n_estimators : int, default=10 The number of base estimators in the ensemble. max_samples : int or float, default=1.0 The number of samples to draw from X to train each base estimator (with replacement by default, see `bootstrap` for more details). - If int, then draw `max_samples` samples. - If float, then draw `max_samples * X.shape[0]` samples. max_features : int or float, default=1.0 The number of features to draw from X to train each base estimator ( without replacement by default, see `bootstrap_features` for more details). - If int, then draw `max_features` features. - If float, then draw `max(1, int(max_features * n_features_in_))` features. bootstrap : bool, default=True Whether samples are drawn with replacement. If False, sampling without replacement is performed. bootstrap_features : bool, default=False Whether features are drawn with replacement. oob_score : bool, default=False Whether to use out-of-bag samples to estimate the generalization error. Only available if bootstrap=True. warm_start : bool, default=False When set to True, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble. See :term:`the Glossary `. n_jobs : int, default=None The number of jobs to run in parallel for both :meth:`fit` and :meth:`predict`. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls the random resampling of the original dataset (sample wise and feature wise). If the base estimator accepts a `random_state` attribute, a different seed is generated for each instance in the ensemble. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. verbose : int, default=0 Controls the verbosity when fitting and predicting. Attributes ---------- estimator_ : estimator The base estimator from which the ensemble is grown. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 estimators_ : list of estimators The collection of fitted sub-estimators. estimators_samples_ : list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. estimators_features_ : list of arrays The subset of drawn features for each base estimator. oob_score_ : float Score of the training dataset obtained using an out-of-bag estimate. This attribute exists only when ``oob_score`` is True. oob_prediction_ : ndarray of shape (n_samples,) Prediction computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, `oob_prediction_` might contain NaN. This attribute exists only when ``oob_score`` is True. See Also -------- BaggingClassifier : A Bagging classifier. References ---------- .. [1] L. Breiman, "Pasting small votes for classification in large databases and on-line", Machine Learning, 36(1), 85-103, 1999. .. [2] L. Breiman, "Bagging predictors", Machine Learning, 24(2), 123-140, 1996. .. [3] T. Ho, "The random subspace method for constructing decision forests", Pattern Analysis and Machine Intelligence, 20(8), 832-844, 1998. .. [4] G. Louppe and P. Geurts, "Ensembles on Random Patches", Machine Learning and Knowledge Discovery in Databases, 346-361, 2012. Examples -------- >>> from sklearn.svm import SVR >>> from sklearn.ensemble import BaggingRegressor >>> from sklearn.datasets import make_regression >>> X, y = make_regression(n_samples=100, n_features=4, ... n_informative=2, n_targets=1, ... random_state=0, shuffle=False) >>> regr = BaggingRegressor(estimator=SVR(), ... n_estimators=10, random_state=0).fit(X, y) >>> regr.predict([[0, 0, 0, 0]]) array([-2.8720]) NrTFrrc :t |||||||||| | |  yr r rs r7rzBaggingRegressor.__init__Jr r9c t|dttddgdddtrt dfi|n#t t t _tjj\}}t|jfd t|D}t|jz }|S) aPredict regression target for X. The predicted regression target of an input sample is computed as the mean predicted regression targets of the estimators in the ensemble. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The training input samples. Sparse matrices are accepted only if they are supported by the base estimator. **params : dict Parameters routed to the `predict` method of the sub-estimators via the metadata routing API. .. versionadded:: 1.7 Only available if `sklearn.set_config(enable_metadata_routing=True)` is set. See :ref:`Metadata Routing User Guide ` for more details. Returns ------- y : ndarray of shape (n_samples,) The predicted values. ryrrNFr$rrc 3K|]]}ttj||dzj||dzjj _ywr+)r#rrrrfryr's r7rz+BaggingRegressor.predict..syB  2G0 1  VAE];))&)fQUmD$..66   B r1r2) rrZrIrr all_y_haty_hatrrs `` @@r7ryzBaggingRegressor.predictgs8 &$ 2    %.#    +D)FvFM!GM&+EG&> !(* *~~r9r) rrrrrryrrrrs@r7r/r/sI\@   :?B38r9)NN)Mrrrabcrr functoolsrrwarningsrnumpyrVbaser r r metricsr rtreerrutilsrrrr utils._maskrutils._param_validationrrr utils._tagsrutils.metadata_routingrrrrrrutils.metaestimatorsr utils.multiclassr!utils.parallelr"r# utils.randomr$utils.validationr%r&r'r(r)r*_baser,r-__all__iinfoint32rrr8rBrqrrrrrr.r/rr9r7rTs '@@.@ *FF"0;.57  2 3 "((288   +0m+j #L2R,'Rj yyx]~{]r9