L i4ddlmZddlmZddlmZddlmZddlZddlZddl m Z ddl m Z ddl m Z ddl Z ddlZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlm Z ddlm!Z!ddlm"Z"ddlm#Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)ddl(m*Z*ddl+m,Z,ddl-m.Z.ddl-m/Z/ddl0m1Z1ddl0m2Z2ddl3m4Z4ddl5m6Z6dd l5m7Z7e rdd!l8m9Z9d"Z:ee;Zd7d%Z?d7d&Z@d8d'ZAGd(d)e#ZBd9d*ZC d:d+ZD d;d,ZE d;d-ZF d;d.ZGdd2ZK d?d3ZLed45 d@d6ZMy)A) annotations)Callable)Sequence) lru_cacheN)Any)cast) TYPE_CHECKING) _deprecated)convert_positional_args)warn_experimental_argument)compute_hypervolume) _solve_hssp)BaseDistribution)CategoricalChoiceType) get_logger)_CONSTRAINTS_KEY)&_INDEPENDENT_SAMPLING_WARNING_TEMPLATE) _process_constraints_after_trial) BaseSampler)LazyRandomState) RandomSampler)_ParzenEstimator)_ParzenEstimatorParameters)IntersectionSearchSpace)_GroupDecomposedSearchSpace)_SearchSpaceGroup)_fast_non_domination_rank)_is_pareto_front)StudyDirection) FrozenTrial) TrialState)Studyg-q=ztpe:relative_paramsicFttjd|zdS)Ng?minmathceilxs b/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/optuna/samplers/_tpe/sampler.py default_gammar,5s tyyq!2 &&cLttjd|dzzdS)Ng?g?r$r%r)s r+hyperopt_default_gammar/9s tyy3' ,,r-c|dk(rtjgS|dkrtj|Stjd|z d|dz }tjd}tj||gdS)Nrr$?)numaxis)npasarrayoneslinspace concatenate)r*rampflats r+default_weightsr<=shAvzz"~ Rwwqz{{37CQV4wwr{~~tTl33r-ceZdZdZegdddddddd d eed ddddd d d  dd ZddZ ddZ ddZ ddZ d dZ d!dZ d"dZ d#dZ d$dZ d%dZe d&dZed'dZd(dZ d)dZy )* TPESamplera0.Sampler using TPE (Tree-structured Parzen Estimator) algorithm. On each trial, for each parameter, TPE fits one Gaussian Mixture Model (GMM) ``l(x)`` to the set of parameter values associated with the best objective values, and another GMM ``g(x)`` to the remaining parameter values. It chooses the parameter value ``x`` that maximizes the ratio ``l(x)/g(x)``. For further information about TPE algorithm, please refer to the following papers: - `Algorithms for Hyper-Parameter Optimization `__ - `Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures `__ - `Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance `__ For multi-objective TPE (MOTPE), please refer to the following papers: - `Multiobjective Tree-Structured Parzen Estimator for Computationally Expensive Optimization Problems `__ - `Multiobjective Tree-Structured Parzen Estimator `__ For the `categorical_distance_func`, please refer to the following paper: - `Tree-Structured Parzen Estimator Can Solve Black-Box Combinatorial Optimization More Efficiently `__ Please also check our articles: - `Significant Speed Up of Multi-Objective TPESampler in Optuna v4.0.0 `__ - `Multivariate TPE Makes Optuna Even More Powerful `__ Example: An example of a single-objective optimization is as follows: .. testcode:: import optuna from optuna.samplers import TPESampler def objective(trial): x = trial.suggest_float("x", -10, 10) return x**2 study = optuna.create_study(sampler=TPESampler()) study.optimize(objective, n_trials=10) .. note:: :class:`~optuna.samplers.TPESampler`, which became much faster in v4.0.0, c.f. `our article `__, can handle multi-objective optimization with many trials as well. Please note that :class:`~optuna.samplers.NSGAIISampler` will be used by default for multi-objective optimization, so if users would like to use :class:`~optuna.samplers.TPESampler` for multi-objective optimization, ``sampler`` must be explicitly specified when study is created. Args: consider_prior: Enhance the stability of Parzen estimator by imposing a Gaussian prior when :obj:`True`. The prior is only effective if the sampling distribution is either :class:`~optuna.distributions.FloatDistribution`, or :class:`~optuna.distributions.IntDistribution`. .. warning:: Deprecated in v4.3.0. ``consider_prior`` argument will be removed in the future. The removal of this feature is currently scheduled for v6.0.0, but this schedule is subject to change. From v4.3.0 onward, ``consider_prior`` automatically falls back to ``True``. See https://github.com/optuna/optuna/releases/tag/v4.3.0. prior_weight: The weight of the prior. This argument is used in :class:`~optuna.distributions.FloatDistribution`, :class:`~optuna.distributions.IntDistribution`, and :class:`~optuna.distributions.CategoricalDistribution`. consider_magic_clip: Enable a heuristic to limit the smallest variances of Gaussians used in the Parzen estimator. consider_endpoints: Take endpoints of domains into account when calculating variances of Gaussians in Parzen estimator. See the original paper for details on the heuristics to calculate the variances. n_startup_trials: The random sampling is used instead of the TPE algorithm until the given number of trials finish in the same study. n_ei_candidates: Number of candidate samples used to calculate the expected improvement. gamma: A function that takes the number of finished trials and returns the number of trials to form a density function for samples with low grains. See the original paper for more details. weights: A function that takes the number of finished trials and returns a weight for them. See `Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures `__ for more details. .. note:: In the multi-objective case, this argument is only used to compute the weights of bad trials, i.e., trials to construct `g(x)` in the `paper `__ ). The weights of good trials, i.e., trials to construct `l(x)`, are computed by a rule based on the hypervolume contribution proposed in the `paper of MOTPE `__. seed: Seed for random number generator. multivariate: If this is :obj:`True`, the multivariate TPE is used when suggesting parameters. The multivariate TPE is reported to outperform the independent TPE. See `BOHB: Robust and Efficient Hyperparameter Optimization at Scale `__ and `our article `__ for more details. .. note:: Added in v2.2.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v2.2.0. group: If this and ``multivariate`` are :obj:`True`, the multivariate TPE with the group decomposed search space is used when suggesting parameters. The sampling algorithm decomposes the search space based on past trials and samples from the joint distribution in each decomposed subspace. The decomposed subspaces are a partition of the whole search space. Each subspace is a maximal subset of the whole search space, which satisfies the following: for a trial in completed trials, the intersection of the subspace and the search space of the trial becomes subspace itself or an empty set. Sampling from the joint distribution on the subspace is realized by multivariate TPE. If ``group`` is :obj:`True`, ``multivariate`` must be :obj:`True` as well. .. note:: Added in v2.8.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v2.8.0. Example: .. testcode:: import optuna def objective(trial): x = trial.suggest_categorical("x", ["A", "B"]) if x == "A": return trial.suggest_float("y", -10, 10) else: return trial.suggest_int("z", -10, 10) sampler = optuna.samplers.TPESampler(multivariate=True, group=True) study = optuna.create_study(sampler=sampler) study.optimize(objective, n_trials=10) warn_independent_sampling: If this is :obj:`True` and ``multivariate=True``, a warning message is emitted when the value of a parameter is sampled by using an independent sampler. If ``multivariate=False``, this flag has no effect. constant_liar: If :obj:`True`, penalize running trials to avoid suggesting parameter configurations nearby. .. note:: Abnormally terminated trials often leave behind a record with a state of ``RUNNING`` in the storage. Such "zombie" trial parameters will be avoided by the constant liar algorithm during subsequent sampling. When using an :class:`~optuna.storages.RDBStorage`, it is possible to enable the ``heartbeat_interval`` to change the records for abnormally terminated trials to ``FAIL``. .. note:: It is recommended to set this value to :obj:`True` during distributed optimization to avoid having multiple workers evaluating similar parameter configurations. In particular, if each objective function evaluation is costly and the durations of the running states are significant, and/or the number of workers is high. .. note:: Added in v2.8.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v2.8.0. constraints_func: An optional function that computes the objective constraints. It must take a :class:`~optuna.trial.FrozenTrial` and return the constraints. The return value must be a sequence of :obj:`float` s. A value strictly larger than 0 means that a constraints is violated. A value equal to or smaller than 0 is considered feasible. If ``constraints_func`` returns more than one value for a trial, that trial is considered feasible if and only if all values are equal to 0 or smaller. The ``constraints_func`` will be evaluated after each successful trial. The function won't be called when trials fail or they are pruned, but this behavior is subject to change in the future releases. .. note:: Added in v3.0.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v3.0.0. categorical_distance_func: A dictionary of distance functions for categorical parameters. The key is the name of the categorical parameter and the value is a distance function that takes two :class:`~optuna.distributions.CategoricalChoiceType` s and returns a :obj:`float` value. The distance function must return a non-negative value. While categorical choices are handled equally by default, this option allows users to specify prior knowledge on the structure of categorical parameters. When specified, categorical choices closer to current best choices are more likely to be sampled. .. note:: Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v3.4.0. ) selfconsider_prior prior_weightconsider_magic_clipconsider_endpointsn_startup_trialsn_ei_candidatesgammaweightsseedz4.4.06.0.0)previous_positional_arg_namesdeprecated_versionremoved_versionTr1F N)r@rArBrCrDrErFrGrH multivariategroupwarn_independent_sampling constant_liarconstraints_funccategorical_distance_funcc ~|s?tjjddd}tj|dt t ||||| |xsi|_||_||_ ||_ | |_ t| |_ t| |_| |_| |_d|_d|_t)d |_| |_||_t0|_| r t5d | r(| s t7d t5d t9d|_| r t5d | t5d| t5dyy)Nz`consider_prior`z4.3.0rI)named_verr_verzI From v4.3.0 onward, `consider_prior` automatically falls back to `True`.)rArBrCrGrOrT)rHT)include_prunedrOzF``group`` option can only be enabled when ``multivariate`` is enabled.rPrRrSrT)r _DEPRECATION_WARNING_TEMPLATEformatwarningswarn FutureWarningr_parzen_estimator_parameters_n_startup_trials_n_ei_candidates_gamma_warn_independent_samplingr_rngr_random_sampler _multivariate_group_group_decomposed_search_space_search_space_groupr _search_space_constant_liar_constraints_funcr_parzen_estimator_clsr ValueErrorr)r?r@rArBrCrDrErFrGrHrOrPrQrRrSrTmsgs r+__init__zTPESampler.__init__!sRJ;;BB'wgCC MM%`a  -G% 31%&?&E2 - )"2 / *C'#D) ,$7) RV+=A 4DI+!1%5"  &~ 6  \ 'w /2Md2SD /  & 7  ' &'9 : $ 0 &'B C 1r-c|jjj|jj yN)rdrngrHre reseed_rng)r?s r+rtzTPESampler.reseed_rng}s(  '')r-c |jsiSi}|jxs |j }|jr|jJ|jj |||_|j j D]9}t|jD]\}}|jr|||<;|S|jj ||jD]\}}|jr|||<|Srr) rfrkrgrh calculateri search_spacessorteditemssinglerj)r?studytrial search_spaceuse_trial_cache sub_spacerV distributions r+infer_relative_search_spacez&TPESampler.infer_relative_search_spaces!!I46 ,,GD4G4G0G ;;66B BB'+'J'J'T'T(D $"55CC 6 *01B*C6&D,#**, )5L&6 6  "&"4"4">">uo"V"\"\"^ . D,""$!-L  . r-c ^|jr|jJi}|jjD]]}i}t|j D]\}}|j r|||<|j |j|||_n|j|||}|ik7r|jrutj|}tdt|tD]C} |jj|j t"d| tz|| | tzE|S)Nr:)rgrirwrxryrzupdate_sample_relativerkjsondumpsrangelen_SYSTEM_ATTR_MAX_LENGTH_storageset_trial_system_attr _trial_id_RELATIVE_PARAMS_KEY) r?r{r|r}paramsrrVr params_stris r+sample_relativezTPESampler.sample_relatives- ;;++7 77F!55CC Q ! *01B*C:&D,'..0-9 T*: d33E5,OP  Q**5%FF R#>?   r-c|ik(riStjtjf}|jd|d}t ||j kriS|j |||dS)NFTdeepcopystates use_cacher~)r!COMPLETEPRUNED _get_trialsrr`_sample)r?r{r|r}rtrialss r+rzTPESampler._sample_relativesl 2 I%%z'8'89""E&D"Q v;// /I||E5,|MMr-c Ptjtjf}|jd|d}t ||j kr|j j|||S|jr|jr}tfd|Dritjtj|j|j j j"|j j"d|i}|j%||||j& S)NFTrc3:K|]}|jvywrr)r).0r| param_names r+ z0TPESampler.sample_independent..sB%:-Bsz=dynamic search space is not supported for `multivariate=True`)r trial_numberindependent_sampler_name sampler_namefallback_reasonr)r!rrrrr`resample_independentrcrfany_loggerwarningrr[number __class____name__rrk)r?r{r|rparam_distributionrrr}s ` r+rzTPESampler.sample_independents %%z'8'89""E&D"Q v;// /''::uj*<   * *t/A/AB6BB:AA#-%*\\151E1E1O1O1X1X%)^^%<%<[  #$67 ||E5,DL_L_H_|`   r-c|jjs |js |jSg}d}|jj t d|x}r=|j||dz }|jj t d|x}r=t|dk(r |jStjdj|}|j|j|S)Nrr) state is_finishedrfr system_attrsgetrappendrrloadsjoinr)r?r| params_strsr params_str_irs r+ _get_paramszTPESampler._get_paramss ;; " " $D,>,><<  #00448L7MQqc5RSSlS   | , FA$00448L7MQqc5RSSlS { q << BGGK01 ell# r-c|Dcic]}|g}}|D]u}|j|}|j|jks6|jD]-\}}||}||j|j |/w|jD cic]\} } | t j | c} } Scc}wcc} } wrr)rkeysryrto_internal_reprr5r6) r?rr}rvaluesr|rrparamkvs r+_get_internal_reprzTPESampler._get_internal_reprsLX)XZ*b.)X)X TE%%e,F  "fkkm30<0B0B0DT,J ":.E:&--l.K.KE.RST T .4\\^.sFe z111Fs')T) handle_below)rkr!rrrrrsum _split_trialsrbrl_build_parzen_estimatorsamplerdrsra_compute_acquisition_funcr>_compareryto_external_repr)r?r{r|r}r~rrtn below_trials above_trials mpe_below mpe_above samples_below acq_func_valsretrdists r+rzTPESampler._sample s    )):+<+P>PQF )):+<+<=F""E&O"\   !'DA5<<188+CaDFD FvF F%2   KKN  " "$ . & " l00 <D1 00 <E1 "((8M8MN 66}iQZ[ !!-? , 2 2 4 E J"33C ODC O E 3Es F Fc|j||}|r|jr|Dcgc]2}|j|j|jk4}}t |||j |}t j|jsJ|j|||j|} n|j|||j} t| ts td| Scc}w)Nz5_parzen_estimator_cls must override _ParzenEstimator.)r_is_multi_objectiverr,_calculate_weights_below_for_multi_objectiverlr5isfiniteallrmr_ isinstancer RuntimeError) r?r{r}rr observationsr|param_mask_below weights_belowmpes r+rz"TPESampler._build_parzen_estimator3s..v|D E557SY JO !!#t'7'7'>'C'C'EE   Ivt55 M;;}-113 33,,lD,M,M}C,,lD,M,MC#/0VW W % s7C:cT|j|}|j|}||z }|Srr)log_pdf)r?samplesrrlog_likelihoods_belowlog_likelihoods_abovers r+rz$TPESampler._compute_acquisition_funcPs7 !* 1 1' : ) 1 1' :-0EE r-cxtt|jj}|dk(rt d|d||jk7rt d|d|jdt j |}|jDcic]\}}|||jc}}Scc}}w)Nrz0The size of `samples` must be positive, but got .zwThe sizes of `samples` and `acquisition_func_vals` must be same, but got (samples.size, acquisition_func_vals.size) = (z, z).) nextiterrsizernr5argmaxryitem)clsracquisition_func_vals sample_sizebest_idxrrs r+rzTPESampler._compare[s4 01277 ! OP[}\]^_ _ /44 4=#8#=#=">bB  992329--/B$!Q1X;##%%BBBsB6c (ddddddttdS)aReturn the the default parameters of hyperopt (v0.1.2). :class:`~optuna.samplers.TPESampler` can be instantiated with the parameters returned by this method. Example: Create a :class:`~optuna.samplers.TPESampler` instance with the default parameters of `hyperopt `__. .. testcode:: import optuna from optuna.samplers import TPESampler def objective(trial): x = trial.suggest_float("x", -10, 10) return x**2 sampler = TPESampler(**TPESampler.hyperopt_parameters()) study = optuna.create_study(sampler=sampler) study.optimize(objective, n_trials=10) Returns: A dictionary containing the default parameters of hyperopt. Tr1FrN)r@rArBrCrDrErFrG)r/r<r-r+hyperopt_parameterszTPESampler.hyperopt_parametersms'B##'"' "!+&  r-c<|jj||yrr)re before_trial)r?r{r|s r+rzTPESampler.before_trials ))%7r-c|tjtjtjfvsJ|jt |j||||j j||||yrr)r!rFAILrrlrre after_trial)r?r{r|rrs r+rzTPESampler.after_trialsd,,jooz?P?PQQQQ  ! ! - ,T-C-CUESX Y ((ufEr-) r@boolrAfloatrBrrCrrDintrErrFzCallable[[int], int]rGzCallable[[int], np.ndarray]rHz int | NonerOrrPrrQrrRrrS/Callable[[FrozenTrial], Sequence[float]] | NonerTzQdict[str, Callable[[CategoricalChoiceType, CategoricalChoiceType], float]] | NonereturnNone)rr)r{r"r|r rdict[str, BaseDistribution])r{r"r|r r}rrdict[str, Any]) r{r"r|r rstrrrrr)r|r rr)rlist[FrozenTrial]r}rrdict[str, np.ndarray]) r{r"r|r r}rr~rrr) r{r"r}rrrrrrr)rrrrrrr np.ndarray)rrrrrzdict[str, int | float])rr)r{r"r|r rr) r{r"r|r rr!rzSequence[float] | Nonerr)r __module__ __qualname____doc__r r,r<rprtrrrrrrrrr classmethodr staticmethodrrrrr-r+r>r>Hs Vp ' #& $!$(#( "!&3/>"*.#LP 'JDJD JD " JD ! JDJDJD$JD-JDJDJDJD$(JDJD J!JD$ ^%JD( )JD! JDX*#. $:#.>Y 6 N N#. N>Y N  N" " "  " - " " H$ =' =7R =  =(((2 (  (  (T2"     : & $ $   C+CDNC CC"( ( T8 F F F F ' F  Fr-r>ctj|d}tjd|zd|z}t||dk(<|S)Nrr3g?g?)r5maxmaximumEPS) loss_vals worst_pointreference_points r+_get_reference_pointrsA&&+Kjj{!2C+4EFO,/OOq() r-cg}g}g}g}|D]}|jtjk(r|j|2|r t |dkDr|j|T|jtj k(r|j||jtj k(r|j|Jt|||\} } td|t| z }t|||\} } td|t| z }t||\} }| | z| z}| | z|z|z}|jd|jd||fS)Nrc|jSrrrr|s r+z_split_trials..  r-keyc|jSrrrrs r+rz_split_trials..rr-) rr!rr_get_infeasible_trial_scorerr_split_complete_trialsrr_split_pruned_trials_split_infeasible_trialssort)r{rn_belowconstraints_enabledcomplete_trials pruned_trialsrunning_trialsinfeasible_trialsr|below_completeabove_complete below_pruned above_prunedbelow_infeasibleabove_infeasiblerrs r+rrskOMN  ;;*,, ,  ! !% ( %@%G!%K  $ $U + [[J// /  " "5 ) [[J-- -   ' 5 &223G!5mUG!TL,!Ws<001G)ABSU\)]&&!L03CCL!L03CCnTL4545  %%r-ct|t|}t|jdkr t|||St |||S)Nr)r&r directions'_split_complete_trials_single_objective&_split_complete_trials_multi_objective)rr{r$s r+r r sE'3v;'G 5  !6vugNN5feWMMr-c|jtjk(rt|d}nt|dd}|d|||dfS)Nc6tt|jSrrrrvaluers r+rz9_split_complete_trials_single_objective..eU[[9Qr-rc6tt|jSrrr6rs r+rz9_split_complete_trials_single_objective..r8r-T)rreverse) directionrMINIMIZErxrr{r$ sorted_trialss r+r2r2sL .111v+QR v+Q[_` ' "M'($; ;;r-c |dk(r gt|fS|t|k(r t|gfSd|cxkrt|ksJJtj|Dcgc]}|jc}}||j Dcgc]}|t jk(rdndc}z}t||}tj|d\}}ttj|tj||kd} t|d| d ztj| d zk(sJtjt|} | || k} | j|kr|| d z| d zk(sJ|| d zk(} || } || jz }t!t#| j%t#| | |t#t'| }tj(| |} t+t-t| j/}t1t|Dcgc] }||vs||}}t1t|Dcgc] }||vs||}}||fScc}wcc}wcc}wcc}w) Nrr1)r$T) return_counts)initialr)listrr5arrayrr1rMAXIMIZEruniquerrcumsumraranger_solve_hssp_with_cachetupleravelrrsetrtolistr)rr{r$r|lvalsdnondomination_ranksranks rank_countslast_rank_before_tiebreakindices indices_below need_tiebreak rank_i_lvals subset_sizeselected_indicesbelow_indices_setrrrs r+r3r3si!|4< CK F|R w $V $$ $$ $ HH7uell7 8E UEUEU Va>222d; VVE3E7K#6dKE; #BFF5;1G71R+S]_$` a u40145C\_`C`9aa bb biiF $G/3LLMMG#.237PST7TTTT+/H1/LL ]+  2 22 1 ,$$& ' '-( )  &|4 5    -1AB D}';';'=>?',S['9T!QBS=SF1ITLT',S['9X!QFW=WF1IXLX  %%38 V,UXs$I)I.) I33I3 I8I8ct|jdkDrqt|jj\}}t j |r| t dfS|jtjk(r| |fS| | fSy)Nrinf)rg) rintermediate_valuesrryr'isnanrr;rr<)r|r{stepintermediate_values r+_get_pruned_trial_scorerbs 5 $ $%)#&u'@'@'F'F'H#I  ::( )5%,& & __ 7 7 75,, ,5--- -r-cbt|t|}t|fd}|d|||dfS)Nct|Srr)rb)r|r{s r+rz&_split_pruned_trials.. s5LUTY5Zr-r)r&rrxr=s ` r+r!r!s;'3v;'G6'Z[M ' "M'($; ;;r-c|jjt}|.tjd|j dt dStd|DS)NzTrial z[ does not have constraint values. It will be treated as a lower priority than other trials.r]c3,K|] }|dkDs |ywrNr)rrs r+rz._get_infeasible_trial_score...s2AE12s )rrrr\r]rrr)r| constraints r+rr$sa##''(89J U\\N#I I U|2j222r-cbt|t|}t|t}|d|||dfS)Nr)r&rrxr)rr$r>s r+r"r"1s:'3v;'G6'BCM ' "M'($; ;;r-c dfd }tj|Dcgc] }|| c}}tj|dt}tj|}|dkr|Stj|Dcgc]}|j c}|}||j D cgc]} | tjk(rdndc} z}t|} t|d} || } t| | d} tj| r|Stj| jd t }tj"|t$ }t'|j d kr&|Dcgc]}| t| || dz c}|| <ntj(| | z d || <tj*| | ddtj,f}|| xxt/|Dcgc]\}}t|||f| c}}zcc<tj*|t1tj0|tz t||<|Scc}wcc}wcc} wcc}wcc}}w)Nc@duxstd|DS)Nc3&K|] }|dk ywrgr)rcs r+rzR_calculate_weights_below_for_multi_objective.._feasible..?s.W!qAv.Ws)r)r|rSs r+ _feasiblez?_calculate_weights_below_for_multi_objective.._feasible>s%4'W3.W?OPU?V.W+WWr-r1rr@F)assume_unique_lexsortedT) assume_paretor)dtyperBr3)r|r rr)r5r6wherer count_nonzerorr1rrFrrr r'isinfeyeshaperzerosrrprodrnewaxis enumerater)r{rrSrnr is_feasiblern_below_feasiblerOrP ref_pointon_front pareto_solshvloo_matcontribsloo limited_solsrs ` r+rr9s  X**LAqilABKHH[#s3M'' 41 JJ,7Q7 8 EE UEUEU Va>222d; VVE$U+IuEH/K [)4 HB zz"~vvk''*$77Gxx(6H 5  !  $[%5yPTU U   WWY%<2Fzz+{1bjj=/IJ OXY`Oa EKQ  QV 4i @  "$Hs266(;KS7Q,QSV!WM+ ?B 8 V  sI;I'I,I *I% r)maxsizect|t|f}tj||}tj|}tj|}t ||||Srr)rr5reshaperEr)rank_i_lvals_tuplerank_i_indices_tuplerYref_point_tuple lvals_shaperXrank_i_indicesr~s r+rJrJcsX+,c/.BCK::0+>LXX23N)I |^[) LLr-)r*rrr)r*rrr)rrrr) r{r"rrr$rr%rr+tuple[list[FrozenTrial], list[FrozenTrial]])rSequence[FrozenTrial]r{r"r$rrr)r|r r{r"rztuple[float, float])r|r rr)rrr$rrr)r{r"rrrSrrr) rtuple[float, ...]rztuple[int, ...]rYrrrrr)N __future__rcollections.abcrr functoolsrrr'typingrrr r\numpyr5optunar optuna._convert_positional_argsr optuna._experimentalr optuna._hypervolumer optuna._hypervolume.hssproptuna.distributionsrroptuna.loggingroptuna.samplers._baserrrr"optuna.samplers._lazy_random_stateroptuna.samplers._randomr%optuna.samplers._tpe.parzen_estimatorrroptuna.search_spacer$optuna.search_space.group_decomposedrroptuna.study._multi_objectiverroptuna.study._study_directionr optuna.trialr r! optuna.studyr"rrrrrr,r/r<r>rrr r2r3rbr!rr"rrJrr-r+rs "$$  C;3016%2HB->1BL7LBC:8$#"  X ,'-4] F] F@$& $&+$&69$&PT$&0$&NN !N*/N:=N0N< !<*/<:=<0<"& !"&*/"&:="&0"&J < !<*/<:=<0< 3< !<,/<0<' '#'F' 'T 1 M) M) M M' M  M Mr-