`L i- 2ddlZddlmZddlmZddlmZmZmZddl Z ddl Z ddl m Z ddlmZddlmZddlmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>dd l?m@Z@dd lAmBZBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKmLZLdd lMmNZNdd lOmPZPdd lQmRZRmSZSmTZTmUZUddlVmWZWmXZXmYZYmZZZm[Z[m\Z\ddl]m^Z^m_Z_m`Z`ddlambZbddlcmdZdmeZeide)de*de/de0de-de2de+dedee7dde;de<dee1d d!e.d"e,d#ee1d$ d%eed$ d&ed'eiZfid(ed)ed*eed+,d-eed./d0ed1d2d3e3d4ee3d+5d6e"d7e>d8ee>d./d9e$d:e6d;e8d?d@ee!dA?idBe(dCee!dDdAEdFee dDGdHee!dDd>EdIee6dDGdJee8dDGdKee$dDGdLee!dMdAEdNee dMGdOee!dMd>EdPee6dMGdQee8dMGdRee$dMGdSee!dTdAEdUee dTGdVee!dTd>EdWee6dTGee8dTGee$dTGee!dXdAEee dXGee!dXd>Eee6dXGee8dXGee$dXGedY ZgdZZhe:ehed[Ziid\ed]e&d^e'd_ee'd./d`e#daedbe9dcee9dDGddee9dXGdeee9dMGdfee9dTdghdiee9dDdghdjee9dTdkhdlee9dDdkhdmee9dAndoedpeedDGeedXGeedMGe%e4ee=dqZjekZleljejeljegeljefeljeihdrZnhdsZoenjeoZqhdtZrhduZshdvZthdwZuhdxZvhdyZwhdzZxhd{Zyhd|Zzhd}Z{hd~Z|hdZ}ddhZ~dZdZdZe jjdeezdZe jjdee{dZdZe jjdeeeleqz dZdZe jjdeeeleqz dZe jjdeeegeqz dZe jjdejdZddge je jgfddge je jgfddge je jgfddge jdgfddge jdgfgZe jjdeejj#efj#e jjdedZe jjdegj#e jjdee jdd>ggdfe jdd>ggdfgzdZe jjdegj#dZdZdZe jj/de jjdeeeleqz eejz dZe jj/de jjdeeyexzdZe jjdeeydZe jjdeeydZe jj/de jjde^dZe jjdeexdZe jjdeevdZe jjdeevdZe jjdeevjAexdZdZdZe jjdeerdZe jjdeereszdZe jjdeerdZdZe jjdeerdZdZe jjdeeeljAeefe|z dZe jjdeeeljAeefe|z dZe jjdeeeleefz e|z enz dZe jjdeeeleefz e|z eqz dZe jjdeexewze|z dZe jjdeeye|z dZdZe jjdeexd3hz dZe jjdeeweyzdZe jjdeeejeqz dZe jjdegdZe jjded+fed+fe d.fee!d?d.fe$d.fe5d+fe6d.fe8d.fe:d+fg e jjdeegdZdZdZdZdZdZdZdZieeeege eeege!eege$eeege3eeege6eeege8eeege>eeege"eeege1egee1d egee1d ege7eegeEegeeege*eege-eegie/eege0eegeegeHege.egeBege,ege)ege+eegeCegeIegeDegeFegeGegeJegeKege;eegermsM$)&)009 D +F + / /Q / 72:: F G(unnormalized_multilabel_confusion_matrix/unnormalized_multilabel_confusion_matrix_sample) samplewiserr6unnormalized_zero_one_lossrr.r0rf2_score)beta f0.5_score?matthews_corrcoef_scoreweighted_f0.5_scoreweightedaverageruweighted_f1_scorer|weighted_f2_scoreweighted_precision_scoreweighted_recall_scoreweighted_jaccard_scoremicro_f0.5_scoremicromicro_f1_scoremicro_f2_scoremicro_precision_scoremicro_recall_scoremicro_jaccard_scoremacro_f0.5_scoremacromacro_f1_scoremacro_f2_scoremacro_precision_scoresamples) macro_recall_scoremacro_jaccard_scoresamples_f0.5_scoresamples_f1_scoresamples_f2_scoresamples_precision_scoresamples_recall_scoresamples_jaccard_scorerc t|i|\}}}t|t|z }tj||tj|j tj d|fdtjggS)a' The dimensions of precision-recall pairs and the threshold array as returned by the precision_recall_curve do not match. See func:`sklearn.metrics.precision_recall_curve` This prevents implicit conversion of return value triple to an higher dimensional np.array of dtype('float64') (it will be of dtype('object) instead). This again is needed for assert_array_equal to work correctly. As a workaround we pad the threshold array with NaN values to match the dimension of precision and recall arrays respectively. rconstant) pad_widthmodeconstant_values)r-lenrharraypadrffloat64nan)rjrk precisionrecall thresholdspad_threshholdss rl(precision_recall_curve_padded_thresholdsrsz%;D$KF$K!Ivz)ns:6O 88   FF!!"**-o.!#     rn)r2r-rrrrunnormalized_log_lossrrr1weighted_roc_aucsamples_roc_auc micro_roc_auc ovr_roc_aucovr)r| multi_classweighted_ovr_roc_auc ovo_roc_aucovoweighted_ovo_roc_aucpartial_roc_auc)max_fprr weighted_average_precision_score)samples_average_precision_scoremicro_average_precision_scorerr,rr5> rr,rrrrrrrrrpr>rrsrr2r0rrr1rr.rrr-rv>rrsr0rr.rv>r1rr >rrsrr2r0rr.rr-r rrrrv>"rrsrrr2r0rrrrrr.rrr}rrrrrrrrr-rrr`r_rorprvrrry>r6r r5>rrr,rr1rrrrrrrr rrrr>rr6r rrrrrrr}rrrrrrrrrrrrrrrr\rorrrry> r/rr%r'r"r*r(rr3rr4r#>rr!rrr6r rrrrr%r'rrr"rrWr*rrrr(rxr3rrr4r\r>rsr/rr2rr0rr.rrr}rrr$rr&r-rr rrrr`r_r#rYrZrorvrry>r!rrr*>rr$r&rYctt|j|jdz}||z }||z }||fS)zMake targets strictly positiverc)absminy1y2offsets rl_require_positive_targetsrBs? RVVXrvvx( )A -F&LB&LB r6Mrnctt|j|jdz }|jtj}|jtj}||z }||z }||fS)z$Make targets strictly larger than -1gGz?)rrrfrhrrs rl_require_log1p_targetsrJsc RVVXrvvx( )D 0F 2:: B 2:: B&LB&LB r6Mrncttzttztztt k(sJttztk(sJyN)SYMMETRIC_METRICSNOT_SYMMETRIC_METRICSsetTHRESHOLDED_METRICS"METRIC_UNDEFINED_BINARY_MULTICLASS ALL_METRICSrnrltest_symmetry_consistencyrTsX   ! " # - - [      5 5#% ?? ?rnnamectd}|jddd}|jddd}|tvrt||\}}n|tvrt ||\}}|jddd}|jddd}t |}|tvr/|tvr t||||||d|zyJdt||||||d|zy) Nrrtsizerz%s is not symmetricerr_msgz This case is currently unhandled) rTrandintMETRICS_REQUIRE_POSITIVE_YrMETRICS_WITH_LOG1P_YrrMETRIC_UNDEFINED_BINARYMULTILABELS_METRICSrJ)r random_statey_truey_pred y_true_bin y_pred_binmetrics rltest_symmetric_metricr`s &a(L  ! !!QU ! 3F  ! !!QU ! 3F ))266B % %/?%%a%:J%%a%:J  F && & & z:.z:.-4  =< <5 66 " 66 ")D0 rncFtd}t|}d}tdD]m}|jddd}|jddd}|tvrt ||\}}|||}|||}t j||rkd}n|rt|dy) NrTrtrrF seems to be symmetric) rTrrangerrrrhallclose ValueError) rrralways_symmetric_rrnominalswappeds rltest_not_symmetric_metricrs&a(L  F  1X %%a%7%%a%7 - -6vvFNFF(({{7G,$   D6!7899rnc0d}d}t|tjt|d5t|dddt |tjt |d5t |dddy#1swYGxYw#1swYyxYw)Nr r0z is not symmetricmatchr)rpytestraisesAssertionErrorrr)symnot_syms rltest_symmetry_testsrs CG# ~y8I-J K'g&'g& zC50F)G H'!#&'' '' ''s B+ B B  Bcxtd}|jddd}|jddd}|tvrt||\}}n|tvrt ||\}}t ||d\}}t5t|}t||||||d|zdddy#1swYyxYw)Nrrtrrr %s is not sample order invariantr) rTrrrrrrDrNrrJ)rrrry_true_shuffley_pred_shufflers rltest_sample_order_invariancers&a(L  ! !!QU ! 3F  ! !!QU ! 3F ))266B % %/?%,VV!%L"NN   T" 66 " >> 26=    s >)B00B9cdtd}|jddd}|jddd}|j|j}||j ddz}t |||d\}}}t D]*}t|}t||||||d |z ,tD]*}t|}t||||||d |z ,tD]I}t|}t||||||d |z t||||||d |z Ky) NrrtrrrcTrekeepdimsrrr) rTruniformshapergrDrrrJTHRESHOLDED_MULTILABEL_METRICSMULTIOUTPUT_METRICS) rrry_scorerry_score_shufflerrs rl7test_sample_order_invariance_multilabel_and_multioutputrsi%a(L ! !!QX ! 6F  ! !!QX ! 6F"" "5G w{{D{11G6=a73NNO$ T" 66 " >> 26=  / T" 67 # >? 36=  $  T" 67 # >? 36=  66 " >> 26=   rnc<td}|jddd}|jddd}|tvrt||\}}n|tvrt ||\}}t |}t |}tj|tj|}}t|jdt|jdtj|d}tj|d} tj|d} tj|d} t5t|} | ||} t| ||| d|z t| ||| d |z t| || | d |z t| ||| d |z t| ||| d |z t| || | d |z t| ||| d |z t| || | d|z t| ||| d|z tj t"5| || dddtj t"5| | |dddtj t"5| || dddtj t"5| | |dddtj t"5| || dddtj t"5| | | ddd|t$t&zt(zvrqd|vrBtj*t,5t/j0| | | sJ dddn+tj t"5| | | ddddddy#1swYqxYw#1swYSxYw#1swY5xYw#1swYxYw#1swYxYw#1swYxYw#1swY^xYw#1swYjxYw#1swYyxYw)Nrrtrrrc)rc)rcrz,%s is not representation invariant with listrz3%s is not representation invariant with np-array-1dz7%s is not representation invariant with np-array-columnz@%s is not representation invariant with mix np-array-1d and listzK%s is not representation invariant with mix np-array-1d and np-array-columnzD%s is not representation invariant with mix list and np-array-columnroc_auc)rTrrrrrlistrhrrLndimreshaperNrrJrrrrrrwarnsr mathisnan)rrrry1_listy2_listy1_1dy2_1d y1_column y2_columny1_rowy2_rowrmeasures rl&test_format_invariance_with_1d_vectorsrs&a(L   a  /B   a  /B ))*2r2B % %'B/B2hG2hG88B<"5Euzz1%uzz1% 5'*I 5'*I ZZw 'F ZZw 'F  f+T"R. 7G $ BTI  5% IDP  9i ( MPTT   5' " V   7E " V   5) $ 2    9e $ 2    7I & V   9g & V ]]: & " 5& ! " ]]: & " 65 ! " ]]: & $ 7F # $ ]]: & $ 67 # $ ]]: & & 9f % & ]]: & & 69 % &  "@ @CV V D \\"89>::fVV&<===>>]]:.+66*+Kf+f+Z " " " " $ $ $ $ & & & &>>++Kf+f+s1DP5 N.?!P N;*!P O!P6 O!P! O"+!P O.;PO:1"P PP.N8 3P;O PO PO P"O+ 'P.O7 3P:P ?PP PPc@td}|jddd}|jddd}tjddg|}tjddg|}d}ddg}t 5t |}|||} |} |t vr t| |} | ||} t| | dj| | |jd |jd } t| | d j| |tvrwt| | } | ||} t| | d j| | |jd |jd } t| | d j| dddy#1swYyxYw)Nrrtrreggsspam pos_label+{0} failed string vs number invariance testrO2{0} failed string object vs number invariance test)labelsz,{0} failed string vs number invariance test) rTrrhrrNCLASSIFICATION_METRICSMETRICS_WITH_POS_LABELrrLformatrfMETRICS_WITH_LABELS) rrrry1_stry2_str pos_label_str labels_strrmeasure_with_number metric_strmeasure_with_strmeasure_with_strobjs rl7test_classification_invariance_string_vs_numbers_labelsr-ss &a(L   a  /B   a  /B XXvv& ' +F XXvv& ' +FM&!J  &'-$Rn ) ) }EJ%ff5  AHHN )s);V]]3=OP  HOOPTU & & J?J)&&9  # FMMdS  #-V]]3-?sAS"T  ##FMMdS E&&&s 8DFFctd}|jddd}|jddd}tjddg|}d}t 5t |}|t vr||}|tvr t||}|||}|||} t|| dj| ||jd |} t|| d j| netjt5|||dddtjt5||jd |ddddddy#1swYLxYw#1swYxYw#1swYyxYw) Nrrtrrrrrrrrr)rTrrhrrNrrr"rrLr#rfrrr) rrrrr%r'rr*r)r+r,s rl4test_thresholded_invariance_string_vs_numbers_labelsr/ss&a(L   a  /B   a  /B XXvv& ' +FM  /$T* . .J--$Z=I "(R. )&"5  # ELLTR  #-V]]3-?"D  ##LSS z* #vr" #z* /v}}S)2. /9//4 # # / /9//s=B'E. E !E.,E"E.E E."E+ 'E..E7rcrzy_true, y_scorec|tk(r|g}|g}tjtd5|||dddy#1swYyxYw)Nzcontains (NaN|infinity)r)rrrr)rrrs rl)test_regression_thresholded_inf_nan_inputr1sH ) z)C D vw   s >A)rcrtcvtj|js+d}tj|j rd}n-d}n*d}tj|j rd}nd}d|d|}t j t|5|||dddy#1swYyxYw) z{check that classification metrics raise a message mentioning the occurrence of non-finite values in the target vectors.rNaNzinfinity or a value too largerzInput z contains rN)rhisfiniteallr anyrrr)rrr input_nameunexpected_valuers rl!test_classification_inf_nan_inputr:s ;;v  " " $ 88F    !$ >  88G  "$ > zl*-=,>?G z 1 vw   s  B//B8cgdgd}}d}tjt|5|||dddy#1swYyxYw)zocheck that classification metrics raise a message of mixed type data with continuous/binary target vectors.)abr<皙?皙?333333?zJClassification metrics can't handle a mix of binary and continuous targetsrN)rrr)rrrrs rl+test_classification_binary_continuous_inputrBsD&GFT  z 1 vw   s 9Act|}|tvrddg}n|tvrddg}nddg}t|dD]\}}||g|gy)Nrcrtgffffffrrepeat)rrrr)rrvaluesijs rlcheck_single_samplerIse  F ))Q % %Qq)1sQCrnct|}tddgdD]<\}}}}|tj||ggtj||gg>y)NrrcrD)rrrhr)rrrGrHkls rlcheck_single_sample_multioutputrN-sV  Fq!fQ/7 1arxx!Q!288aVH#567rnignorect|yr)rIrs rltest_single_samplerR4s rnct|yr)rNrQs rltest_single_sample_multioutputrTDs $D)rnctjgdgdgdg}tjddgddgddgg}t|}tjt 5|||dddy#1swYyxYw)N)rcrrrc)rrcrcrc)rcrcrrcrrcrhrrrrr)rrrrs rl(test_multioutput_number_of_output_differrWJsm XX|\<@ AF XX1v1v1v. /F  F z "vvs  A22A;c 6td}|jddd}|jddd}t|}|||}tdD]F}|j |j d}t ||dd|f|dd|f|d|zHy) Nrrtrrrr2rcz'%s is not dimension shuffling invariantr)rTrrr permutationrrJ)rrrrrerrorrperms rl=test_multioutput_regression_invariance_to_dimension_shufflingr]Ts&a(L  ! !!QW ! 5F  ! !!QW ! 5F  F 66 "E 1X '' Q8 6!T'?F1d7O 4 =F  rnz1ignore::sklearn.exceptions.UndefinedMetricWarning coo_containercd}d}td|d|d\}}td|d|d\}}tj|dg|zgg}tj|dg|zgg}||}||}t|}t|} |D cgc] } t| } } | D cgc] } t| } } tD]z} t | }t |trd|_| |_ |||}t||||d| z t|| | |d | z t||| |d | z |ycc} wcc} w) NrK2rcrT n_features n_classesr n_samplesallow_unlabeledtmpzO%s failed representation invariance between dense and sparse indicator formats.rz\%s failed representation invariance between dense array and list of list indicator formats.zW%s failed representation invariance between dense and list of array indicator formats.) r rhvstackrrr isinstancer __module____name__rJrK)r^rcrdrrry1_sparse_indicatory2_sparse_indicatory1_list_array_indicatory2_list_array_indicatorr<y1_list_list_indicatory2_list_list_indicatorrrrs rl)test_multilabel_representation_invariancerqgsII *  EAr +  EAr B!y)* +B B!y)* +B'+'+"2h"2h/FG!d1gGG/FG!d1gGG#' T" fg & %F "FOR.  &(; < 6    )+A B %    *,C D %  =' HGs D=%Ec dgdgddgggdggdgtjgddggdg}t|}|D]-}tjt 5|||ddd/y#1swY:xYw)Nrcrtr)rrt)rrcrobjectdtyperV)rmultilabel_sequencesrseqs rl+test_raise_value_error_multilabel_sequencesrxs qcAq6   "q!fX.  F# ]]: &  3     s  A((A1 cNd}d}td}|jd||f}|jd||f}|j|j}t|}|t vr|n|}|||d} |||d} t d| zdd t| | |z d | y) NrtrrrTr]F0We failed to test correctly the normalize optionr Failed with )rTrnormalrrrrMrJ rrcrdrrrrmetricspredmeasure_normalizedmeasure_not_normalizeds rl+test_normalize_option_binary_classificationrsII%a(L  ! !!Yi\ ! BF  ! !!Yi\ ! BF!!v||!4G$G117vD >$VTUC !! B *tf%rnc>d}d}td}|jd||f}|jd||f}|j||f}t|}|tvr|n|}|||d} |||d} t d| zdd t | | |z d | y) NrKrrrTr]Frzr{rr|)rTrrrrrMrJr~s rl/test_normalize_option_multiclass_classificationrsII%a(L  ! !!Yi\ ! BF  ! !!Yi\ ! BF""I(>"?G$G117vD >$VTUC !! B *tf%rncjd}d}td}td|dd|\}}td|dd|\}}|j|j}|dg|zz }|dg|zz }t|}|t vr|n|} ||| d} ||| d } t d | zdd t| | |z d | y)NrKdrrcT)rbrcrrerdrr]Frzr{rr|)rTr rrrrrMrJ) rrcrdrrrrrrrrrs rl/test_normalize_option_multilabel_classificationrs II%a(L/ IAv/ IAv"" "5G qcIoF qcIoF$G117vD >$VTUC !! B *tf%rnc |j\}}|||d}t|t|D cgc]} ||dd| f|dd| fc} |||d} t| ||j|j|||d} t| t j |t j |dt} t j | dk7r-|||d} t| t j|| n|||d} t| d|rN|||d}t|t j t|D cgc]} ||| || c} tjt5|||d dddtjt5|||d dddycc} wcc} w#1swYAxYw#1swYyxYw) Nr~rrr)rerurz)weightsrunknowngarbage) rrJrravelrhmeanrgintr|rrr)rrry_true_binarizey_pred_binarize is_multilabelrdrc label_measurerG micro_measure macro_measurerweighted_measuresample_measures rl_check_averagingr1s+00Iy6648M9%  ?1a4(/!Q$*? @ 667;Mvo3357L7L7NO 667;MM277=#9:ff_1C8G vvg!!&&*E("**]G*TU!&&*E(!, B  GG#9-?1-q/AB   z "2vvy12 z "2vvy122S @2222s#G G G$ G0$G-0G9ct|jd}t|}|tvrt ||||||y|t vrt ||||||yt d)N multilabelz2Metric is not recorded as having an average option)rR startswithrMETRICS_WITH_AVERAGINGr"THRESHOLDED_METRICS_WITH_AVERAGINGr)rrrrrrrrs rlcheck_averagingrgsm"6*55lCM  F %% FFO_m  3 3 FG_g} MNNrnc6d\}}td}|jd||f}|jd||f}|j||f}tj |}|j |}|j |} t ||||| |y)N)r`r2rr)rTrrrCfit transformr) rrdrcrrrrlbrrs rltest_averaging_multiclassrxs Iy%a(L  ! !!Yi\ ! BF  ! !!Yi\ ! BF""I(>"?G    f %Bll6*Oll6*OD&/6?GTrncd\}}td|d|d\}}|dd}|dd}tdjd|f}|}|} t||||| |y) N)(rrcrFrarrr)r rTr}r) rrdrcryrrrrrs rltest_averaging_multilabelrs{!Iy )  DAqsVF rsVF #**Y*@GOOD&/6?GTrnctjd}tjd}tjd}|}|}t||||||yNrr2)rhzerosrrrrrrrs rl$test_averaging_multilabel_all_zeroesrsH XXg F XXg FhhwGOOD&/6?GTrnctjd}tjd}|}|}dd}t|||||dy)Nrc&tt|||Sr)r7r.)rrr|s rlrmz=test_averaging_binary_multilabel_all_zeroes..s=R'>rnT)r)r)rhrr)rrrr binary_metrics rl+test_averaging_binary_multilabel_all_zeroesrsK XXg F XXg FOOM rnctjd}tjd}tjd}|}|}t||||||yr)rhonesrrs rl"test_averaging_multilabel_all_onesrsH WWW F WWW FgggGOOD&/6?GTrnc tjjd}|jddt |}|dk(r t |dn|}|||d}t ||||tjt |d |z ||||}tjt5t ||td |d |d |#1swYnxYw||||j}t ||d|d|d | |tj||dtj||dd} t || d|z |ddd} tj|} d| ddd<|ddd} |ddd} || | | }|||| }t ||d|d|d | |jds$dD]}t ||||||zd|z !dj!t#|t#|t#|dz}tjt|5|||tj$||gdddy#1swYyxYw)Nrrc rr5)rL sample_weight)rzAFor %s sample_weight=None is not equivalent to sample_weight=onesrz>Unweighted and weighted scores are unexpectedly almost equal (z) and (z) for zFWeighted scores for array and list sample_weight input are not equal (z != rdz.Weighting %s is not equal to repeating samplesrtzUZeroing weights does not give the same result as removing the corresponding samples ( unnormalized)rtrAz/%s sample_weight is not invariant under scalingzJFound input variables with inconsistent numbers of samples: \[{}, {}, {}\]r)rhrandom RandomStaterrrrJrrrrrtolistrEcopyrr#rShstack)rrrrrngrunweighted_scoreweighted_scoreweighted_score_listrepeat_weighted_scoresample_weight_subsetsample_weight_zeroed y1_subset y2_subsetweighted_score_subsetweighted_score_zeroedscaling error_messages rlcheck_sample_weight_invariancers ))   "CKK2CGK4M&*-C%CWVq !Fb"D9r2RWW3r7%;<S B-@N ~ & (.9).$ @    !R}7K7K7MN . 6# "m!, "m!, @4G)A.77=1 !114a4I14a4I"9,@#2r9MN !"7 ? ??> *  G r2]W-DEIDP   ##)6  l2. ]0Ka0O$  z 7Pr2RYY }/M%NOPPPs7!CC!?!I))I2cd}td}|j|f}|j|f}t|}t||||y)Nr`rr)rT random_samplerr)rrdrrrrs rl(test_regression_sample_weight_invariancer&sUI%a(L  ' 'i\ ' :F  ' 'i\ ' :F  F"4@rncd}td}|j|f}|j|f}t|}|j|dz f}tjt d5||||ddd|j|dzfj |df}tjt d 5||||dddy#1swY`xYw#1swYyxYw) Nr`rrrcz'Found input variables with inconsistentrrrtz)Sample weights must be 1D array or scalar)rTrrrrrr )rrdrrrrrs rl*test_regression_with_invalid_sample_weightr7sI%a(L  ' 'i\ ' :F  ' 'i\ ' :F  F ..Y]4D.EM z)R S<vv];<!..Y]4D.EMM AM z)T U<vv];<< << <> &tVV\J&tVVVDrnctd}tddddd\}}tddddd\}}tj||g}tj||g}|j |j }||j dd z}t|}|tvrt||||yt||||y) Nrrcrr`FrarTr) rTr rhrgrrrgrrr) rrryaybrrrrs rl(test_multilabel_sample_weight_invariancers&a(L *bRW EAr +bRW EArYYBx F YYBx F"" "5G w{{D{11G  F ""&tVVWE&tVVVDrnctd}|jddd}|jddd}t|}t||||y)NrrtrYr)rTrrr)rrrrrs rl)test_multioutput_sample_weight_invariancersT &a(L  ! !!QW ! 5F  ! !!QW ! 5F  F"4@rnctjgdgdg}tjgdgdg}tjgd}tjgd}tjgd}tj|d\}}tD]Y}||g||gfD]L\}} |tvr| j d kDrt |} | || |d } | || d } t| | |N[y) N)rcrcrrrrrcrc)rrcrcr)rrcrt)rrtr2)r2rrcrtT)return_inverserc)r r|r~)rhruniquerrr rrL) y_true_multilabely_pred_multilabely_true_multiclassy_pred_multiclassr rinverse_labelsrrrr score_labelsscores rltest_no_averaging_labelsrs, !=>, !=>++ XXl #F &>A~& D  1 2  1 2 DNFF..6;;? &F!&&NL6648E |U>-B C D Drnc"td}d\}}|jdd||f}|jdd||f}t|}|||}tt ||D])}|dd|f} |dd|f} || | } t || +y)NrrrKrtr)rTrrrrrK rrrdrcrrrrr\ y_score_perm y_true_perm current_scores rl-test_multilabel_label_permutations_invariancers&a(L Iy  ! !!Qi-C ! DF""1ay).D"EG  F 67 #EU9-y92q$w' QWo {L9 E=1 2rnctd}d\}}|jdd||f}|j|j}||j ddz}d||j ddk(df<d||j ddk(df<t |}|||}t t||D]Q}|dd|f} |dd|f} || | } |tk(rtj| sJ| d kDrDJt|| Sy) NrrrtrrcTrrKg.A) rTrrrrgrrrr#rhr5rKrs rl?test_thresholded_multilabel_multioutput_permutations_invariancers+&a(L Iy  ! !!Qi-C ! DF"" "5G w{{D{11G%&F6::a=A q !$%F6::a=A q !  F 67 #EU9-y9 6q$w' QWo {L9 3 3;;}- -- 3& && } 5 6rncd\}}td}|j||}tj| }||j dj ddz }|j d||}t|}|||}tt||D]r} tj|t} tj|| t| <|dd| f} tj| |} || | } t|| ty)N)rr2rrrdrcrrt)rTrandrhrrgr rrrrrrarangertakerK)rrdrcrrrrrrr\ inverse_permrrrs rl.test_thresholded_metric_permutation_invariancers"Iy%a(L 95G 667( DTXX2X&..r155G  ! !!YY ! ?F  F 67 #EU9-y92xx 5 #%99Y#7 T$Z q,/ ggdF+ {L9 E=12rn metric_namecLtjjd}tjdgdzdgdzzt}|j dd|j }d }tjt| 5t|||dddy#1swYyxYw) N*rr2rrtrtrrz7Labels in y_true and y_pred should be of the same type.r) rhrrrrsrrrr TypeErrorr!)rrrrrs rl"test_metrics_consistent_type_errorrs ))   #C 6(Q,&A-V  " T"&)44[AII%6!%;!AWG z 1r2s  CC$c dt||}|j||}|j||} |||fi|} |jd|j|d||d<|jd} t| tj r|j| ||d< t j|t j| d} | rc||| fi|} t| | t|||| fi|}t|| t||||fi|}t|| t|td5||| fi|} tt|j| || t|dddy#t ttf$rd} YwxYw#1swYyxYw)N)devicerrVTF)atol)array_api_dispatch) rIasarraygetrhrhndarrayr RuntimeErrorrrJrErrF)rarray_namespacer  dtype_namea_npb_np metric_kwargsxpa_xpb_xp metric_nprVnumpy_as_array_works metric_xpmetric_xp_mixed_1metric_xp_mixed_2s rlcheck_array_api_metricrMs ov 6B ::d6: *D ::d6: *DtT3]3I)5)+ / *6*4* o& ##M2K+rzz*')zz+fz'M m$ % 4 4#477    + #4??   + #4??   + 4 0 477  bjj3R 8  +   7 |Z 0% % %6  s),F ;F& F#"F#&F/c tjgd}tjgd}t||||||dtjgd|}t|||||||y)Nr)rrcrrcrrrr?@?rtrhrrrrr r y_true_np y_pred_nprs rl,check_array_api_binary_classification_metricr(sm&I&I  HH1DM  #rnctjgd}tjgd}ddd}t||}|D]A}t||||f||dd|tjgd| } t||||f||| d|Cy) N)rrcrtr2)rrcrrtrrrzr@rwg?r{rparamsrr rtrhr(_get_metric_kwargs_for_array_api_testingr rrr rr&r'additional_paramsmetric_kwargs_combinationsrrs rl0check_array_api_multiclass_classification_metricr3s&I&I2"J "4        !5ZH      '   rnc4tjddgddgddgg|}tjddgddgddgg|}ddd}t||}|D]A}t||||f||dd|tjgd |} t||||f||| d|Cy) Nrcrrtr*r+r{r,r)r!r?r"r.r0s rl0check_array_api_multilabel_classification_metricr5s1a&1a&1a&1DI1a&1a&1a&1DI2"J "4         C      '   rnct|tr|jjn |j}|dk(r'tt dkrt jdtjgd|}tjgd|}i}t|j}d|vrd|d<t||||f||d|d|vr/tjgd ||d<t||||f||d|yy) Nr&z1.14.0zJmean_poisson_deviance's dependency `xlogy` is available as of scipy 1.14.0)r"r?r#g@rt)rwrwrtrtrrr)r?r"?rw) rhrfuncrjrQrPrskiprhrrrr) rrr r func_namer&r'r metric_paramss rl!check_array_api_regression_metricr=s(267(C $$I++ ]8=T0T X -Z@I))皙?rw333333?rt)r@rArC)rwrDr? dense_outputFr7T)rhrrrr)rrr rX_npY_nprs rlcheck_array_api_metric_pairwiserHUs 88_o6j ID 88_o6j IDM6*555(- n%         )- n%     rngr8c#VK|jD]\}}|D]}||f ywr)items)metric_checkersrcheckerscheckers rl!yield_metric_checker_combinationsrNs<+113" "G'/ ! ""s')z#array_namespace, device, dtype_name)idszmetric, check_funcc|||||yrr)rrr r check_funcs rltest_array_api_compliancerRsv ;rn df_lib_namepandaspolarscLtj|}|jgd}|jgd}t|} ||j |j }t|||y#t $rtj |dY7wxYw)N)r!r#rr#)r#r!r!r!z can not deal with 1d inputs)r importorskipSeriesrto_numpyrr:rJ)rrSdf_librrrexpected_metrics rltest_metrics_dataframe_seriesr\s  -F ]]- .F ]]/ 0F  %FB !2FOO4EFF66*O< B {m#?@ABs%A??!B#"B#cig}|jD]U\}}|t|jvrg}|D]/}|D](}|j}|||<|j |*1|}W|S)zHelper function to enable specifying a variety of additional params and their corresponding values, so that they can be passed to a metric function when testing for array api compliance.)rJrrrappend) rr-r2paramrFnew_combinationsrkvalue new_kwargss rlr/r/s#% 6 v  &)44 4 0 4F 4#[[] $) 5! '' 3 4 4 &6" 6 &%rncZtjjd}|jddd}|jddd}|tvrt ||\}}|t vr(|jddd}|jddd}t|}|||}t|ttjtfsJt|tjtjfrJt|tr(td|Dstd|DsJyyy) zEnsure that the returned values of all metrics are consistent. It can either be a float, a numpy array, or a tuple of floats or numpy arrays. It should not be a numpy float64 or float32. rrtrrrc3<K|]}t|tywr)rhrb.0vs rl z2test_returned_value_consistency..* s7A:a'7sc3PK|]}t|tj ywr)rhrhrres rlrhz2test_returned_value_consistency..* s ? *+Jq"** %? s$&N)rhrrrrrrrrhrbrtuplerfloat32r6)rrrrrrs rltest_returned_value_consistencyrl s ))   "C [[AE[ *F [[AE[ *F ))266B &&Q0Q0  F 66 "E eeRZZ7 88 8%"**bjj!9:: :%7773? /4? <   < 7 rn)r  functoolsrinspectr itertoolsrrrnumpyrhrsklearn._configrsklearn.datasetsr sklearn.exceptionsr sklearn.metricsr r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6sklearn.metrics._baser7sklearn.metrics.pairwiser8r9r:r;r<r=r>r?r@rArBsklearn.preprocessingrC sklearn.utilsrDsklearn.utils._array_apirErFrGrHsklearn.utils._testingrIrJrKrLrMrNsklearn.utils.fixesrOrPrQsklearn.utils.multiclassrRsklearn.utils.validationrSrTREGRESSION_METRICSr!r CURVE_METRICSrdictrupdaterMETRIC_UNDEFINED_MULTICLASSunionrrrr"r$METRICS_WITH_NORMALIZE_OPTIONrrrrrMETRICS_WITHOUT_SAMPLE_WEIGHTrrrrrmark parametrizesortedrrrrrrrr-r/infrinvalids_nan_infrFr1r:rBrIrNfilterwarningsrRrTrWr]rqrxrr intersectionrrrrrrrrrrrrrrrrrrrrstrrsrrr(r3r5r=rArHarray_api_metric_checkersrNrRr\r/rlrrnrlrs 22 *;5------------\8    1! JI3EF.,4  *  2 %&D 8.AB6"#>G$9C2.%g.C3&O  0U#K 6$%56"$6/0K66w#86$L%6&]'6(!'-5"I)6,]-6./60L162364 !,566'+C0768096:7; M;6<:>=6>jqI?6@ LA6BW\:FC6DgmZHE6F W3GG6Hgh8I6Jgk7CK6LW_gFM6N',@O6P7='BQ6R W3GS6Tgh8U6Vgk7CW6XW_gFY6Z",@"='B!+ysK)< YQG& J#L)D$]IF*k6r@F n ,  WX?  *  ( ]  zB w}i@ W]G< 7='uM Gzu 7='uM  Gzu! &w}c:' (6) *')+ 0(/(&--Dg%V-R0? Df &' )* %& =!  (&=%B%B&" &"(#L!"*D " H H!!  @(9!:; < B(=!>?:@:4 '  F3{#&HH I  ** Z F3{#&HH I{+{+| F3-.1SS T22j!45'/6'/VVbffbff Vbffbff Vbffbff Vbffa[Vbffa[  e'..02D2K2K2MN*,<= > #9#@#@#BC &&!Q# &&!Q#    D ,#9#@#@#BC D $7H%   K - - ! "  #  & H%(;>Q(Q!RS*T&*(;!<=>(;!<= > $OP.9H :QH V(;!<= >  (E!FGH8(E!FGH8 F0==>QR S,,^32lO"(>!?@ UA U F),NN OUU$(>!?@UAU&(>!?@UAU^PB   K%%c*<&=> ' (AA   K%%c*<&=> ' (<<&   K  ! " ' ( " " E E   K  ! " ' ( - -EE"  = = ' (EE.  !> >?A AD2 F&*T)UU V22$ F14GG H66B F3*+.PP Q22*(>? 4@ 4 $' 4 5 1 %u-  & % u D  f 67 4= @:% P% P' T.b:e488e  488 e88e488e( 488")e24883e<488=eF488GeP488QeZ=>[e\ !.1R0S]e^ !-0Q/R_e` )5aeh78iej)5ker)5sez)5{eB)5CeJ)5KeR)SeX=>YeZ=>[e\:;]e^;<_e`12aeb#)5%cej12kel!@ Amen67oep9:qer34set78uev01wex)5ye@ )5"45IeP7P" )-/' -/P/RS<T  <8(<={(;< ==> =&* !45 6 rn