L i֦jddlZddlZddlZddlmZmZddlmZddlm Z m Z ddl m Z ddl Z ddlmZgdZGdd eZd Zd Zd Zd ZGddZGddZedgdZGddeZGddeZGddeZGddZdZdZdZ dZ!d"d Z" d#d!Z#y)$N) defaultdict namedtuple) attrgetter)AnyOptional) deprecated) DeviceType) EventListFormattedTimesMixinIntervalKernel FunctionEventFunctionEventAvg StringTable MemRecordsAccceZdZdZfdZdZdZdZdZdZ e dZ dd Z d Z d Zd ed efdZ ddZdZxZS)r z'A list of Events (for pretty printing).c|jdd}|jdd}|jdd}t||i|||_||_d|_||_y)N use_deviceprofile_memoryF with_flops)popsuper__init__ _use_device_profile_memory _tree_built _with_flops)selfargskwargsrrr __class__s b/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/torch/autograd/profiler_util.pyrzEventList.__init__sfZZ d3 $4e<ZZ e4  $)&)%- %cr|j|j|jd|_y)NT)_populate_cpu_children_remove_dup_nodes_set_backward_stacktracesrrs r" _build_treezEventList._build_tree's. ##%   &&(r#c"|jSN)tabler(s r"__str__zEventList.__str__-szz|r#c t}tt|D]}||j||jj||jk(s=t||jj dk(sc||j ||j_||j ||j_||j D]}||j|_|j|t|dk(ryt|Dcgc] \}}||vs |}}}|j|j|Rcc}}w)Nr) setrangelen cpu_parentname cpu_childrenkernelsadd enumerateclearextend)r to_deleteidxchindevnew_evtss r"r&zEventList._remove_dup_nodes0s2ISY' 'I((4S ,,11T#Y^^CDI00==>!C8z2EventList._populate_cpu_children..asu||U]]&Cr#c\|jj|jj gSr+) time_rangestartendrFs r"rHz2EventList._populate_cpu_children..ss&5#3#3#9#9E.bw_parents*{a  00r#)stack sequence_nrrB fwd_thread)r fwd_stacksrWtprds @r"r'z#EventList._set_backward_stacktracess 1  .C~%#))*?__cjj1J&$'IIJqM  .  #C#A}||///]]ALL1 ? *1 CI "CI #r#c&td|DS)Nc34K|]}|jywr+)self_cpu_time_total.0rGs r" z0EventList.self_cpu_time_total..s?5,,?)sumr(s r"rmzEventList.self_cpu_time_totals?$???r#c Vt||||||||j|j|| S)aPrint an EventList as a nicely formatted table. Args: sort_by (str, optional): Attribute used to sort entries. By default they are printed in the same order as they were registered. Valid keys include: ``cpu_time``, ``cuda_time``, ``xpu_time``, ``cpu_time_total``, ``cuda_time_total``, ``xpu_time_total``, ``cpu_memory_usage``, ``cuda_memory_usage``, ``xpu_memory_usage``, ``self_cpu_memory_usage``, ``self_cuda_memory_usage``, ``self_xpu_memory_usage``, ``count``. top_level_events_only(bool, optional): Boolean flag to determine the selection of events to display. If true, the profiler will only display events at top level like top-level invocation of python `lstm`, python `add` or other functions, nested events like low-level cpu/cuda/xpu ops events are omitted for profiler result readability. time_unit(str, optional): A time unit to be used for all values in the table. Valid options are: ``s``, ``ms`` and ``us``. Returns: A string containing the table. ) sort_by row_limitmax_src_column_widthmax_name_column_widthmax_shapes_column_widthheaderrrtop_level_events_only time_unit) _build_tablerr) rrtrurvrwrxryrzr{s r"r,zEventList.tables?@ !5"7$;//''"7  r#c^ddl}|jsdn |j}t|d5}d}|jd|D]}|j|jdj |j|j j|j j|js |jnd|jd|jd |jD]P}|jd |jd |j jd |jd |d|d |dz }Rt|dkDr=|j|jdz |j |j#|jddddy#1swYyxYw)zExport an EventList as a Chrome tracing tools file. The checkpoint can be later loaded and inspected under ``chrome://tracing`` URL. Args: path (str): Path where the trace will be written. rNcudaw[zc{{"name": "{}", "ph": "X", "ts": {}, "dur": {}, "tid": {}, "pid": "CPU functions", "args": {{}}}}, z " node_id:z , thread_id:z "z {"name": "z", "ph": "s", "ts": z , "tid": z , "pid": "CPU functions", "id": z, "cat": "cpu_to_z", "args": {}}, r/])osropenwrite trace_nameformatrJrK elapsed_us is_remoterBrEr6r2seektellSEEK_SETtruncate)rpathr device_namefnext_idrW_s r"export_chrome_tracezEventList.export_chrome_traces $($4$4f$:J:J $_+ G GGCL! !>>)' (.v,,113"}} )#++l3::,bQ ( !AGG%cnn%56!!$!5!5 67""%**.!!( ***57( ( qLG!'! !D4y1}qvvx!|R[[1 GGCLW+ + + s E/F##F,c gdS)N)rmself_cuda_time_totalself_xpu_time_totalself_privateuse1_time_totalr(s r"supported_export_stacks_metricsz)EventList.supported_export_stacks_metrics s   r#rmetricc ||jvr%tdt|jztjdd}t |d5}|D]}|j st |j dkDs)t||jddjddjd d}t|dkDstd }t|j D]}||j|z }|d z }|dd d ztt|z}|j|dz dddy#1swYyxYw)Nzmetric should be one of: z ; ____rrr~devicexpu privateuse1;rM  ) r ValueErrorstr maketransrrer2getattrreplaceintreversed translater) rrrtranslate_tablerrW metric_value stack_strentrys r" export_stackszEventList.export_stackss> ==? ?+d::<=> --&9 $_ 2 299SYY!!3#*vx8 1 9 $L <(1,$& %-cii%8-E%)III%,I-%.crNS$83s.get_keyGsEIIEMM"E%%&EOO$E,,- C& 3,,-$ 3u1123!#u{{#4$455: r#rrrNr)rrrrrr7r valuesrrrrerr)rrrrstatsravg_listrWs @r" key_averageszEventList.key_averages-s.9DEU9V  38_ $ C .0@BX c#h   LLN''//''    'C "3#34CI(#% )$&!  ' r#cNt}|D]}||z }d|_d|_|S)zVAverages all events. Returns: A FunctionEventAvg object. NTotal)rrC)r total_statrWs r" total_averagezEventList.total_averagens; &'  "C # J!JN "! r#)NdK7PNFN)FrF)__name__ __module__ __qualname____doc__rr)r-r&r%r'propertyrmr,rrrrrr __classcell__)r!s@r"r r s1& "(C-J#4@@  "#, \6p 2#2s24$$ ?B r#r cNd}d}||k\r ||z ddS||k\r ||z ddS|ddS)+Define how to format time in FunctionEvent..A@@.3fsmsusr)time_us US_IN_SECONDUS_IN_MSs r" _format_timer|sU"LH,L(-Q//(H$S),,c]" r#cD|dk(r|dk(s Jd|y|dz|z ddS)rrzExpected time_us == 0 but got NaNgY@.2f%r)r total_time_uss r"_format_time_sharers?!|G=gYGG|o -c 2! 44r#cd}d|z}d|z}t||k\r |dz|z ddSt||k\r |dz|z ddSt||k\r |dz|z ddSt|dzS)z&Return a formatted memory size string.i?rz GBz MBz KBz B)absr)nbytesKBMBGBs r"_format_memoryrs B B B 6{b3,#C(,, V 3,#C(,, V 3,#C(,,6{T!!r#c tfdS)Nc.tt|Sr+)rr)rr4s r"rHz!_attr_formatter..sgdD.A!Br#)rr4s`r"_attr_formatterrs B CCr#ceZdZdZedZedZedZedZedZ edZ e dZ e d Z e ed e d Zy )r z{Helpers for FunctionEvent and FunctionEventAvg. The subclass should define `*_time_total` and `count` attributes. cpu_time device_timecpu_time_totaldevice_time_totalrmself_device_time_totalc\|jdk(rdSd|jz|jz SNrgr)countrr(s r"rzFormattedTimesMixin.cpu_times+jjAosQ31D1D+Dtzz+QQr#c\|jdk(rdSd|jz|jz Sr)rrr(s r"rzFormattedTimesMixin.device_times+jjAosT31G1G+G$**+TTr#z<`cuda_time` is deprecated, please use `device_time` instead.categoryc|jSr+)rr(s r" cuda_timezFormattedTimesMixin.cuda_times r#N)rrrrr cpu_time_strdevice_time_strcpu_time_total_strdevice_time_total_strself_cpu_time_total_strself_device_time_total_strrrrr FutureWarningrrr#r"r r s #:.L%m4O()9:+,?@-.CD!01I!J RRUUF   r#r ceZdZdZdZy)r c ||_||_yr+)rKrL)rrKrLs r"rzInterval.__init__s r#c4|j|jz S)z4 Returns the length of the interval )rLrKr(s r"rzInterval.elapsed_ussxx$**$$r#N)rrrrrrr#r"r r s %r#r r )r4rdurationc\eZdZdZddddddddddddej ddddddddfdZdZdZd Z e d Z e d Z e e d e dZe dZe dZe dZe e de dZe dZe e de dZe dZdZy)rz.Profiling information about a single function.NrFrMc||_||_||_||_||_t |||_||_||_g|_ d|_ g|_ d|_ ||_ ||_||_| |_| |_| |_| |_| |_||_||_||_||_||_||n||_||_||_||_d|_d|_d|_ y)Nr/rM)!idrEr4rrr rJrBrgr6rr5r3rconcrete_inputskwinputsrercrcpu_memory_usagedevice_memory_usagerNrrfrO device_indexdevice_resource_idrflopsrself_cpu_percenttotal_cpu_percenttotal_device_percent)rr r4rBstart_usend_usrrgrrercrr rrNrrfrErOrrrrrr r rs r"rzFunctionEvent.__init__s:#  "/)$,Xv$>! )3%'  1337-9*9(0    )3%5(; & ( +'2!-(0F6H  )$) 2D "!#$&!r#c|jtjk(sJ|jj t |||yr+)rOr rPr6rVr )rr4rrs r" append_kernelzFunctionEvent.append_kernels5:>>111 F4:;r#c|jtjk(sJt|tsJ|jtjk(sJ|j j |y)zAppend a CPU child of type FunctionEvent. One is supposed to append only direct children to the event to have correct self cpu time being reported. N)rOr rP isinstancerr5rV)rchilds r"rTzFunctionEvent.append_cpu_childsX :>>111%///  JNN222   'r#c|jtjk(sJt|tsJ|jtjk(sJ||_y)a$Set the immediate CPU parent of type FunctionEvent. One profiling FunctionEvent should have only one CPU parent such that the child's range interval is completely inside the parent's. We use this connection to determine the event is from top-level op or not. N)rOr rPrrr3)rr_s r"rUzFunctionEvent.set_cpu_parent!sK:>>111&-000!!Z^^333 r#c|js|jtjk7ry|jt d|j Dz S)Nrc34K|]}|jywr+)r rors r"rpz6FunctionEvent.self_cpu_memory_usage..3s+ ',E " "+ rq)rNrOr rPr rrr5r(s r"self_cpu_memory_usagez#FunctionEvent.self_cpu_memory_usage/sJ ==D,, >$$s+ 040A0A+ (   r#c|js|jtjk7ry|jt d|j Dz S)Nrc34K|]}|jywr+)rrs r"rpz9FunctionEvent.self_device_memory_usage..;s. */E % %. rq)rNrOr rPrrrr5r(s r"self_device_memory_usagez&FunctionEvent.self_device_memory_usage7sJ ==D,, >''#. 373D3D. +   r#zO`self_cuda_memory_usage` is deprecated. Use `self_device_memory_usage` instead.rc|jSr+r#r(s r"self_cuda_memory_usagez$FunctionEvent.self_cuda_memory_usage?s ,,,r#cr|jtjk(r|jj Syr)rOr rPrJrr(s r"rzFunctionEvent.cpu_time_totalGs*   z~~ -??--/ /r#c|js|jtjk7ry|jt d|j Dz S)Nrc34K|]}|jywr+)rrs r"rpz4FunctionEvent.self_cpu_time_total..Rs) %*E ) rq)rNrOr rPrrrr5r(s r"rmz!FunctionEvent.self_cpu_time_totalNsJ ==D,, >""S) .2.?.?) &   r#c|js |jsy|jtjk(ra|j s9t d|jDt d|jDzSt d|jDS|jtjtjtjtjfvsJ|jjS)Nrc34K|]}|jywr+rrokinfos r"rpz2FunctionEvent.device_time_total..]De5>>Drqc34K|]}|jywr+r)ror=s r"rpz2FunctionEvent.device_time_total..]sK-/B((Krqc34K|]}|jywr+r,r-s r"rpz2FunctionEvent.device_time_total..br/rq)rNrrOr rPrrrr6r5CUDA PrivateUse1MTIAHPUrJrr(s r"rzFunctionEvent.device_time_totalVs ==   z~~ ->>Dt||DDsK373D3DKH Dt||DDD##&& (  ??--/ /r#zA`cuda_time_total` is deprecated. Use `device_time_total` instead.c|jSr+r1r(s r"cuda_time_totalzFunctionEvent.cuda_time_totalls %%%r#cp|js |jsy|jtjk(r)|j t d|jDz S|jtjtjtjtjfvsJ|j S)Nrc34K|]}|jywr+r1rs r"rpz7FunctionEvent.self_device_time_total..ys0,1''0rq) rNrrOr rPrrrr5r3r4r5r6r(s r"rz$FunctionEvent.self_device_time_totalts ==   z~~ -))C0595F5F0- ##&& (  )) )r#zK`self_cuda_time_total` is deprecated. Use `self_device_time_total` instead.c|jSr+rr(s r"rz"FunctionEvent.self_cuda_time_totals ***r#c|jSr+rr(s r"rCzFunctionEvent.keys yyr#c |j}|j}|j}djgd|jd|j d|j d|jd|jd|jd|jjd |jjd t|jDcgc]}|jc}d |d |d|j d |jdt|j d|j"d |d|d|j$d|j&d|j(d|j*dScc}w)Nrz)rrrjoinr r4rrOrErrJrKrLrr5rBrr rNrrfr)rrrrrs r"__repr__zFunctionEvent.__repr__soo ** "66 y y  y  y y { y/ y$J\J\I] y^ y++, y,5 y6:ll^ yCM yNRN_N_M` ya y--. y.6 y7;7J7J6K yL y t7H7H Ie IJK yLM yNYM yZ` yal_l ym yII;  y'  y(,{{m  y4B  yCFdFWFWBXAY  yZ  y !% 5 56  y 78  y 9D}  y ES  y TgRg  y h  y    y '2  y 37..1A  y BJ  y KOJZJZI[  y \g  y hlgugufv  y wx  y !JsF )rrrrr rPrrrTrUrr r#rrr&rrmrr8rrrCrFrr#r"rrse8NN 7>'@< ( !    Y-  -   00*K&  &** U+  +  r#rc*eZdZdZddZdZdZdZy)rz:Used to average stats over multiple FunctionEvent objects.Ncbd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ d|_ d|_ d|_ d|_d|_d|_d|_d|_d|_t(j*|_d|_d|_y)NrF)rCrrErNrrrrrmrrrrercr rr r#r5r3r rPrOrrr(s r"rzFunctionEventAvg.__init__s"&  # $)-#$&'() +,#7;,0%) $( %&() *+"-.%;?37'1~~$ r#c"|j|j|_|j|_|j|_|j|_|j|_|j |_|j |_|j|_|j|_|j|_ |j|_ |j|_ |j|_ |j|_ t|tt fsJ|j|jk(sJ|xj"|j"z c_|xj$|j$z c_|xj&|j&z c_|xj(|j(z c_|xj*|j*z c_|xj,|j,z c_|xj.|j.z c_|xj0|j0z c_|xj2|j2z c_|j4|j4|_|S|j4|xj4|j4z c_|Sr+)rCrErNrr3r5rrrercrOrrrrrrrrrmrr rr r#rrrothers r"r7zFunctionEventAvg.adds 88 yyDH ==DL!NNDM"__DN#..DO % 2 2D !&!4!4D  % 2 2D DJDJ$00D "__DN#..DO&+&>&>D #%-1A!BCCCyyDHH$$$ u333 %"9"99   E$=$==  ##u'C'CC# !7!77   E$=$==  ""e&A&AA" %%)G)GG% ekk! :: DJ [[ $ JJ%++ %J r#c$|j|Sr+)r7rJs r"__iadd__zFunctionEventAvg.__iadd__sxxr#c<|jsdn |j}|j}|j}|j}d|jd|j d|j d|d|d|d|dt|jd |jd|d |d S) Nr~zZ>Z=[[efjfwfwexy M(8'9;-vk]Zhilmqm~m~iiAA $ 5 56a }NS`Raab d r#)rN)rrrrrr7rMrFrr#r"rrsD2$L  r#rceZdZdZy)rcpt|dkDrtjj|n|||<||Srb)r2torch_C _demangle)rrCs r" __missing__zStringTable.__missing__s2033x!|EHH&&s+S Cyr#N)rrrrVrr#r"rrsr#rceZdZdZdZdZy)rz=Acceleration structure for accessing mem_records in interval.c||_g|_g|_t|dkDrPt t |Dcgc]\}}|dj |fc}}}t|\|_|_yycc}}wr) _mem_records _start_nses_indicesr2rQr8start_nszip)r mem_recordsirtmps r"rzMemRecordsAcc.__init__sn'&(#% { a 9[;QR41a1Q4==?A.RSC.13i +D dm RsA5 c#Ktj|j|dz}tj|j|dz}t ||D] }|j |j |"yw)z Return all records in the given interval To maintain backward compatibility, convert us to ns in function iN)bisect bisect_leftrZ bisect_rightr1rYr[)rrr start_idxend_idxr_s r" in_intervalzMemRecordsAcc.in_intervalsp &&t'7'7DI %%d&6&6 Fy'* 6A##DMM!$45 5 6sA7A9N)rrrrrrhrr#r"rrsG86r#rc4gd}tfd|DS)N))autograd/__init__ _make_grads)rjbackward)z torch/tensorrl)_internal/common_utils prof_callable)rmprof_func_call)rmprof_meth_callc3@K|]}|dvxr|dv yw)rr/Nr)rorrs r"rpz&_filter_stack_entry..s)OAaDEM3adem4Os)all)rfiltered_entriess` r"_filter_stack_entryrts O>NO OOr#z[memory]z [OutOfMemory]c.ttddddddg}||vS)Nz profiler::_record_function_enterz$profiler::_record_function_enter_newzprofiler::_record_function_exitz aten::is_leafzaten::output_nrzaten::_version)MEMORY_EVENT_NAMEOUT_OF_MEMORY_EVENT_NAME)r4filtered_out_namess r" _filter_namery s2  *.)  % %%r#cNt}||}|r|jdrd}|S)Nz ProfilerStep#z ProfilerStep*)r startswith)r4 with_wildcard string_tables r" _rewrite_namer~3s,=L  D ??? +"D Kr#c j01234t|dk(rytd|D} td|D} |dj} | s | r tdtd|D}td|D}t t |fd d | || }t d |Ddz}| t||}t d|Ddz}| t||}d}|}d}|Dcgc]4}|jt|jdkDs)|j6}}t|dkD}|r#t d|Ddz}| t||}dg}|r|jd|gdz }| | jnd}| r"|jd|d|d|d|dg|r1|jddg| r| r|j|dd|dg|jdtd|D}|r|jdd 0dg4dg10 g2d50124fd! }d"}|||r|||d#|zdD] }|| |r|jd$|||r|jd%||d&'|rj|Dcgc]}|jdkDs|j }}t|dk7r1|t|\}} |jd(| ||nd)}4d}!1d}"2d}#d}g33fd*}$d}%d}&|D]}|%|jz }%|jtj k(r|j"r|&|j$z }&K|jtj&tj(tj*fvs|j,r|&|j$z }&||$d+|#z|$|| r|$d+|#z|$d,|$|"|$|!j.||$|"d-}'d.}(d})|D]}|)|k(rn| r |j0|)d#z })|j2}*|t|*|d/z k\r |*d|d/z d0z}*t5|j|%|_|j8st5|j:|%nd|_|*g}+|r0|j>},|t|,|d/z k\r |,d|d/z d0z},|+|,gz }+|+|j6|(|j|j@| |j<|(|j:|jB| |(|jD|jF| gz }+| rt5|j$|&|_$|+j|(|j$|jJ| |jH|(|jL|jN| |(|jP|jR| g|rv|+jtU|jVtU|jXg| r;| r9|+jtU|jZtU|j\g|+j|j^|r|+j|j`|r'|+jtc|jdd||rA|jdkr|+jd1n |+j|jzd2|rAd}-t|jdkDr|'|jd|}-|+j|-|$|!j.|+|s=dgt|d#z z}.|jd#dD]"}/|$|!j.|.|'|/|gz$|.jd|$|!j.|.|$|"|$d3|(|%tg|%| | r1|$d| | jndd4|(|&tg|&| dji3Scc}wcc}w)6zUPrint a summary of events (which can be a list of FunctionEvent or FunctionEventAvg).rrc3:K|]}|jdkDywrNr<rns r"rpz_build_table..MsOu%66:Oc3:K|]}|jdkDywrr%rns r"rpz_build_table..NsP77!;Prz9use_device is None, but there is device performance data.c3lK|],}|jduxrt|jdkD.ywr)rr2rns r"rpz_build_table..Vs;    4 ' GC0B0B,Ca,G G24c3lK|],}|jduxrt|jdkD.ywr)rr2rns r"rpz_build_table..[s;    D ( IS1D1D-E-I IrNc|t|jddjddjddS)Nr~rrr)rr)rWrts r"rHz_build_table..ds5OOFH5WUH-W]H5 !r#T)rCreverserc3FK|]}t|jywr+)r2rCrorWs r"rpz_build_table..qs;SCL;s!c3XK|]"}tt|j$ywr+)r2rrrs r"rpz_build_table..us KSc#c&6&6"78Ks(* c3@K|]}td|Dyw)c32K|]}t|ywr+r2)rors r"rpz)_build_table...s25CJ2sN)max)rores r"rpz_build_table..sGu2E22GsNamez Overload Name)z Self CPU %zSelf CPUz CPU total %z CPU totalz CPU time avgNonezSelf z %z totalz time avgzCPU Memz Self CPU Memz Memz # of Callsc3:K|]}|jdk7yw)rMN)rErs r"rpz_build_table..s=s*=rzNode IDrcdxxd|zt|zdzdzzz cc<dxxd|zdzzz cc<dxx|zz cc<y)Nrz{: }r-)r)paddingtext_dir SPACING_SIZEheader_sep_lstline_length_lstrow_format_lsts r" add_columnz _build_table..add_columnshq H s7| +c 1S<5G H  qS7]cL.@AAg 44r#c $gd}|dkDsJtdttj|dz t t |dz }|dk\r|t |ksJt dtj|dz|t|fS)N)FLOPsKFLOPsMFLOPsGFLOPsTFLOPsPFLOPsrr/ g) rminmathlog10floatr2powfloorr)r flop_headers log_flopss r"auto_scale_flopsz&_build_table..auto_scale_flopss qyy3tzz%014eC .appends a dr#=z1This report only display top-level ops statisticsctt||kDr)t||z }||d}t|dkDrd|ddz}|S)Nr...r)rsrc_column_widthoffsets r" trim_pathz_build_table..trim_pathsI t9' 'Y!11F=D4y1}tABx' r#c\d}d}|dk(r ||z ddS|dk(r ||z ddS|dk(r|ddS|S)Nrrrrrrr)r default_strr{rrs r"override_time_unitz(_build_table..override_time_unitsc&    ,S13 3 $ (-R0 0 $ c]"% % r#rrz--z8.3fzSelf CPU time total: z time total: )rD)5r2anyr RuntimeErrorr rQrrrerVupperr:rrmrOr rPrrr3r4r5rrr3rCrrrNrrrrrrrrrrrrrrr r rr#rrErrrrE)5rYrtryrurvrwrxrrrzr{has_device_timehas_device_memrhas_input_shapeshas_overload_namesname_column_widthshapes_column_widthDEFAULT_COLUMN_WIDTHflops_column_widthrrWstacks has_stackheadersrappend_node_idrrr raw_flops flops_scale flops_header row_format header_sep line_lengthrVsum_self_cpu_time_totalsum_self_device_time_totalrr event_limitr4 row_valuesr src_field empty_headersrrrrrrs5 ` @@@@@r"r|r|<s2  6{aOOOOPPPN%%J /VWW     ")!  ;F;;a?( 13HIKFKKaO*!"57NO-#syy'<SYYRSAS FF aI GG G! K  +"#35IJ hG' G)3(>*""$FK }% }B'-v&-y)     . NN"m4(K=-   NN< =f==Ny!LTNTN$}oO55 W !$% Q++- .)'()~&&'()#c2*0B3CIIMSYYB B y>Q *:3y>*J '[, NNVL>2 3 ) *J"J"J!!$KJF !"E3#:#:: ??jnn , &#*D*D D & OO&&  ** '#*D*D D &E$s[ !vs[ !BC : :  g &' : Kp6 ) #  S^^%?  1 Kww ,T>SVW>W1W50146>D1  # #%<  << s113J K  V --M%1 &*?!*CC -.K1F1J Lu T =/ )J  '')D)Di   ! ! ""C$:$:I   c..     '9**,F(C $   &2266! ,,&--s/H/H)')<)'>?&s'C'CD   II     ckk *    c#"2"234H5HI J yyA~!!$'!!SYY%`+`+`0*`0)F) NNrrrrFFFN)$rcrRr collectionsrroperatorrtypingrrtyping_extensionsrrStorch.autogradr __all__listr rrrrr r r rrrrrtrvrwryr~r|rr#r"rs  / ( % __D 5 "D  < % % H< =N 'N bN *N b+66, P* &&  cr#