L iddlZddlZddlZddlmZddlmZmZmZddl Z ddl Z ddl m Z ddl Zddl mZddlmZmZmZddlmZmZdd lmZdd lmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$ejJe&Z'e jPjRGd d eZ*e jPjRGd deZ+e jPjRGddeZ,e jPjRGddZ-e jPjRGddZ.e jPjRGddZ/GddZ0y)N)partial)AnyOptionalUnion)lax) FLAX_MODEL_FOR_CAUSAL_LM_MAPPING+FLAX_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING#FLAX_MODEL_FOR_VISION_2_SEQ_MAPPING) ModelOutputlogging)GenerationConfig) !FlaxForcedBOSTokenLogitsProcessor!FlaxForcedEOSTokenLogitsProcessorFlaxForceTokensLogitsProcessorFlaxLogitsProcessorListFlaxMinLengthLogitsProcessor FlaxNoRepeatNGramLogitsProcessor(FlaxSuppressTokensAtBeginLogitsProcessor!FlaxSuppressTokensLogitsProcessorFlaxTemperatureLogitsWarperFlaxTopKLogitsWarperFlaxTopPLogitsWarperc:eZdZUdZdZeejed<y)FlaxGreedySearchOutputz Flax Base class for outputs of decoder-only generation models using greedy search. Args: sequences (`jnp.ndarray` of shape `(batch_size, max_length)`): The generated sequences. N sequences __name__ __module__ __qualname____doc__rrjnpndarray__annotations__h/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/transformers/generation/flax_utils.pyrr7(,Ix $+r'rc:eZdZUdZdZeejed<y)FlaxSampleOutputz Flax Base class for outputs of decoder-only generation models using sampling. Args: sequences (`jnp.ndarray` of shape `(batch_size, max_length)`): The generated sequences. Nrrr&r'r(r+r+Er)r'r+cbeZdZUdZdZeejed<dZ eejed<y)FlaxBeamSearchOutputaT Flax Base class for outputs of decoder-only generation models using greedy search. Args: sequences (`jnp.ndarray` of shape `(batch_size, max_length)`): The generated sequences. scores (`jnp.ndarray` of shape `(batch_size,)`): The scores (log probabilities) of the generated sequences. Nrscores) rr r!r"rrr#r$r%r.r&r'r(r-r-Ss/ (,Ix $+$(FHS[[ !(r'r-ceZdZUejed<ejed<ejed<ejed<eeejfed<y) GreedyStatecur_lenr running_tokenis_sent_finished model_kwargsNrr r!r#r$r%dictstrr&r'r(r0r0ds> [[{{;;kk!sCKK'((r'r0ceZdZUejed<ejed<ejed<ejed<ejed<eeejfed<y) SampleStater1rr2r3prng_keyr4Nr5r&r'r(r9r9msH [[{{;;kk!kksCKK'((r'r9ceZdZUejed<ejed<ejed<ejed<ejed<ejed<eeejfed<y) BeamSearchStater1running_sequencesrunning_scoresrr.r3r4Nr5r&r'r(r<r<wsR [[{{"KK{{ KKkk!sCKK'((r'r<ceZdZdZdZedZdZ d(dede ede ed e e e e jfd e jf d Z d)de ede ed efd Zed ZdZdZd e e effdZ d*de jde ede e jdede e e e jfde ef dZded efdZdedede ed efdZdeded efdZ d+ddde ede ed e ede edede e e e jfd e e e e jffd!Z d,ddde ede ed e ede e jde ed"e edede e e e jfd e e e e jffd#Z d-ddde ede ed e ed$e ed%e eee fde edede e e e jfd&e ed e e e e jffd'Z y).FlaxGenerationMixina A class containing all functions for auto-regressive text generation, to be used as a mixin in [`FlaxPreTrainedModel`]. The class exposes [`~generation.FlaxGenerationMixin.generate`], which can be used for: - *greedy decoding* by calling [`~generation.FlaxGenerationMixin._greedy_search`] if `num_beams=1` and `do_sample=False` - *multinomial sampling* by calling [`~generation.FlaxGenerationMixin._sample`] if `num_beams=1` and `do_sample=True` - *beam-search decoding* by calling [`~generation.FlaxGenerationMixin._beam_search`] if `num_beams>1` and `do_sample=False` You do not need to call any of the above methods directly. Pass custom parameter values to 'generate' instead. To learn more about decoding strategies refer to the [text generation strategies guide](../generation_strategies). ctd)NzbA model class needs to define a `prepare_inputs_for_generation` method in order to use `generate`.)NotImplementedError)selfargskwargss r(prepare_inputs_for_generationz1FlaxGenerationMixin.prepare_inputs_for_generations! p  r'c<|}||r||}||r|S)zc Run generation in untraced mode. This should only be used for debugging purposes. r&)cond_fnbody_fn init_statestates r(_run_loop_in_debugz&FlaxGenerationMixin._run_loop_in_debugs' enENEen r'c|jDcic]*\}}|jds|jds||,}}}|j|f|dd||d<|Scc}}w)Ndecoder_ cross_attnT)params return_dictencoder_outputs)items startswithencode)rC input_idsrPr4argumentvalueencoder_kwargss r(._prepare_encoder_decoder_kwargs_for_generationzBFlaxGenerationMixin._prepare_encoder_decoder_kwargs_for_generations~$0#5#5#7 %'' 3x7J7J<7X eO  +6$++i*s\`*sdr*s &'  s/A!N batch_sizedecoder_start_token_id bos_token_idr4returnc|d|vr|jd}||S|j||}tj|dj ddj |dS)Ndecoder_input_idsi4dtyperraxis)pop_get_decoder_start_token_idr#arrayreshaperepeat)rCr[r\r]r4r`s r()_prepare_decoder_input_ids_for_generationz=FlaxGenerationMixin._prepare_decoder_input_ids_for_generationsv  #(;|(K , 0 01D E  ,((!%!A!ABXZf!gyy/t<DDQKRRS]deRffr'ch||n|jj}||n|jj}||St|jdr`t|jj dr@|jj j |jj jS||St|jdr`t|jj dr@|jj j |jj jSt d)Ndecoderr\r]z\`decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation.)generation_configr\r]hasattrconfigrn ValueError)rCr\r]s r(rhz/FlaxGenerationMixin._get_decoder_start_token_ids&1 #''>>  (4'?|TE[E[EhEh ! -) ) DKK + ++-EF ##::F;;&&== =  %  DKK + ++^< ##00<;;&&33 3 j  r'cztj|dddf|jd|f|jddzS)Nrr)r# broadcast_toshape)tensor num_beamss r(_expand_to_num_beamsz(FlaxGenerationMixin._expand_to_num_beamss?q$w&,,q/91MPVP\P\]^]_P`1`aar'c|S)z This function can be overwritten in the specific modeling_flax_.py classes to allow for custom beam search behavior. Note that the only model that overwrites this method is [`~transformers.FlaxMarianMTModel`]. r&)rClogitss r(_adapt_logits_for_beam_searchz1FlaxGenerationMixin._adapt_logits_for_beam_searchs  r'cN|jstttg}t }|D]F}|j t |jd}|,|j|jHd|jjd}|r|d|z }t|y)z Confirms that the model class is compatible with generation. If not, raises an exception that points to the right class to use. N)defaultzThe current model class (zQ) is not compatible with `.generate()`, as it doesn't have a language model head.z2 Please use one of the following classes instead: ) can_generater r r setgettyperqaddr __class__ TypeError)rCgenerate_compatible_mappingsgenerate_compatible_classes model_mappingsupported_modelsexception_messages r(_validate_model_classz)FlaxGenerationMixin._validate_model_classs   "03;, ( +.% '!= O #0#4#4T$++5FPT#4#U #//334D4M4MN O ,DNN,C,C+DE99 +!'YZuYv%ww!-. .##r'cjg}ttj|jj}d|vsd|vr5|ttj|j jz}|j D]\}}| ||vs|j| |rtd|dy)zXValidates model kwargs for generation. Generate argument typos will also be caught here.rEr4Nz8The following `model_kwargs` are not used by the model: zG (note: typos in the generate arguments will also show up in this list)) rinspect signaturerF parameters__call__rSappendrr)rCr4unused_model_args model_argskeyrXs r(_validate_model_kwargsz*FlaxGenerationMixin._validate_model_kwargss**4+M+MNYYZ  z !^z%A #g// >IIJ JJ&,,. .JC S %:!((- . JK\J]^FF  r'rVror:tracerPlogits_processorc X |j||jjrv|jjt |jk(rJt j |j}||jk7rtjd||_|j}tj|}|jd&i|} |j| j||n t}||ntj j#d}|j$Z|j&N| j)dt*j-d|j&} t/| t0r| d} | |_|j2!|jj4r t7d|jj4sO|sM|j$At9j:|dddf|j$k(dkDrt*j-d|j<d} |jj4rM| j)d |j?||| } |jA| |j2|jB| }|j<d} |j)d duxr|jDdu} | rD|jF8|jDd k(r)tjd |jDdtHn|jFS| s<|jD0t*j-d|jFd|jDd|jF| z|_"ni|jDt jDk(rH|jD| z|_"tK|jdd}|tM|jD||_"|jN?|jN|jDkDr&t7d|jNd|jDd| |jDk\rC|jj4rdnd}t*j-d|d| d|jDd|jQ|| |}|jRsF|jTdk(r7|jW||jD|j$|j&|||| S|jRrZ|jTdk(rK|jY|}|j[||jD|j$|j&|||||| S|jRs|jTdkDr|j]||jT!}d | vr)|j]| d d"|jT!| d d"<d#D]*}|| vs|j]| ||jT!| |<,|j_||jD|j$|j&|j`|jb||||jd| $ Stgd%)'a Generates sequences of token ids for models with a language modeling head. Parameters: input_ids (`jnp.ndarray` of shape `(batch_size, sequence_length)`): The sequence used as a prompt for the generation. generation_config (`~generation.GenerationConfig`, *optional*): The generation configuration to be used as base parametrization for the generation call. `**kwargs` passed to generate matching the attributes of `generation_config` will override them. If `generation_config` is not provided, the default will be used, which had the following loading priority: 1) from the `generation_config.json` model file, if it exists; 2) from the model configuration. Please note that unspecified parameters will inherit [`~generation.GenerationConfig`]'s default values, whose documentation should be checked to parameterize generation. trace (`bool`, *optional*, defaults to `True`): Whether to trace generation. Setting `trace=False` should only be used for debugging and will lead to a considerably slower runtime. params (`dict[str, jnp.ndarray]`, *optional*): Optionally the model parameters can be passed. Can be useful for parallelized generation. logits_processor (`FlaxLogitsProcessorList `, *optional*): Custom logits processors that complement the default logits processors built from arguments and generation config. If a logit processor is passed that is already created with the arguments or a generation config an error is thrown. This feature is intended for advanced users. kwargs (`dict[str, Any]`, *optional*): Ad hoc parametrization of `generate_config` and/or additional model-specific kwargs that will be forwarded to the `forward` function of the model. If the model is an encoder-decoder model, encoder specific kwargs should not be prefixed and decoder specific kwargs should be prefixed with *decoder_*. Return: [`~utils.ModelOutput`]. NaSYou have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use and modify the model generation configuration (see https://huggingface.co/docs/transformers/generation_strategies#default-text-generation-configuration )rattention_maskzThe attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.zJ`decoder_start_token_id` has to be defined for encoder-decoder generation.rdzA decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.rR)r\r]r4 max_lengthz0Using the model-agnostic default `max_length` (=zx) to control the generation length. recommend setting `max_new_tokens` to control the maximum length of the generation.zBoth `max_new_tokens` (=z) and `max_length`(=z) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)max_position_embeddingsz3Unfeasable length constraints: the minimum length (z%) is larger than the maximum length ()r`rVzInput length of z is z, but `max_length` is set to zW. This can lead to unexpected behavior. You should consider increasing`max_new_tokens`.)roinput_ids_seq_lengthrr)rrrPr4)ro) logits_warperrrrPr4)rwlast_hidden_staterdecoder_attention_mask)length_penaltyearly_stoppingrrrPnum_return_sequencesr4z,`Beam sampling is currently not implemented.r&)4rro_from_model_config_original_object_hashhashrfrom_model_configrqwarningswarncopydeepcopyupdaterrjaxrandomPRNGKey pad_token_id eos_token_idrloggerwarning isinstancelistr\is_encoder_decoderrrr#sumrurZrlr]rmax_new_tokens UserWarninggetattrmin min_length_get_logits_processor do_samplerw_greedy_search_get_logits_warper_samplerx _beam_searchrrrrB)rCrVror:rrPrrEnew_generation_configr4rr[rhas_default_max_lengthrinput_ids_stringrkwargs r(generatezFlaxGenerationMixin.generatesxT ""$  $ %%88T=S=S=i=imq&&n>)9(J(J4;;(W%(D,B,BBMMB .CD* $ 6 6  MM*;</(//9&9  ##L$5$5$78/?/K+QhQj (389K9KA9N  ) ) 16G6T6T6` 019q-99L,-+A -9  *  3 3 ; @^@^ij j{{--e!..:GGIae,0A0N0NNORSSl __Q' ;; ) ) 12:#RRS\^dfrs FF'8'O'O.;;) GI )r2!'L!9T!A!nFWFbFbjnFn !&7&F&F&NSdSoSosuSu MMBCTC_C_B`aII   - - 9).?.J.J.V./@/O/O.PPd(3345ff ,=+K+KNb+b  ( ++/?/A/L/LL/@/K/KNb/b!,*1$++?XZ^*_'*6367H7S7SUl3m%0  ' ' 38I8T8TWhWsWs8sEFWFbFbEcd((9(D(D'EQH  #4#?#? ?6:kk6T6T2Ze  NN"#3"4D9M8NO%0012//   55/!5-6  !**/@/J/Ja/O&&!,,!..!..!1)'   ( (->-H-HA-M 33FW3XM<<!,,!..!..+!1)  #,,1B1L1Lq1P11)GXGbGb1cI L0GKG`G` !234GHTeToToHaH ./0CDF L(*.*C*C$U+7H7R7R+D+L'  $$!,,!..!..0??0??!1%6%K%K)%  &&TU Ur'ct}|j3|jdk7r$|jt|j|j5|jdk7r&|jt |jd|j 5|j dkr&|jt|j d|S)z This class returns a [`FlaxLogitsProcessorList`] list object that contains all relevant [`FlaxLogitsWarper`] instances used for multinomial sampling. g?rr)top_kmin_tokens_to_keep)top_pr)r temperaturerrrrrr)rCrowarperss r(rz&FlaxGenerationMixin._get_logits_warpers *+  ( ( 49J9V9VZ]9] NN67H7T7TU V  " " .3D3J3Ja3O NN/6G6M6Mbcd e  " " .3D3J3JS3P NN/6G6M6Mbcd er'rcNt}|jJ|j>|jdkDr/|jt |j|j|j $|jt |j |j/|jt|j|j|j$|jt|j|jy|}|dkDs |j |n|dz}t|dd-t|jdkDr||jddz }|jt!|j|t|ddA|jDcgc]}||dzdz |dg}}|jt#||j$3|j$dkDr$|jt'|j$|j)||}|Scc}w)z This class returns a [`FlaxLogitsProcessorList`] list object that contains all relevant [`FlaxLogitsProcessor`] instances used to modify the scores of the language model head. Nrdrforced_decoder_idsr)rrrrrforced_bos_token_idrforced_eos_token_idrrsuppress_tokensrbegin_suppress_tokensrlenrrrno_repeat_ngram_sizer_merge_criteria_processor_list)rCrorr processors begin_indexirs r(rz)FlaxGenerationMixin._get_logits_processors1-.   ( ( 4!..:!,,r1   ,->-I-IK\KiKij   0 0 <   ?@Q@e@ef g  0 0 <   12C2N2NPaPuPuv   , , 8   ?@Q@a@ab c  2 2 >.K)1,0A0U0U0] 1_  )+?FR)<<=A0CCBGJJ   89J9`9`bmn  $&:D A MARAeAe"<=%!,q0!A$7" "   <=OP Q  1 1 =BSBhBhklBl   >?P?e?ef g88EUV "sH" default_list custom_listct|dk(r|S|D]K}|D]D}t|t|usd}td|dt|d|d|d|d|d |d M|j||S) Nrzlogits processorz A custom z of type z with values zP has been passed to `generate`, but it has already been created with the values z. z has been created by passing the corresponding arguments to generate or by the model's config default values. If you just want to change the default values of zL consider passing them as arguments to `generate` instead of using a custom .)rrrrextend)rCrrr}custom object_types r(rz2FlaxGenerationMixin._merge_criteria_processor_list*s { q  # G% <4=0"4K$#K= $v,}U[T\]XX_W``bcjbklUU`TabVVaUbbc e   K(r'rrrc njjnjjnjj|j\} } t j t jndt j t jt j | } t j| ft j} tj| |d} t j| ft j} jjr jnj |fi|}t#| | || |} fd}fd}|jddkDr|| } |sj%||| } ntj&||| } t)| j*S)Nrbrrr1rr2r3r4c|jk(}tj|j}tj||}|Szstate termination condition fn.r1r#allr3 logical_orrKhas_reached_max_lengthall_sequence_finishedfinish_generationrs r(greedy_search_cond_fnzAFlaxGenerationMixin._greedy_search..greedy_search_cond_fnkC%*]]j%@ "$'GGE,B,B$C ! #/EG\ ] %% %r'c |jfd i|j}|jdddf}|j||j}t j |d}||jz |jzz}|j|k(z}|dddf}tj|j|d|jf} j||j}t|jdz||||S)state update fn.rPNrdrerrr) r2r4rzrr1r#argmaxr3rdynamic_update_sliceupdate_inputs_for_generationr0) rK model_outputsrz next_tokennext_is_sent_finishednext_sequencesnext_model_kwargsrrmodelrrPrCs r(greedy_search_body_fnzAFlaxGenerationMixin._greedy_search..greedy_search_body_fnrs !%"5"5[f[HZHZ[M"))!R%0F&eoovu}}MFF4J#u'='=&== uOeOe@eeJ$)$:$:jL>X$Y !#AtG,J 55eoozTUW\WdWdSefN $ A A-QVQcQc d  )((!6.  r'rr)rorrrrur#riint32fullrrzerosbool_rqrdecoderFr0rL while_looprr)rCrVrrrrrrPr4r[r1rr3rKrrrs` ```` ` @r(rz"FlaxGenerationMixin._greedy_search?s$.#9Zt?U?U?`?` '3'?|TE[E[EhEh '3'?|TE[E[EhEh 'oo Gyy,BZSYY`de yySYY? ))G$HHj*5|399U ,,Y 6J 99j]#))D ${{== 49t99)Z`S_` #-%   &  2 ??1  !)%0E++,ACXZ_`ENN#8:OQVWE%@@r'rc  njjnjjnjj||ntj j d}|j\} } tjtjndtjtjtj| } tj| ftj} tj| |d} tj| ftj}j j"r j$nj&|fi| } t)| | |||| }fd} fd}|jddkDr||}|sj+|||}ntj,|||}t/|j0S) Nrrbr)r1rr2r3r:r4c|jk(}tj|j}tj||}|Srrrs r(sample_search_cond_fnz:FlaxGenerationMixin._sample..sample_search_cond_fnrr'ctjj|j\}} |jfdi|j }|j dddf} |j||j} |||j}tjj||d}||jz |jzz}|j| k(z}|dddf}tj|j|d|jf}j||j }t|jdz|||||S)rrPNrdrerr)r1rr2r3r4r:)rrsplitr:r2r4rzrr1 categoricalr3rrrr9)rKr: prng_key_nextrrzrrrrrrrrrrPrCs r(sample_search_body_fnz:FlaxGenerationMixin._sample..sample_search_body_fnsG&)jj&6&6u~~&F #Hm!%"5"5[f[HZHZ[M"))!R%0F&eoovu}}MF"665==AF//&r/JJ#u'='=&== uOeOe@eeJ$)$:$:jL>X$Y !#AtG,J 55eoozTUW\WdWdSefN $ A A-QVQcQc d  )((!6.&  r'rr)rorrrrrrrur#rirrrrrrrqrrrFr9rLrr+r)rCrVrrrr:rrrrPr4r[r1rr3rKrr rs` ``` `` ` @r(rzFlaxGenerationMixin._samples$.#9Zt?U?U?`?` '3'?|TE[E[EhEh '3'?|TE[E[EhEh '389K9KA9N'oo Gyy,BZSYY`de yySYY? ))G$HHj*5|399U ,,Y 6J 99j]#))D ${{== 4:t99)Z`S_` #-%   &  > ??1  !)%0E++,ACXZ_`ENN#8:OQVWE%//::r'rrrc  dddnjj||njj}njjnjjnjj | | njj } |j\} tjtjndtj|tj}tj| } |jdtjf|tj} tjf|tj}tj| |d}tjftj}tj tjdgt#jd gd z zzd g}tj$ft#jd z}j&j(r j*nd | vr| d d | d d <d D]}|| vs| || |<j,|fi| } t/| ||| ||| }fd}d f d }t1||jd|}|sj3|||}ntj4|||}tj6|j8d }tj:|ddddf|j<|j>} tj:|dddf|j@|jB}| ddd| ddf} |ddd| f}tE| |S)z This beam search function is heavily inspired by Flax's official example: https://github.com/google/flax/blob/main/examples/wmt/decode.py c|jdk(r|S|j|jd|jdzf|jddzS)z8Flattens the first two dimensions of a non-scalar array.rrrNndimrjru)rvs r(flatten_beam_dimz:FlaxGenerationMixin._beam_search..flatten_beam_dim sN{{a >>6<<?V\\!_#D"FVWVXIY"YZ Zr'cj|jdk(r|S|j||f|jddzS)zFUnflattens the first, flat batch*beam dimension of a non-scalar array.rrNr )rvr[rws r(unflatten_beam_dimz.unflatten_beam_dims9{{a >>:y"9FLL.gather_beams..gather_fns$;;!#!M!-"=>>r')r#rjaranger tree_utiltree_map)nestedrr[ new_num_beamsrrs ` @r( gather_beamsz6FlaxGenerationMixin._beam_search..gather_beamssO KK : 56-G*VcIdM ?==)))V< rr.r3r4c|j k}dk(r$dkDr|jddddf z zz }n(|jddddf|jz zz }tj|jtj |j ddtjd}tj||kD}tj|jduz}||z|zS)z+beam search state termination condition fn.neverrNrTrfkeepdimsr) r1r>r#wherer3rr.nprianyr) rKnot_max_length_yetbest_running_scoreworst_finished_scoreimprovement_still_possiblestill_open_beamdecoder_prompt_lenrrrs r(beam_search_cond_fnz=FlaxGenerationMixin._beam_search..beam_search_cond_fnbs "'!; (^c-A%*%9%9!RaR%%@"44G&"&+%9%9!RaR%%@]]%77NJ&"$'99&& 1t(TVXV^V^_eVf$ *-1CFZ1Z)[ &!$(>(> ?>UYCY Z[O%7:TT Tr'c D $tj|jdd|j|z f )|f}(|fd*i|j},|j dddf )}t jj ),fd|j}+j|}t jj|}'$|j$||j},| )}|tj|jdz}|j d}|j# )|zf}d)z}tj$||\} } | |z} %|j| |} tj| |zd} tj&| | dd|jf}|dddd|jf#k(}| |t)j*d zz}tj$|)d }%||g| )\}}| |jd z!z &zz } tj,|j.j1dd |j "d uz}||z}| |t)j*d zz } tj2|j4|gd }tj2|j6| gd }tj2|j.|gd }tj$|)d }%|||g| )\}}}%| | )}%|| )}t jj$fd ||d<+j9||j}t;|jd z||||||S)zbeam search state update fn.rrPNrdc|SNr&)rvr[rwrs r(zOFlaxGenerationMixin._beam_search..beam_search_body_fn..s1&*iPr'rre)krrTr!c|Sr/r&)xrs r(r0zOFlaxGenerationMixin._beam_search..beam_search_body_fn..sP`abPcr'past_key_values)r1r>r=r.rr3r4)r dynamic_slicer=r1r4rzrrrr4r{nn log_softmaxr# expand_dimsr>rurjrrr$rirtr3r concatenaterr.rr<)-rKinput_ids_length input_tokenrrzcache log_probs vocab_size beams_to_keeptopk_log_probs topk_indicestopk_beam_indicestopk_running_sequencestopk_idstopk_sequencesdid_topk_just_finishedrunning_topk_log_probsnext_topk_indicesnext_running_sequencesnext_running_scoresbeams_in_batch_are_full add_penaltymerged_sequences merged_scoresmerged_is_sent_finishedtopk_merged_indicesr next_scoresrnext_running_indices next_cacherr[r+rrrrrrrrwrPrCrs- r(beam_search_body_fnz=FlaxGenerationMixin._beam_search..beam_search_body_fns+!!++5==+;;<,<=K"+SfS@R@RSM' (<(,JQRSM&)oou7M7MOe6fmn&o #"%))MY"G"J AM!=2IJL_akmvB >NK)>$00ACTV`bk#l %e-A:yYJ/2}}/E/EFceo/pM+ , $ A A-QVQcQc d " )2"8"(!6. r')r:re)rr.)r)#rorrrrrrrur#rirrrrrrtiler$onesrqrrrFr<rrLrr%r3r#rr=r.r>r-)rCrVrrrrrrrrPrr4r1rr=r3r>r.rrKr,rT none_finishedr[r+rrrrwrs` ` ```` ` @@@@@@@r(rz FlaxGenerationMixin._beam_searchs& [ N =$$.#9Zt?U?U?`?` '3'?|TE[E[EhEh '3'?|TE[E[EhEh +9+E4KaKaKpKp+9+E4KaKaKpKp$8$D $J`J`JuJu *3& Iwyy,BZSYY`de yySYY? ))G$'__R0HHj)Z@,VYV_V_` HHj)Z%H,^a^g^gh44Y 9U99j)%QRSXY++,?ATV[\ENN#68KUSE 6 6Q? IImAtTM:EOOUMdMde =D15<<AUAUV%Yq2G3G2G/J%KL !&,A-A,A)A"BC#iGGr')NNN)NN)NNTNN)NNNNTNN) NNNNNNTNN) NNNNNNTNNN)!rr r!r"rF staticmethodrLrZintrr6r7r#r$rlrhrxr{rrrrboolrrrrrrrfloatrrr&r'r(r@r@sa  15&*9= g g!)  gsm g tC$456 g  g [_ &.sm JRSV-  >bb/04S>*9=*.37>BSV;;SV$$45SV3;;' SV  SV c3;;./0 SV##:;SVj4DI` 5+5"5##:; 5 ! 5n-- ! 0%)&*&*>B379=TATASMTAsm TA sm TA ##:; TATAc3;;./0TAtC$456TAr%)&*&**.>B;?379=_;_;SM_;sm _; sm _; 3;;' _;##:;_; 78_;_;c3;;./0_;tC$456_;H%)&*&**.59>B37.29=RHRHSMRHsm RH sm RH ! RH!tSy!12RH##:;RHRHc3;;./0RH'smRHtC$456RHr'r@)1rrr functoolsrtypingrrrflaxr jax.numpynumpyr#r$r models.autor r r utilsr r configuration_utilsrflax_logits_processrrrrrrrrrrr get_loggerrrstruct dataclassrr+r-r0r9r<r@r&r'r(rhsH$ ''  )1       H % ,[ , , ,{ , , ); ) ) )))))))))FHFHr'