L i[rddlZddlmZddlmZmZmZmZmZm Z m Z ddl m Z ddl Z ddl mZmZgdZe dZe dZGd d eZd eed eegefd eefdZGddeZ d*dedeedeedeed eeeeeeeff dZ d*dedeedeedeed ef dZdeed eefdZ d+dede eeefdeded ef dZ d,dededed eed eeeff d!Z d-d"e eeefdeded#e d ef d$Z! d.d%ededed eefd&Z" d/d"eeded efd'Z#d(ed eefd)Z$y)0N)Iterable)AnyCallable NamedTupleOptionaloverloadTypeVarUnion)Self)_VFTensor)PackedSequenceinvert_permutationpack_padded_sequencepad_packed_sequence pad_sequenceunpad_sequence pack_sequenceunpack_sequence_T_RceZdZUejed<ejed<eejed<eejed<y)PackedSequence_data batch_sizessorted_indicesunsorted_indicesN)__name__ __module__ __qualname__torchr __annotations__rX/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/torch/nn/utils/rnn.pyrrs4 ,,U\\**u||,,r$roptionalfnreturnc|y||SNr#)r&r's r%bindr+ s h<r$c eZdZdZ d dedeedeedeedef fd ZdefdZe d!d e jd e d e defd Z e d"d eeee j efd ee jd e d e def dZ e d!ded e d e defdZ dededefdZ dededefdZdededefdZdefdZdefdZdefdZdefdZdefdZdefdZdefdZdefdZede fdZde fdZxZS)#rapHolds the data and list of :attr:`batch_sizes` of a packed sequence. All RNN modules accept packed sequences as inputs. Note: Instances of this class should never be created manually. They are meant to be instantiated by functions like :func:`pack_padded_sequence`. Batch sizes represent the number elements at each sequence step in the batch, not the varying sequence lengths passed to :func:`pack_padded_sequence`. For instance, given data ``abc`` and ``x`` the :class:`PackedSequence` would contain data ``axbc`` with ``batch_sizes=[2,1,1]``. Attributes: data (Tensor): Tensor containing packed sequence batch_sizes (Tensor): Tensor of integers holding information about the batch size at each sequence step sorted_indices (Tensor, optional): Tensor of integers holding how this :class:`PackedSequence` is constructed from sequences. unsorted_indices (Tensor, optional): Tensor of integers holding how this to recover the original sequences with correct order. .. note:: :attr:`data` can be on arbitrary device and of arbitrary dtype. :attr:`sorted_indices` and :attr:`unsorted_indices` must be ``torch.int64`` tensors on the same device as :attr:`data`. However, :attr:`batch_sizes` should always be a CPU ``torch.int64`` tensor. This invariant is maintained throughout :class:`PackedSequence` class, and all functions that construct a :class:`PackedSequence` in PyTorch (i.e., they only pass in tensors conforming to this constraint). rrrrr(c <t||gt||||Sr*)super__new___packed_sequence_init_args)clsrrrr __class__s r%r/zPackedSequence.__new__Js0w  'k>3C  r$c t||jj|jt |j dt |j dS)Nc"|jSr* pin_memoryts r%z+PackedSequence.pin_memory..bs  r$c"|jSr*r5r7s r%r9z+PackedSequence.pin_memory..cs !,,.r$)typerr6rr+rrselfs r%r6zPackedSequence.pin_memory\sRtDz II "    $$&> ? &&(@ A   r$dtype non_blockingcopycyr*r#)r=r>r?r@s r%tozPackedSequence.tof r$devicecyr*r#)r=rDr>r?r@s r%rBzPackedSequence.tonsr$othercyr*r#)r=rFr?r@s r%rBzPackedSequence.towrCr$argskwargscJ|jj|i|jur|Sttdj t |j fd}t |jfd}t||j||S)a Perform dtype and/or device conversion on `self.data`. It has similar signature as :meth:`torch.Tensor.to`, except optional arguments like `non_blocking` and `copy` should be passed as kwargs, not args, or they will not apply to the index tensors. .. note:: If the ``self.data`` Tensor already has the correct :class:`torch.dtype` and :class:`torch.device`, then ``self`` is returned. Otherwise, returns a copy with the desired configuration. c&|ddk7xr|ddk7S)NrrDr>r#r7s r%r9z#PackedSequence.to..s1!1!Eadgor$c>|jjfiSr*rBrDr8rrIs r%r9z#PackedSequence.to..stqttDKK/J6/Jr$c>|jjfiSr*rMrNs r%r9z#PackedSequence.to..sdkk1LV1Lr$) rrBdictfilteritemsr+rrr;r)r=rHrIrrrs ` @r%rBzPackedSequence.tostyy||T,V, 499 KEv||~VF"##%JN $%%'L  4:dD$4$4nFVW Wr$c tjd|jj|jjj |i|}|j r|j |i|Sd|d<|j |i|S)Nr#r>rDcudarD)r!tensorrr>rDrBis_cudar=rHrIexs r%rUzPackedSequence.cudas} PU\\"DIIOODIIrDrBr;rXs r%r[zPackedSequence.cpus PU\\"DIIOODII>U "477D+F+ + xtww'''r$cB|jtjSNr>)rBr!doubler<s r%r_zPackedSequence.doubleswwU\\w**r$cB|jtjSr])rBr!floatr<s r%razPackedSequence.floatwwU[[w))r$cB|jtjSr])rBr!halfr<s r%rdzPackedSequence.halfwwUZZw((r$cB|jtjSr])rBr!longr<s r%rgzPackedSequence.longrer$cB|jtjSr])rBr!intr<s r%rizPackedSequence.intswwUYYw''r$cB|jtjSr])rBr!shortr<s r%rkzPackedSequence.shortrbr$cB|jtjSr])rBr!int8r<s r%charzPackedSequence.charrer$cB|jtjSr])rBr!uint8r<s r%bytezPackedSequence.byterbr$c.|jjS)z+Return true if `self.data` stored on a gpu.)rrWr<s r%rWzPackedSequence.is_cudasyy   r$c6|jjS)z6Return true if `self.data` stored on in pinned memory.)r is_pinnedr<s r%rtzPackedSequence.is_pinnedsyy""$$r$NNN)..)....) rrr __doc__r rr r/r6rr!r>boolrBr strrDrirrUr[r_rardrgrkrnrqpropertyrWrt __classcell__)r2s@r%rr&s5!L)-+/-1    f%  !(  #6*    $ D ! {{   ;>'*  sELL#567 $     !    XXsXtX>(#(((((((++*t*)d))d)(T(*t*)d)*d*!!!%4%r$rrrrrc| t|}|*|jjdk7r td||||fSt |t t frt|dk(sJ|d|d||fS)Nr[abatch_sizes should always be on CPU. Instances of PackedSequence should never be created manually. They should be instantiated by functions like pack_sequence and pack_padded_sequences in nn.utils.rnn. https://pytorch.org/docs/stable/nn.html#torch.nn.utils.rnn.pack_sequencer)rrDr; ValueError isinstancelisttuplelenrrrrs r%r0r0s-n=    " "e +[ [.2BBB$u .3t9>AAAwQ1AAAr$cDt||||\}}}}t||||Sr*)r0rrs r%_packed_sequence_initrs7 ;U k>+;;7D+~'7 $ ^=M NNr$ permutationc |ytj|tj}|jd|tjd|j |j |S)zReturns the inverse of ``permutation``. This is useful for converting between sorted and unsorted indices in a :class:`~nn.utils.rnn.PackedSequence`. Args: permutation (Tensor, optional): a 1-D tensor of indices to invert N) memory_formatrrD)r! empty_likelegacy_contiguous_formatscatter_arangenumelrD)routputs r%rrs[   k9W9W XF OO ; Q (9(9(;KDVDVW Mr$inputlengths batch_firstenforce_sortedc"t|tjs\tjj rt j ddtj|tjd}n |jtj}|rd}nMtj|d \}}|j|j}|rd nd }|j||}tj|||\}}t|||dS) aPacks a Tensor containing padded sequences of variable length. :attr:`input` can be of size ``T x B x *`` (if :attr:`batch_first` is ``False``) or ``B x T x *`` (if :attr:`batch_first` is ``True``) where ``T`` is the length of the longest sequence, ``B`` is the batch size, and ``*`` is any number of dimensions (including 0). For unsorted sequences, use `enforce_sorted = False`. If :attr:`enforce_sorted` is ``True``, the sequences should be sorted by length in a decreasing order, i.e. ``input[:,0]`` should be the longest sequence, and ``input[:,B-1]`` the shortest one. `enforce_sorted = True` is only necessary for ONNX export. It is an inverse operation to :func:`pad_packed_sequence`, and hence :func:`pad_packed_sequence` can be used to recover the underlying tensor packed in :class:`PackedSequence`. Note: This function accepts any input that has at least two dimensions. You can apply it to pack the labels, and use the output of the RNN with them to compute the loss directly. A Tensor can be retrieved from a :class:`PackedSequence` object by accessing its ``.data`` attribute. Args: input (Tensor): padded batch of variable length sequences. lengths (Tensor or list(int)): list of sequence lengths of each batch element (must be on the CPU if provided as a tensor). batch_first (bool, optional): if ``True``, the input is expected in ``B x T x *`` format, ``T x B x *`` otherwise. Default: ``False``. enforce_sorted (bool, optional): if ``True``, the input is expected to contain sequences sorted by length in a decreasing order. If ``False``, the input will get sorted unconditionally. Default: ``True``. .. warning:: The dim of ``input`` tensor will be truncated if its length larger than correspond value in ``length``. Returns: a :class:`PackedSequence` object zpack_padded_sequence has been called with a Python list of sequence lengths. The tracer cannot track the data flow of Python values, and it will treat them as constants, likely rendering the trace incorrect for any other combination of lengths.r|) stacklevelr[rTr^NT) descendingrr})rr!r _C_get_tracing_statewarningswarn as_tensorint64rBsortrD index_selectr _pack_padded_sequencer)rrrrr batch_dimrrs r%rrsX gu|| , 88 & & ( MML  //'UK**5;;*/"'**W"F'**5<<8$A! ""9n=11%+ND+ {ND IIr$sequence padding_value total_lengthcF|jjd}|||krtd|d||}tj|j |j|||\}}|j }|+|rdnd}|j||||jfS||fS)aPad a packed batch of variable length sequences. It is an inverse operation to :func:`pack_padded_sequence`. The returned Tensor's data will be of size ``T x B x *`` (if :attr:`batch_first` is ``False``) or ``B x T x *`` (if :attr:`batch_first` is ``True``) , where ``T`` is the length of the longest sequence and ``B`` is the batch size. Example: >>> from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence >>> seq = torch.tensor([[1, 2, 0], [3, 0, 0], [4, 5, 6]]) >>> lens = [2, 1, 3] >>> packed = pack_padded_sequence( ... seq, lens, batch_first=True, enforce_sorted=False ... ) >>> packed PackedSequence(data=tensor([4, 1, 3, 5, 2, 6]), batch_sizes=tensor([3, 2, 1]), sorted_indices=tensor([2, 0, 1]), unsorted_indices=tensor([1, 2, 0])) >>> seq_unpacked, lens_unpacked = pad_packed_sequence(packed, batch_first=True) >>> seq_unpacked tensor([[1, 2, 0], [3, 0, 0], [4, 5, 6]]) >>> lens_unpacked tensor([2, 1, 3]) .. note:: :attr:`total_length` is useful to implement the ``pack sequence -> recurrent network -> unpack sequence`` pattern in a :class:`~torch.nn.Module` wrapped in :class:`~torch.nn.DataParallel`. See :ref:`this FAQ section ` for details. Args: sequence (PackedSequence): batch to pad batch_first (bool, optional): if ``True``, the output will be in ``B x T x *`` format, ``T x B x *`` otherwise. padding_value (float, optional): values for padded elements. total_length (int, optional): if not ``None``, the output will be padded to have length :attr:`total_length`. This method will throw :class:`ValueError` if :attr:`total_length` is less than the max sequence length in :attr:`sequence`. Returns: Tuple of Tensor containing the padded sequence, and a Tensor containing the list of lengths of each sequence in the batch. Batch elements will be re-ordered as they were ordered originally when the batch was passed to ``pack_padded_sequence`` or ``pack_sequence``. rzgExpected total_length to be at least the length of the longest sequence in input, but got total_length=z and max sequence length being r}) rsizer~r _pad_packed_sequencerrrr[) rrrrmax_seq_length padded_outputrrrs r%rrXsn))..q1N . ( ,~-L^L\^  & 55 x++[-M7 00#$A!  & &y2B C $((* +   ' !!r$ sequences padding_sidectjjsStjjs5t |t sdt |}t|t|}n+t |tjr|jd}tjjj||||S)aOPad a list of variable length Tensors with :attr:`padding_value`. ``pad_sequence`` stacks a list of Tensors along a new dimension, and pads them to equal length. :attr:`sequences` can be list of sequences with size ``L x *``, where `L` is length of the sequence and ``*`` is any number of dimensions (including ``0``). If :attr:`batch_first` is ``False``, the output is of size ``T x B x *``, and ``B x T x *`` otherwise, where ``B`` is the batch size (the number of elements in :attr:`sequences`), ``T`` is the length of the longest sequence. Example: >>> from torch.nn.utils.rnn import pad_sequence >>> a = torch.ones(25, 300) >>> b = torch.ones(22, 300) >>> c = torch.ones(15, 300) >>> pad_sequence([a, b, c]).size() torch.Size([25, 3, 300]) Note: This function returns a Tensor of size ``T x B x *`` or ``B x T x *`` where `T` is the length of the longest sequence. This function assumes trailing dimensions and type of all the Tensors in sequences are same. Args: sequences (list[Tensor]): list of variable length sequences. batch_first (bool, optional): if ``True``, the output will be in ``B x T x *`` format, ``T x B x *`` otherwise. padding_value (float, optional): value for padded elements. Default: ``0``. padding_side (str, optional): the side to pad the sequences on. Default: ``'right'``. Returns: Tensor of size ``T x B x *`` if :attr:`batch_first` is ``False``. Tensor of size ``B x T x *`` otherwise zJpad_sequence: Expected iterable for input sequences, but got arg of type: r)r!jit is_tracing is_scriptingrrr; RuntimeErrorrr unbindr_nnr)rrrrmsgs r%rrsR II "eii&<&<&>)X.\ ?#% s# #)$  i .!((+I 88<< $ $  r$padded_sequencescg}|s|jdd|jd}tj||j}t ||D] \}}||k}||} |j | "|S)aEUnpad padded Tensor into a list of variable length Tensors. ``unpad_sequence`` unstacks padded Tensor into a list of variable length Tensors. Example: >>> from torch.nn.utils.rnn import pad_sequence, unpad_sequence >>> a = torch.ones(25, 300) >>> b = torch.ones(22, 300) >>> c = torch.ones(15, 300) >>> sequences = [a, b, c] >>> padded_sequences = pad_sequence(sequences) >>> lengths = torch.as_tensor([v.size(0) for v in sequences]) >>> unpadded_sequences = unpad_sequence(padded_sequences, lengths) >>> torch.allclose(sequences[0], unpadded_sequences[0]) True >>> torch.allclose(sequences[1], unpadded_sequences[1]) True >>> torch.allclose(sequences[2], unpadded_sequences[2]) True Args: padded_sequences (Tensor): padded sequences. lengths (Tensor): length of original (unpadded) sequences. batch_first (bool, optional): whether batch dimension first or not. Default: ``False``. Returns: a list of :class:`Tensor` objects rr}r) transpose_shaper!rrDzipappend) rrrunpadded_sequences max_lengthidxseqlengthmask unpacked_seqs r%rrsB ##Aq)!''*J ,,z'.. 9C+W50 VV|4y !!,/0 r$ctj|Dcgc]}|jdc}}tt |||Scc}w)aPacks a list of variable length Tensors. Consecutive call of the next functions: ``pad_sequence``, ``pack_padded_sequence``. ``sequences`` should be a list of Tensors of size ``L x *``, where `L` is the length of a sequence and `*` is any number of trailing dimensions, including ``0``. For unsorted sequences, use `enforce_sorted = False`. If ``enforce_sorted`` is ``True``, the sequences should be sorted in the order of decreasing length. ``enforce_sorted = True`` is only necessary for ONNX export. Example: >>> from torch.nn.utils.rnn import pack_sequence >>> a = torch.tensor([1, 2, 3]) >>> b = torch.tensor([4, 5]) >>> c = torch.tensor([6]) >>> pack_sequence([a, b, c]) PackedSequence(data=tensor([1, 4, 6, 2, 5, 3]), batch_sizes=tensor([3, 2, 1]), sorted_indices=None, unsorted_indices=None) Args: sequences (list[Tensor]): A list of sequences of decreasing length. enforce_sorted (bool, optional): if ``True``, checks that the input contains sequences sorted by length in a decreasing order. If ``False``, this condition is not checked. Default: ``True``. Returns: a :class:`PackedSequence` object r)r)r!rrrr)rrvrs r%rrsDBoo)>> from torch.nn.utils.rnn import pack_sequence, unpack_sequence >>> a = torch.tensor([1, 2, 3]) >>> b = torch.tensor([4, 5]) >>> c = torch.tensor([6]) >>> sequences = [a, b, c] >>> print(sequences) [tensor([1, 2, 3]), tensor([4, 5]), tensor([6])] >>> packed_sequences = pack_sequence(sequences) >>> print(packed_sequences) PackedSequence(data=tensor([1, 4, 6, 2, 5, 3]), batch_sizes=tensor([3, 2, 1]), sorted_indices=None, unsorted_indices=None) >>> unpacked_sequences = unpack_sequence(packed_sequences) >>> print(unpacked_sequences) [tensor([1, 2, 3]), tensor([4, 5]), tensor([6])] Args: packed_sequences (PackedSequence): A PackedSequence object. Returns: a list of :class:`Tensor` objects T)r)rr)rrrunpacked_sequencess r%rrBs.4!44DRV Wg'(8'tT r$ru)FT)FN)Frright)F)T)%rcollections.abcrtypingrrrrrr r typing_extensionsr r!r r __all__rrrr+rrr0rrrrirwrrarrxrrrrr#r$r%rs$PPP"   T] T]-j-8B<XrdBh%7HRL j%_j%b%)'+)- B B&!BV$Bv& B  668F+Xf-= => BD%)'+)- O  O&! OV$ Ov& O  OHV$4&9I* BJ BJ 649$ %BJBJ BJ  BJN"& J"J"J"J"3- J"  66> J"` AVT&\)*AAA A  AN.. .. &\ .f $F|$$$Nnfr$