L iddlZddlmZddlmZmZddlZddlmZ ddl m Z ddl m Z dgZedd ZGd de eZy) N)Iterator)OptionalTypeVar)Dataset)SamplerDistributedSampler_T_coT) covariantczeZdZdZ ddedeedeedededed dfd Zd e e fd Z d efd Z d ed dfdZ y)ra' Sampler that restricts data loading to a subset of the dataset. It is especially useful in conjunction with :class:`torch.nn.parallel.DistributedDataParallel`. In such a case, each process can pass a :class:`~torch.utils.data.DistributedSampler` instance as a :class:`~torch.utils.data.DataLoader` sampler, and load a subset of the original dataset that is exclusive to it. .. note:: Dataset is assumed to be of constant size and that any instance of it always returns the same elements in the same order. Args: dataset: Dataset used for sampling. num_replicas (int, optional): Number of processes participating in distributed training. By default, :attr:`world_size` is retrieved from the current distributed group. rank (int, optional): Rank of the current process within :attr:`num_replicas`. By default, :attr:`rank` is retrieved from the current distributed group. shuffle (bool, optional): If ``True`` (default), sampler will shuffle the indices. seed (int, optional): random seed used to shuffle the sampler if :attr:`shuffle=True`. This number should be identical across all processes in the distributed group. Default: ``0``. drop_last (bool, optional): if ``True``, then the sampler will drop the tail of the data to make it evenly divisible across the number of replicas. If ``False``, the sampler will add extra indices to make the data evenly divisible across the replicas. Default: ``False``. .. warning:: In distributed mode, calling the :meth:`set_epoch` method at the beginning of each epoch **before** creating the :class:`DataLoader` iterator is necessary to make shuffling work properly across multiple epochs. Otherwise, the same ordering will be always used. Example:: >>> # xdoctest: +SKIP >>> sampler = DistributedSampler(dataset) if is_distributed else None >>> loader = DataLoader(dataset, shuffle=(sampler is None), ... sampler=sampler) >>> for epoch in range(start_epoch, n_epochs): ... if is_distributed: ... sampler.set_epoch(epoch) ... train(loader) Ndataset num_replicasrankshuffleseed drop_lastreturnc|3tjs tdtj}|3tjs tdtj}||k\s|dkrt d|d|dz d||_||_||_d|_ ||_ |jrmt|j |jzdk7rHtjt|j |jz |jz |_n:tjt|j |jz |_|j|jz|_||_||_y)Nz,Requires distributed package to be availablerz Invalid rank z%, rank should be in the interval [0, ])dist is_available RuntimeErrorget_world_sizeget_rank ValueErrorr r repochrlenmathceil num_samples total_sizerr)selfr r rrrrs b/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/torch/utils/data/distributed.py__init__zDistributedSampler.__init__BsZ  $$&"#QRR..0L <$$&"#QRR==?D < 4!8v%J>c$,,/$2C2CCqH $yyT\\"T%6%66$:K:KK D  $yyT\\):T=N=N)NOD **T->->>  c|jrutj}|j|j|j ztj t|j|j}n'ttt|j}|jsZ|jt|z }|t|kr ||d|z }n:||tj|t|z zd|z }n|d|j}t||jk(sJ||j |j|j"}t||j$k(sJt'|S)N) generator)rtorch Generator manual_seedrrrandpermrr tolistlistrangerr!rrrr r iter)r"gindices padding_sizes r#__iter__zDistributedSampler.__iter__ks7 <<!A MM$))djj0 1nnS%6!DKKMG5T\\!234G~~??S\9Ls7|+7=L11Gdii s7|0K&LL!\ /0G7|t...$))doo8I8IIJ7|t/////G}r%c|jS)N)r )r"s r#__len__zDistributedSampler.__len__sr%rc||_y)a1 Set the epoch for this sampler. When :attr:`shuffle=True`, this ensures all replicas use a different random ordering for each epoch. Otherwise, the next iteration of this sampler will yield the same ordering. Args: epoch (int): Epoch number. N)r)r"rs r# set_epochzDistributedSampler.set_epochs  r%)NNTrF)__name__ __module__ __qualname____doc__rrintboolr$rr r3r5r7r%r#rrs.f'+"''sm'sm '  '  '' 'R(5/:   s t r%)rcollections.abcrtypingrrr(torch.distributed distributedrtorch.utils.data.datasetrtorch.utils.data.samplerr__all__r rr>r%r#rFsD $$ ,,  4(EEr%