L i+.ddlZddlmZmZmZddlmZddlZddlmZddl m Z ddl m Z mZddlmZmZmZddlmZmZd d lmZd d lmZd d lmZmZmZmZgd ZdddZ GddeZ!Gdde!Z"Gdde!Z#Gdde!Z$Gdde!Z%Gdde%Z&Gdde%Z'Gdd e%Z(Gd!d"e%Z)Gd#d$eZ*Gd%d&e*e"Z+Gd'd(e*e#Z,Gd)d*e*e$Z-Gd+d,e*e&Z.Gd-d.e*e'Z/Gd/d0e*e(Z0y)1N)LiteralOptionalUnion) deprecated)Tensor)reproducibility_notes) functionalinit) _size_1_t _size_2_t _size_3_t) ParameterUninitializedParameter)LazyModuleMixin)Module)_pair_reverse_repeat_tuple_single_triple) Conv1dConv2dConv3dConvTranspose1dConvTranspose2dConvTranspose3d LazyConv1d LazyConv2d LazyConv3dLazyConvTranspose1dLazyConvTranspose2dLazyConvTranspose3da* :attr:`groups` controls the connections between inputs and outputs. :attr:`in_channels` and :attr:`out_channels` must both be divisible by :attr:`groups`. For example, * At groups=1, all inputs are convolved to all outputs. * At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels and producing half the output channels, and both subsequently concatenated. * At groups= :attr:`in_channels`, each input channel is convolved with its own set of filters (of size :math:`\frac{\text{out\_channels}}{\text{in\_channels}}`).aWhen `groups == in_channels` and `out_channels == K * in_channels`, where `K` is a positive integer, this operation is also known as a "depthwise convolution". In other words, for an input of size :math:`(N, C_{in}, L_{in})`, a depthwise convolution with a depthwise multiplier `K` can be performed with the arguments :math:`(C_\text{in}=C_\text{in}, C_\text{out}=C_\text{in} \times \text{K}, ..., \text{groups}=C_\text{in})`.) groups_notedepthwise_separable_noteceZdZUgdZdeej iZdededeedefdZe ed<e e ed<e ed <e e d fed <e e d fed <e e e e d ffed <e e d fed<eed<e e d fed<e ed<eded<eed<eeed< dde d e d e e d fd e e d fd e e e e d ffde e d fdede e d fde dededddffd ZddZdZfdZxZS)_ConvNd) stridepaddingdilationgroups padding_modeoutput_padding in_channels out_channels kernel_sizebiasinputweightreturncyNselfr1r2r0s [/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/torch/nn/modules/conv.py _conv_forwardz_ConvNd._conv_forwardEsr- _reversed_padding_repeated_twicer..r/r'r(r) transposedr,r*zerosreflect replicatecircularr+Nc | | d}t|| dkr td|| zdk7r td|| zdk7r tdddh}t|tr7||vrtd|d ||dk(rt d |Dr td hd }| |vrtd |d| d||_||_||_||_ ||_ ||_ ||_ ||_ | |_| |_t|jtrddgt!|z|_|dk(rt%||t't!|dz ddD]=\}}}||dz z}|dz}||j"d|z<||z |j"d|zdz<?nt)|jd|_|r-t+t-j.||| zg|fi||_n,t+t-j.||| zg|fi||_| r%t+t-j.|fi||_n|j5dd|j7y)Ndevicedtyperz!groups must be a positive integer'in_channels must be divisible by groupsz(out_channels must be divisible by groupssamevalidzInvalid padding string z, should be one of c3&K|] }|dk7 yw)rNr6).0ss r9 z#_ConvNd.__init__..us(@Aa(@sz8padding='same' is not supported for strided convolutions>r?r@rBrAzpadding_mode must be one of z, but got padding_mode=''rr0)super__init__ ValueError isinstancestranyr-r.r/r'r(r)r=r,r*r+lenr<ziprangerrtorchemptyr2r0register_parameterreset_parameters)r8r-r.r/r'r(r)r=r,r*r0r+rErFfactory_kwargsvalid_padding_stringsvalid_padding_modesdki total_paddingleft_pad __class__s r9rRz_ConvNd.__init__Ws %+U;  Q;@A A  1 $FG G & A %GH H!' 1 gs #33 -g[8KLaKbc& S(@(@%@ NL 2 2./B.CC[\h[iijk '(&    $, ( dllC (56FS=M4MD 1& "k5[1AA1Er2+N GAq!%&QKM,1HCKD99!a%@%099!a%!)D 5J a5D 1 #  ,&"8G;G$DK$ !;G;G$DK !%++l"Mn"MNDI  # #FD 1 r;cJtj|jtjd|j ctj |j\}}|dk7r;dtj|z }tj|j | |yyy)N)arr)r kaiming_uniform_r2mathsqrtr0_calculate_fan_in_and_fan_outuniform_)r8fan_in_bounds r9r]z_ConvNd.reset_parameterss| dkkTYYq\: 99 ::4;;GIFA{DIIf-- dii%7 !r;cd}|jdt|jzk7r|dz }|jdt|jzk7r|dz }|jdt|jzk7r|dz }|jdk7r|dz }|j |d z }|j d k7r|d z }|jd i|jS) NzI{in_channels}, {out_channels}, kernel_size={kernel_size}, stride={stride})rz, padding={padding})rz, dilation={dilation}z!, output_padding={output_padding}rz, groups={groups}z , bias=Falser?z, padding_mode={padding_mode}r6) r(rWr)r,r*r0r+format__dict__)r8rLs r9 extra_reprz_ConvNd.extra_reprs W <<4#dll"33 3 & &A ==D3t}}#55 5 ( (A   $T-@-@)A"A A 4 4A ;;!  $ $A 99   A    ' 0 0Aqxx($--((r;cLt||t|dsd|_yy)Nr+r?)rQ __setstate__hasattrr+)r8staterfs r9rwz_ConvNd.__setstate__s' U#t^, 'D -r;NNr3N)__name__ __module__ __qualname__ __constants__rrZr__annotations__r:intlisttuplerrUboolrrRr]rurw __classcell__rfs@r9r&r&7s Mx 56O%+3;F3C &*3i/sCx #s(O 3c3h' ((CHo#s(O# KEFF N 6 Z Z Z 38_ Z c3h Z sE#s(O+, Z S/Z Z c3hZ Z Z IJZ  Z x 8) ((r;r&ceZdZddjdieezdzZ ddedededed e e efd ed ed e d e dddffd Z deded eefdZdedefdZxZS)raApplies a 1D convolution over an input signal composed of several input planes. In the simplest case, the output value of the layer with input size :math:`(N, C_{\text{in}}, L)` and output :math:`(N, C_{\text{out}}, L_{\text{out}})` can be precisely described as: .. math:: \text{out}(N_i, C_{\text{out}_j}) = \text{bias}(C_{\text{out}_j}) + \sum_{k = 0}^{C_{in} - 1} \text{weight}(C_{\text{out}_j}, k) \star \text{input}(N_i, k) where :math:`\star` is the valid `cross-correlation`_ operator, :math:`N` is a batch size, :math:`C` denotes a number of channels, :math:`L` is a length of signal sequence. u This module supports :ref:`TensorFloat32`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. * :attr:`stride` controls the stride for the cross-correlation, a single number or a one-element tuple. * :attr:`padding` controls the amount of padding applied to the input. It can be either a string {{'valid', 'same'}} or a tuple of ints giving the amount of implicit padding applied on both sides. * :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. {groups_note} Note: {depthwise_separable_note} Note: {cudnn_reproducibility_note} Note: ``padding='valid'`` is the same as no padding. ``padding='same'`` pads the input so the output has the shape as the input. However, this mode doesn't support any stride values other than 1. Note: This module supports complex data types i.e. ``complex32, complex64, complex128``. Args: in_channels (int): Number of channels in the input image out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int, tuple or str, optional): Padding added to both sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` padding_mode (str, optional): ``'zeros'``, ``'reflect'``, ``'replicate'`` or ``'circular'``. Default: ``'zeros'`` au Shape: - Input: :math:`(N, C_{in}, L_{in})` or :math:`(C_{in}, L_{in})` - Output: :math:`(N, C_{out}, L_{out})` or :math:`(C_{out}, L_{out})`, where .. math:: L_{out} = \left\lfloor\frac{L_{in} + 2 \times \text{padding} - \text{dilation} \times (\text{kernel\_size} - 1) - 1}{\text{stride}} + 1\right\rfloor Attributes: weight (Tensor): the learnable weights of the module of shape :math:`(\text{out\_channels}, \frac{\text{in\_channels}}{\text{groups}}, \text{kernel\_size})`. The values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{in} * \text{kernel\_size}}` bias (Tensor): the learnable bias of the module of shape (out_channels). If :attr:`bias` is ``True``, then the values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{in} * \text{kernel\_size}}` Examples:: >>> m = nn.Conv1d(16, 33, 3, stride=2) >>> input = torch.randn(20, 16, 50) >>> output = m(input) .. _cross-correlation: https://en.wikipedia.org/wiki/Cross-correlation .. _link: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md Nr-r.r/r'r(r)r*r0r+r>r3c | | d} t|} t|}t|tr|n t|}t|}t|||| |||dtd||| f i| yNrDFr)rrTrUrQrRr8r-r.r/r'r(r)r*r0r+rErFr^ kernel_size_stride_padding_ dilation_rfs r9rRzConv1d.__init__=s%+U;{+ &/(#67GG`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. * :attr:`stride` controls the stride for the cross-correlation, a single number or a tuple. * :attr:`padding` controls the amount of padding applied to the input. It can be either a string {{'valid', 'same'}} or an int / a tuple of ints giving the amount of implicit padding applied on both sides. * :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. {groups_note} The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`dilation` can either be: - a single ``int`` -- in which case the same value is used for the height and width dimension - a ``tuple`` of two ints -- in which case, the first `int` is used for the height dimension, and the second `int` for the width dimension Note: {depthwise_separable_note} Note: {cudnn_reproducibility_note} Note: ``padding='valid'`` is the same as no padding. ``padding='same'`` pads the input so the output has the shape as the input. However, this mode doesn't support any stride values other than 1. Note: This module supports complex data types i.e. ``complex32, complex64, complex128``. Args: in_channels (int): Number of channels in the input image out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int, tuple or str, optional): Padding added to all four sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` padding_mode (str, optional): ``'zeros'``, ``'reflect'``, ``'replicate'`` or ``'circular'``. Default: ``'zeros'`` aE Shape: - Input: :math:`(N, C_{in}, H_{in}, W_{in})` or :math:`(C_{in}, H_{in}, W_{in})` - Output: :math:`(N, C_{out}, H_{out}, W_{out})` or :math:`(C_{out}, H_{out}, W_{out})`, where .. math:: H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - \text{dilation}[0] \times (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[1] - \text{dilation}[1] \times (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor Attributes: weight (Tensor): the learnable weights of the module of shape :math:`(\text{out\_channels}, \frac{\text{in\_channels}}{\text{groups}},` :math:`\text{kernel\_size[0]}, \text{kernel\_size[1]})`. The values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{in} * \prod_{i=0}^{1}\text{kernel\_size}[i]}` bias (Tensor): the learnable bias of the module of shape (out_channels). If :attr:`bias` is ``True``, then the values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{in} * \prod_{i=0}^{1}\text{kernel\_size}[i]}` Examples: >>> # With square kernels and equal stride >>> m = nn.Conv2d(16, 33, 3, stride=2) >>> # non-square kernels and unequal stride and with padding >>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2)) >>> # non-square kernels and unequal stride and with padding and dilation >>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1)) >>> input = torch.randn(20, 16, 50, 100) >>> output = m(input) .. _cross-correlation: https://en.wikipedia.org/wiki/Cross-correlation .. _link: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md Nr-r.r/r'r(r)r*r0r+r>r3c | | d} t|} t|}t|tr|n t|}t|}t|||| |||dtd||| f i| yr)rrTrUrQrRrs r9rRzConv2d.__init__s{%+U;[) -(#67E'N(O          !H     r;r1r2c ~|jdk7rltjtj||j|j|||j t d|j|jStj||||j |j|j|jSr) r+rconv2drr<r'rr)r*r(r7s r9r:zConv2d._conv_forwards    '884@@tGXGX a    xx 64dllDMM4;;  r;cP|j||j|jSr5rrs r9rzConv2d.forward#rr;r6r)r|r}r~rsrrrrr rrUrrrRrrr:rrrs@r9rrvs "8 8 p q8 =p'q8 =p+`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. * :attr:`stride` controls the stride for the cross-correlation. * :attr:`padding` controls the amount of padding applied to the input. It can be either a string {{'valid', 'same'}} or a tuple of ints giving the amount of implicit padding applied on both sides. * :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. {groups_note} The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`dilation` can either be: - a single ``int`` -- in which case the same value is used for the depth, height and width dimension - a ``tuple`` of three ints -- in which case, the first `int` is used for the depth dimension, the second `int` for the height dimension and the third `int` for the width dimension Note: {depthwise_separable_note} Note: {cudnn_reproducibility_note} Note: ``padding='valid'`` is the same as no padding. ``padding='same'`` pads the input so the output has the shape as the input. However, this mode doesn't support any stride values other than 1. Note: This module supports complex data types i.e. ``complex32, complex64, complex128``. Args: in_channels (int): Number of channels in the input image out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int, tuple or str, optional): Padding added to all six sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` padding_mode (str, optional): ``'zeros'``, ``'reflect'``, ``'replicate'`` or ``'circular'``. Default: ``'zeros'`` a Shape: - Input: :math:`(N, C_{in}, D_{in}, H_{in}, W_{in})` or :math:`(C_{in}, D_{in}, H_{in}, W_{in})` - Output: :math:`(N, C_{out}, D_{out}, H_{out}, W_{out})` or :math:`(C_{out}, D_{out}, H_{out}, W_{out})`, where .. math:: D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - \text{dilation}[0] \times (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor .. math:: H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - \text{dilation}[1] \times (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - \text{dilation}[2] \times (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor Attributes: weight (Tensor): the learnable weights of the module of shape :math:`(\text{out\_channels}, \frac{\text{in\_channels}}{\text{groups}},` :math:`\text{kernel\_size[0]}, \text{kernel\_size[1]}, \text{kernel\_size[2]})`. The values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{in} * \prod_{i=0}^{2}\text{kernel\_size}[i]}` bias (Tensor): the learnable bias of the module of shape (out_channels). If :attr:`bias` is ``True``, then the values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{in} * \prod_{i=0}^{2}\text{kernel\_size}[i]}` Examples:: >>> # With square kernels and equal stride >>> m = nn.Conv3d(16, 33, 3, stride=2) >>> # non-square kernels and unequal stride and with padding >>> m = nn.Conv3d(16, 33, (3, 5, 2), stride=(2, 1, 1), padding=(4, 2, 0)) >>> input = torch.randn(20, 16, 10, 50, 100) >>> output = m(input) .. _cross-correlation: https://en.wikipedia.org/wiki/Cross-correlation .. _link: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md Nr-r.r/r'r(r)r*r0r+r>r3c | | d} t|} t|}t|tr|n t|}t|}t|||| |||dtd||| f i| yr)rrTrUrQrRrs r9rRzConv3d.__init__s}%+U;{+ &/(#67GGU>U=VW %+U;              r;r1 output_sizer'r(r/num_spatial_dimsr)c&|t|j}|S|j|dzk(} | rdnd} t|| |zk(r|| d}t||k7r5t d|d|jd|d| |zdt|d t j jttg} t j jttg} t|D]l} |j| | zdz || zd|| zz ||| nd|| dz zzdz}| j|| j| | || zdz ntt|D]D}||}| |}| |}||ks||kDst d |d | d | d |jddd t j jttg}t|D]} |j|| | | z |}|S) NrPr ConvTransposezD: for zD input, output_size must have z or z elements (got )zrequested an output size of z, but valid sizes range from z to z (for an input of ) rr,dimrWrSrZjitannotaterrrYsizeappend)r8r1rr'r(r/rr)ret has_batch_dimnum_non_spatial_dims min_sizes max_sizesradim_sizercrmin_sizemax_sizeress r9_output_paddingz _ConvTransposeNd._output_paddings  $--.CP M"IIK+;a+??M(511 ;#7:J#JJ)*>*?@ ;#33 #$4#5WUYY[MIhiyhz{.1AAB/RUVaRbQccdf  **49b9I **49b9I+, ?ZZ$8 89A=J'!*n%&.&:x{"1~)++   *  1q !9A!=> ?3{+, "1~$Q<$Q<(?dXo$6{mD ){$yk9KEJJLYZY[L\K]]^`  ))$$T#Y3C+, : ;q>IaL89 :C r;rzr{r5) r|r}r~rRrrrrrrrs@r9rrs#  # ^)-33d3i(3S 3 c 3 #Y 3349%3 c3r;rceZdZdjdieedzZ ddededededed ed ed e d ed e dddffd Z dde de eede fdZxZS)ruX Applies a 1D transposed convolution operator over an input image composed of several input planes. This module can be seen as the gradient of Conv1d with respect to its input. It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation as it does not compute a true inverse of convolution). For more information, see the visualizations `here`_ and the `Deconvolutional Networks`_ paper. This module supports :ref:`TensorFloat32`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. * :attr:`stride` controls the stride for the cross-correlation. * :attr:`padding` controls the amount of implicit zero padding on both sides for ``dilation * (kernel_size - 1) - padding`` number of points. See note below for details. * :attr:`output_padding` controls the additional size added to one side of the output shape. See note below for details. * :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but the link `here`_ has a nice visualization of what :attr:`dilation` does. {groups_note} Note: The :attr:`padding` argument effectively adds ``dilation * (kernel_size - 1) - padding`` amount of zero padding to both sizes of the input. This is set so that when a :class:`~torch.nn.Conv1d` and a :class:`~torch.nn.ConvTranspose1d` are initialized with same parameters, they are inverses of each other in regard to the input and output shapes. However, when ``stride > 1``, :class:`~torch.nn.Conv1d` maps multiple input shapes to the same output shape. :attr:`output_padding` is provided to resolve this ambiguity by effectively increasing the calculated output shape on one side. Note that :attr:`output_padding` is only used to find output shape, but does not actually add zero-padding to output. Note: In some circumstances when using the CUDA backend with CuDNN, this operator may select a nondeterministic algorithm to increase performance. If this is undesirable, you can try to make the operation deterministic (potentially at a performance cost) by setting ``torch.backends.cudnn.deterministic = True``. Please see the notes on :doc:`/notes/randomness` for background. Args: in_channels (int): Number of channels in the input image out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): ``dilation * (kernel_size - 1) - padding`` zero-padding will be added to both sides of the input. Default: 0 output_padding (int or tuple, optional): Additional size added to one side of the output shape. Default: 0 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 a~ Shape: - Input: :math:`(N, C_{in}, L_{in})` or :math:`(C_{in}, L_{in})` - Output: :math:`(N, C_{out}, L_{out})` or :math:`(C_{out}, L_{out})`, where .. math:: L_{out} = (L_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{dilation} \times (\text{kernel\_size} - 1) + \text{output\_padding} + 1 Attributes: weight (Tensor): the learnable weights of the module of shape :math:`(\text{in\_channels}, \frac{\text{out\_channels}}{\text{groups}},` :math:`\text{kernel\_size})`. The values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{out} * \text{kernel\_size}}` bias (Tensor): the learnable bias of the module of shape (out_channels). If :attr:`bias` is ``True``, then the values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{out} * \text{kernel\_size}}` .. _`here`: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md .. _`Deconvolutional Networks`: https://www.matthewzeiler.com/mattzeiler/deconvolutionalnetworks.pdf Nr-r.r/r'r(r,r*r0r)r+r>r3c | | d} t|}t|}t|}t| } t|}t||||||| d|||| f i| yNrDT)rrQrRr8r-r.r/r'r(r,r*r0r)r+rErFr^rfs r9rRzConvTranspose1d.__init__x%+U;k* '"8$ 0              r;r1rc |jdk7r tdt|jtsJd}|j |||j |j|j||j}tj||j|j|j |j||j|jS)Nr?z:Only `zeros` padding mode is supported for ConvTranspose1dr)r+rSrTr(rrr'r/r)rconv_transpose1dr2r0r*r8r1rrr,s r9rzConvTranspose1d.forwards    'L $,,...--   KK LL     MM !!  KK II KK LL  KK MM  r;r6 rrrrTrr?NNr5)r|r}r~rsrrrrr rrrRrrrrrrs@r9rr.s> > | }> =|'}> =|+<}> =~ Z  F$%MT" " "  "  "  " "" " " " IJ"  " H V (492E QW r;rceZdZdjdieedzZ ddededededed ed ed e d ed e dddffd Z dde de eede fdZxZS)ruApplies a 2D transposed convolution operator over an input image composed of several input planes. This module can be seen as the gradient of Conv2d with respect to its input. It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation as it does not compute a true inverse of convolution). For more information, see the visualizations `here`_ and the `Deconvolutional Networks`_ paper. This module supports :ref:`TensorFloat32`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. * :attr:`stride` controls the stride for the cross-correlation. When stride > 1, ConvTranspose2d inserts zeros between input elements along the spatial dimensions before applying the convolution kernel. This zero-insertion operation is the standard behavior of transposed convolutions, which can increase the spatial resolution and is equivalent to a learnable upsampling operation. * :attr:`padding` controls the amount of implicit zero padding on both sides for ``dilation * (kernel_size - 1) - padding`` number of points. See note below for details. * :attr:`output_padding` controls the additional size added to one side of the output shape. See note below for details. * :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but the link `here`_ has a nice visualization of what :attr:`dilation` does. {groups_note} The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`output_padding` can either be: - a single ``int`` -- in which case the same value is used for the height and width dimensions - a ``tuple`` of two ints -- in which case, the first `int` is used for the height dimension, and the second `int` for the width dimension Note: The :attr:`padding` argument effectively adds ``dilation * (kernel_size - 1) - padding`` amount of zero padding to both sizes of the input. This is set so that when a :class:`~torch.nn.Conv2d` and a :class:`~torch.nn.ConvTranspose2d` are initialized with same parameters, they are inverses of each other in regard to the input and output shapes. However, when ``stride > 1``, :class:`~torch.nn.Conv2d` maps multiple input shapes to the same output shape. :attr:`output_padding` is provided to resolve this ambiguity by effectively increasing the calculated output shape on one side. Note that :attr:`output_padding` is only used to find output shape, but does not actually add zero-padding to output. Note: {cudnn_reproducibility_note} Args: in_channels (int): Number of channels in the input image out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): ``dilation * (kernel_size - 1) - padding`` zero-padding will be added to both sides of each dimension in the input. Default: 0 output_padding (int or tuple, optional): Additional size added to one side of each dimension in the output shape. Default: 0 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 a Shape: - Input: :math:`(N, C_{in}, H_{in}, W_{in})` or :math:`(C_{in}, H_{in}, W_{in})` - Output: :math:`(N, C_{out}, H_{out}, W_{out})` or :math:`(C_{out}, H_{out}, W_{out})`, where .. math:: H_{out} = (H_{in} - 1) \times \text{stride}[0] - 2 \times \text{padding}[0] + \text{dilation}[0] \times (\text{kernel\_size}[0] - 1) + \text{output\_padding}[0] + 1 .. math:: W_{out} = (W_{in} - 1) \times \text{stride}[1] - 2 \times \text{padding}[1] + \text{dilation}[1] \times (\text{kernel\_size}[1] - 1) + \text{output\_padding}[1] + 1 Attributes: weight (Tensor): the learnable weights of the module of shape :math:`(\text{in\_channels}, \frac{\text{out\_channels}}{\text{groups}},` :math:`\text{kernel\_size[0]}, \text{kernel\_size[1]})`. The values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{out} * \prod_{i=0}^{1}\text{kernel\_size}[i]}` bias (Tensor): the learnable bias of the module of shape (out_channels) If :attr:`bias` is ``True``, then the values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{out} * \prod_{i=0}^{1}\text{kernel\_size}[i]}` Examples:: >>> # With square kernels and equal stride >>> m = nn.ConvTranspose2d(16, 33, 3, stride=2) >>> # non-square kernels and unequal stride and with padding >>> m = nn.ConvTranspose2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2)) >>> input = torch.randn(20, 16, 50, 100) >>> output = m(input) >>> # exact output size can be also specified as an argument >>> input = torch.randn(1, 16, 12, 12) >>> downsample = nn.Conv2d(16, 16, 3, stride=2, padding=1) >>> upsample = nn.ConvTranspose2d(16, 16, 3, stride=2, padding=1) >>> h = downsample(input) >>> h.size() torch.Size([1, 16, 6, 6]) >>> output = upsample(h, output_size=input.size()) >>> output.size() torch.Size([1, 16, 12, 12]) .. _`here`: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md .. _`Deconvolutional Networks`: https://www.matthewzeiler.com/mattzeiler/deconvolutionalnetworks.pdf Nr-r.r/r'r(r,r*r0r)r+r>r3c | | d} t|}t|}t|}t| } t|}t||||||| d|||| f i| yr)rrQrRrs r9rRzConvTranspose2d.__init__Isv%+U;K( v.?~.              r;r1rc |jdk7r tdt|jtsJd}|j |||j |j|j||j}tj||j|j|j |j||j|jS)z Performs the forward pass. Attributes: input (Tensor): The input tensor. output_size (list[int], optional): A list of integers representing the size of the output tensor. Default is None. r?z:Only `zeros` padding mode is supported for ConvTranspose2drP)r+rSrTr(rrr'r/r)rconv_transpose2dr2r0r*rs r9rzConvTranspose2d.forwardms    'L $,,...--   KK LL     MM !!  KK II KK LL  KK MM  r;r6rr5)r|r}r~rsrrrrr rrrRrrrrrrs@r9rrsB B D EB =D'EB =D+`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. * :attr:`stride` controls the stride for the cross-correlation. * :attr:`padding` controls the amount of implicit zero padding on both sides for ``dilation * (kernel_size - 1) - padding`` number of points. See note below for details. * :attr:`output_padding` controls the additional size added to one side of the output shape. See note below for details. * :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but the link `here`_ has a nice visualization of what :attr:`dilation` does. {groups_note} The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`output_padding` can either be: - a single ``int`` -- in which case the same value is used for the depth, height and width dimensions - a ``tuple`` of three ints -- in which case, the first `int` is used for the depth dimension, the second `int` for the height dimension and the third `int` for the width dimension Note: The :attr:`padding` argument effectively adds ``dilation * (kernel_size - 1) - padding`` amount of zero padding to both sizes of the input. This is set so that when a :class:`~torch.nn.Conv3d` and a :class:`~torch.nn.ConvTranspose3d` are initialized with same parameters, they are inverses of each other in regard to the input and output shapes. However, when ``stride > 1``, :class:`~torch.nn.Conv3d` maps multiple input shapes to the same output shape. :attr:`output_padding` is provided to resolve this ambiguity by effectively increasing the calculated output shape on one side. Note that :attr:`output_padding` is only used to find output shape, but does not actually add zero-padding to output. Note: {cudnn_reproducibility_note} Args: in_channels (int): Number of channels in the input image out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): ``dilation * (kernel_size - 1) - padding`` zero-padding will be added to both sides of each dimension in the input. Default: 0 output_padding (int or tuple, optional): Additional size added to one side of each dimension in the output shape. Default: 0 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 aY Shape: - Input: :math:`(N, C_{in}, D_{in}, H_{in}, W_{in})` or :math:`(C_{in}, D_{in}, H_{in}, W_{in})` - Output: :math:`(N, C_{out}, D_{out}, H_{out}, W_{out})` or :math:`(C_{out}, D_{out}, H_{out}, W_{out})`, where .. math:: D_{out} = (D_{in} - 1) \times \text{stride}[0] - 2 \times \text{padding}[0] + \text{dilation}[0] \times (\text{kernel\_size}[0] - 1) + \text{output\_padding}[0] + 1 .. math:: H_{out} = (H_{in} - 1) \times \text{stride}[1] - 2 \times \text{padding}[1] + \text{dilation}[1] \times (\text{kernel\_size}[1] - 1) + \text{output\_padding}[1] + 1 .. math:: W_{out} = (W_{in} - 1) \times \text{stride}[2] - 2 \times \text{padding}[2] + \text{dilation}[2] \times (\text{kernel\_size}[2] - 1) + \text{output\_padding}[2] + 1 Attributes: weight (Tensor): the learnable weights of the module of shape :math:`(\text{in\_channels}, \frac{\text{out\_channels}}{\text{groups}},` :math:`\text{kernel\_size[0]}, \text{kernel\_size[1]}, \text{kernel\_size[2]})`. The values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{out} * \prod_{i=0}^{2}\text{kernel\_size}[i]}` bias (Tensor): the learnable bias of the module of shape (out_channels) If :attr:`bias` is ``True``, then the values of these weights are sampled from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{groups}{C_\text{out} * \prod_{i=0}^{2}\text{kernel\_size}[i]}` Examples:: >>> # With square kernels and equal stride >>> m = nn.ConvTranspose3d(16, 33, 3, stride=2) >>> # non-square kernels and unequal stride and with padding >>> m = nn.ConvTranspose3d(16, 33, (3, 5, 2), stride=(2, 1, 1), padding=(0, 4, 2)) >>> input = torch.randn(20, 16, 10, 50, 100) >>> output = m(input) .. _`here`: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md .. _`Deconvolutional Networks`: https://www.matthewzeiler.com/mattzeiler/deconvolutionalnetworks.pdf Nr-r.r/r'r(r,r*r0r)r+r>r3c | | d} t|}t|}t|}t| } t|}t||||||| d|||| f i| yr)rrQrRrs r9rRzConvTranspose3d.__init__rr;r1rc |jdk7r tdt|jtsJd}|j |||j |j|j||j}tj||j|j|j |j||j|jS)Nr?z:Only `zeros` padding mode is supported for ConvTranspose3d)r+rSrTr(rrr'r/r)rconv_transpose3dr2r0r*rs r9rzConvTranspose3d.forward,s    'L $,,...--   KK LL     MM !!  KK II KK LL  KK MM  r;r6rr5)r|r}r~rsrrrrr rrrRrrrrrrs@r9rrsA A B CA =B'CA =B+u{{1~REKKPQNRr;ctr5)NotImplementedErrorr8s r9rz&_LazyConvXdMixin._get_num_spatial_dimss!!r;r{)r|r}r~rrrrrr]rrrrrrs@r9rrjsn KsCx ""  '$6$t$: Sf S S"s"r;rcpeZdZdZeZ ddedededededed ed e d d dffd Z d efdZ xZ S)raA :class:`torch.nn.Conv1d` module with lazy initialization of the ``in_channels`` argument. The ``in_channels`` argument of the :class:`Conv1d` is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight` and `bias`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to both sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` padding_mode (str, optional): ``'zeros'``, ``'reflect'``, ``'replicate'`` or ``'circular'``. Default: ``'zeros'`` .. seealso:: :class:`torch.nn.Conv1d` and :class:`torch.nn.modules.lazy.LazyModuleMixin` Nr.r/r'r(r)r*r0r+r>r3c | | d} t |dd|||||d|f i| tdi| |_||_|rtdi| |_yyNrDrFr6rQrRrr2r.r0 r8r.r/r'r(r)r*r0r+rErFr^rfs r9rRzLazyConv1d.__init__s%+U;          ->~> ( .@@DI r;cyNrr6rs r9rz LazyConv1d._get_num_spatial_dimsr;r) r|r}r~rr cls_to_becomerr rrrRrrrs@r9rr8M MTAAA A  A  AAAIJA ABsr;rcpeZdZdZeZ ddedededededed ed e d d dffd Z d efdZ xZ S)raA :class:`torch.nn.Conv2d` module with lazy initialization of the ``in_channels`` argument. The ``in_channels`` argument of the :class:`Conv2d` that is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight` and `bias`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to both sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` padding_mode (str, optional): ``'zeros'``, ``'reflect'``, ``'replicate'`` or ``'circular'``. Default: ``'zeros'`` .. seealso:: :class:`torch.nn.Conv2d` and :class:`torch.nn.modules.lazy.LazyModuleMixin` Nr.r/r'r(r)r*r0r+r>r3c | | d} t |dd|||||d|f i| tdi| |_||_|rtdi| |_yyrrrs r9rRzLazyConv2d.__init__rr;cyNrPr6rs r9rz LazyConv2d._get_num_spatial_dims1rr;r) r|r}r~rrrrr rrrRrrrs@r9rrrr;rcpeZdZdZeZ ddedededededed ed e d d dffd Z d efdZ xZ S)ra#A :class:`torch.nn.Conv3d` module with lazy initialization of the ``in_channels`` argument. The ``in_channels`` argument of the :class:`Conv3d` that is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight` and `bias`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): Zero-padding added to both sides of the input. Default: 0 dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` padding_mode (str, optional): ``'zeros'``, ``'reflect'``, ``'replicate'`` or ``'circular'``. Default: ``'zeros'`` .. seealso:: :class:`torch.nn.Conv3d` and :class:`torch.nn.modules.lazy.LazyModuleMixin` Nr.r/r'r(r)r*r0r+r>r3c | | d} t |dd|||||d|f i| tdi| |_||_|rtdi| |_yyrrrs r9rRzLazyConv3d.__init__Vrr;cyNrr6rs r9rz LazyConv3d._get_num_spatial_dimswrr;r) r|r}r~rrrrr rrrRrrrs@r9rr6s:M MTAAA A  A  AAAIJA ABsr;rcveZdZdZeZ ddedededededed ed ed e d d dffd Z d efdZ xZ S)r aFA :class:`torch.nn.ConvTranspose1d` module with lazy initialization of the ``in_channels`` argument. The ``in_channels`` argument of the :class:`ConvTranspose1d` that is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight` and `bias`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): ``dilation * (kernel_size - 1) - padding`` zero-padding will be added to both sides of the input. Default: 0 output_padding (int or tuple, optional): Additional size added to one side of the output shape. Default: 0 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 .. seealso:: :class:`torch.nn.ConvTranspose1d` and :class:`torch.nn.modules.lazy.LazyModuleMixin` Nr.r/r'r(r,r*r0r)r+r>r3c | | d} t |dd|||||d|| f i| tdi| |_||_|rtdi| |_yyrrr8r.r/r'r(r,r*r0r)r+rErFr^rfs r9rRzLazyConvTranspose1d.__init__v%+U;           ->~> ( .@@DI r;cyrr6rs r9rz)LazyConvTranspose1d._get_num_spatial_dimsrr;r) r|r}r~rrrrr rrrRrrrs@r9r r |4$M $%MT!A!A!A !A  !A " !A!A!A!AIJ!A !AFsr;r cveZdZdZeZ ddedededededed ed ed e d d dffd Z d efdZ xZ S)r!aeA :class:`torch.nn.ConvTranspose2d` module with lazy initialization of the ``in_channels`` argument. The ``in_channels`` argument of the :class:`ConvTranspose2d` is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight` and `bias`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): ``dilation * (kernel_size - 1) - padding`` zero-padding will be added to both sides of each dimension in the input. Default: 0 output_padding (int or tuple, optional): Additional size added to one side of each dimension in the output shape. Default: 0 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 .. seealso:: :class:`torch.nn.ConvTranspose2d` and :class:`torch.nn.modules.lazy.LazyModuleMixin` Nr.r/r'r(r,r*r0r)r+r>r3c | | d} t |dd|||||d|| f i| tdi| |_||_|rtdi| |_yyrrrs r9rRzLazyConvTranspose2d.__init__rr;cyrr6rs r9rz)LazyConvTranspose2d._get_num_spatial_dimsrr;r) r|r}r~rrrrr rrrRrrrs@r9r!r!s4$M $%MT!A!A!A !A  !A " !A!A!A!AIJ!A !AFsr;r!cveZdZdZeZ ddedededededed ed ed e d d dffd Z d efdZ xZ S)r"aeA :class:`torch.nn.ConvTranspose3d` module with lazy initialization of the ``in_channels`` argument. The ``in_channels`` argument of the :class:`ConvTranspose3d` is inferred from the ``input.size(1)``. The attributes that will be lazily initialized are `weight` and `bias`. Check the :class:`torch.nn.modules.lazy.LazyModuleMixin` for further documentation on lazy modules and their limitations. Args: out_channels (int): Number of channels produced by the convolution kernel_size (int or tuple): Size of the convolving kernel stride (int or tuple, optional): Stride of the convolution. Default: 1 padding (int or tuple, optional): ``dilation * (kernel_size - 1) - padding`` zero-padding will be added to both sides of each dimension in the input. Default: 0 output_padding (int or tuple, optional): Additional size added to one side of each dimension in the output shape. Default: 0 groups (int, optional): Number of blocked connections from input channels to output channels. Default: 1 bias (bool, optional): If ``True``, adds a learnable bias to the output. Default: ``True`` dilation (int or tuple, optional): Spacing between kernel elements. Default: 1 .. seealso:: :class:`torch.nn.ConvTranspose3d` and :class:`torch.nn.modules.lazy.LazyModuleMixin` Nr.r/r'r(r,r*r0r)r+r>r3c | | d} t |dd|||||d|| f i| tdi| |_||_|rtdi| |_yyrrrs r9rRzLazyConvTranspose3d.__init__#rr;cyrr6rs r9rz)LazyConvTranspose3d._get_num_spatial_dimsFrr;r) r|r}r~rrrrr rrrRrrrs@r9r"r"rr;r")1rktypingrrrtyping_extensionsrrZrtorch._torch_docsrtorch.nnr rr torch.nn.common_typesr r r torch.nn.parameterrrlazyrmodulerutilsrrrr__all__rr&rrrrrrrrrrrrr r!r"r6r;r9rsS ++( 3*AA@!AA  H!x,Z(fZ(z_AW_ADnAWnAbfAWfAR[w[|_ &_ DB &B Jt &t L***>">"DA!6AJA!6AJB!6BLA*OAJA*OAJA*OAr;