L i d Z ddlmZ ddlmZmZ e r
ddlZddlmZ ej e
Z ed de fd Z
ed dd de fd
Zd
d Zy)a
Since, https://github.com/huggingface/transformers/pull/36963, loading is always performed with models on meta
device. But since the `init_empty_weights` and `find_tied_parameters` functions are from accelerate, and accelerate is
somewhat still a soft dependency, we copy the functions here to be used natively in Transformers.
The `init_empty_weights` and `init_on_device` functions were copied from `accelerate.big_modeling.py`, and the
`find_tied_parameters` was copied from `accelerate.utils.modeling.py`
)contextmanager )is_torch_availableloggingNinclude_buffersc # ~ K t t j d | 5 }| ddd y# 1 sw Y yxY ww)a
A context manager under which models are initialized with all parameters on the meta device, therefore creating an
empty model. Useful when just initializing the model would blow the available RAM.
Args:
include_buffers (`bool`, *optional*):
Whether or not to also put all buffers on the meta device while initializing.
Example:
```python
import torch.nn as nn
from accelerate import init_empty_weights
# Initialize a model with 100 billions parameters in no time and without using any RAM.
with init_empty_weights():
tst = nn.Sequential(*[nn.Linear(10000, 10000) for _ in range(1000)])
```
Any model created under this context manager has no weights. As such you can't do something like
`model.to(some_device)` with it. To load weights inside your empty model, see [`load_checkpoint_and_dispatch`].
Make sure to overwrite the default device_map param for [`load_checkpoint_and_dispatch`], otherwise dispatch is not
called.
meta)r N)init_on_devicetorchdevice)r fs j/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/transformers/integrations/accelerate.pyinit_empty_weightsr $ s7 <
V,o N RS s !=1 =:=r z torch.devicec # F K |r 5 d ddd yt j j |rt j j fd}d fd }|r dD ci c] }|t t
| }}ni } fd} |t j _ |r|t j _ |D ]' }t
t
| |t t
| ) d t j _ |rt j _ |j D ] \ }}t
t
|| y# 1 sw Y yxY wc c}w # t j _ |rt j _ |j D ] \ }}t
t
|| w xY ww)a)
A context manager under which models are initialized with all parameters on the specified device.
Args:
device (`torch.device`):
Device to initialize all parameters on.
include_buffers (`bool`, *optional*):
Whether or not to also put all buffers on the meta device while initializing.
Example:
```python
import torch.nn as nn
from accelerate import init_on_device
with init_on_device(device=torch.device("cuda")):
tst = nn.Linear(100, 100) # on `cuda` device
```
Nc | || |st | j | }| j | j }|j |d< || j | j fi || j |< y y )N
requires_grad)type _parameters__dict__r to)modulenameparam param_clskwargsr old_register_parameters r register_empty_parameterz0init_on_device..register_empty_parameterd s vtU3 V / / 56I''-66F&+&9&9F? #'01C1CD1I1L1LV1T'_X^'_F t $ c z | ||| |,| j | j | j |< y y )N)
persistent)_buffersr )r r bufferr r old_register_buffers r register_empty_bufferz-init_on_device..register_empty_bufferl s> FD&ZH $*OOD$9$<$.patch_tensor_constructor..wrapper{ s %F8 t&v& &r r+ )r- r. r s ` r patch_tensor_constructorz0init_on_device..patch_tensor_constructorz s ' r )T)nnModuleregister_parameterregister_buffergetattrr setattritems)
r r r r$ torch_function_name tensor_constructors_to_patchr/ old_torch_functionr# r s
` @@r r
r
F s *
YY 9 9 ii 7 7`E (J(
#
0C!D D(
$ (
(*$ D'? $ (=BII %#? o E.0HQVXkIl0m n o
'= $ (;BII %7S7Y7Y7[ D 3 !3 E.0B C Da *(
, (> $ (;BII %7S7Y7Y7[ D 3 !3 E.0B C DsC F!D6AF!!E9
F!A E AF!6D?; F!AF F!c t | j d }t | j d }t |j t |j z
}i }|D ]A }|| }|j D ]' \ }} | |u s ||vrg ||< || j | ) C |j D
cg c]& \ }
} t
|
gt t | z ( c} }
S c c} }
w )a
Find the tied parameters in a given model.
The signature accepts keyword arguments, but they are for the recursive part of this function and you should ignore
them.
Args:
model (`torch.nn.Module`): The model to inspect.
Returns:
list[list[str]]: A list of lists of parameter names being all tied together.
Example:
```py
>>> from collections import OrderedDict
>>> import torch.nn as nn
>>> model = nn.Sequential(OrderedDict([("linear1", nn.Linear(4, 4)), ("linear2", nn.Linear(4, 4))]))
>>> model.linear2.weight = model.linear1.weight
>>> find_tied_parameters(model)
[['linear1.weight', 'linear2.weight']]
```
F)remove_duplicateT)dictnamed_parameterssetkeysr6 appendsortedlist) modelr all_named_parameters no_duplicate_named_parameterstied_param_namestied_param_groupstied_param_name
tied_param
param_namer weighttieds r find_tied_parametersrM s > 6 6 6 NO %))?)?QU)?)V$W! /4467#>[>`>`>b:cc + F)/:
!>!D!D!F F J
" %6646%j1!*-44_E FF DUCZCZC\ ]<64FF8d3t9o- . ]] ]s +C4)F)rC z nn.Module)__doc__
contextlibr utilsr r r torch.nnr0
get_logger__name__loggerboolr r
rM r+ r r rV s & /
H % B FD> FDD FD FDR4^r