L iddlZddlmZddlmZddlmZerddlmZddl m Z dd l m Z m Z mZmZdd lmZerddlZej&eZGd d eZy) N) TYPE_CHECKING)version) HfQuantizer)PreTrainedModel)replace_with_aqlm_linear)is_accelerate_availableis_aqlm_availableis_torch_availablelogging)QuantizationConfigMixincveZdZdZdZdgZdZdeffd ZdZ dd Z dd Z dd Z e defd Zdd ZxZS)AqlmHfQuantizerzS Quantizer of the AQLM method. Enables the loading of prequantized models. TaqlmNquantization_configc 4t||fi|||_yN)super__init__r)selfrkwargs __class__s l/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/transformers/quantizers/quantizer_aqlm.pyrzAqlmHfQuantizer.__init__-s ,77#6 cXts tdts tdy)NzGUsing `aqlm` quantization requires Accelerate: `pip install accelerate`zDUsing `aqlm` quantization requires AQLM: `pip install aqlm[gpu,cpu]`)r ImportErrorr )rargsrs rvalidate_environmentz$AqlmHfQuantizer.validate_environment1s+&(gh h "de e#rreturnc|jtjjr'tj}tj d|Stj }tj d|S)NzCUDA available. Assuming AQLM inference on GPU and loading the model in `torch.float16`. To overwrite it, set `dtype` manually.zCUDA is unavailable. Assuming AQLM inference on CPU and loading the model in `torch.float32`. To overwrite it, set `dtype` manually.)torchcuda is_availablefloat16loggerinfofloat32)rdtypes r update_dtypezAqlmHfQuantizer.update_dtype8s] =zz&&(  V   [ rc t||j|jj|j|j_y)N)rlinear_weights_not_to_quantize)r rr,configrmodelrs r$_process_model_before_weight_loadingz4AqlmHfQuantizer._process_model_before_weight_loadingFs; !  $ 8 8+/+C+C+b+b ,0+C+C (rc |Srr.s r#_process_model_after_weight_loadingz3AqlmHfQuantizer._process_model_after_weight_loadingRs rctjtjjdtjdk\}|rytj dtjjddy)Nrz1.0.2Tz$Currently installed `aqlm` version (zw) doesn't support training. If you wish to train a quantized model, please update `aqlm` with `pip install aqlm>=1.0.2`F)rparse importlibmetadatar&warning)raqlm_supports_trainings r is_trainablezAqlmHfQuantizer.is_trainableUsr!(y/A/A/I/I&/Q!RV]VcVcdkVl!l ! NN6y7I7I7Q7QRX7Y6Z[RS rcy)NTr2)rsafe_serializations ris_serializablezAqlmHfQuantizer.is_serializable`sr)r) torch.dtyper r>)r/rr)__name__ __module__ __qualname____doc__requires_calibrationrequired_packagesoptimum_quantizerrrrr*r0r3propertyboolr:r= __classcell__)rs@rrr$sj 7,C7f  D  Ddrr)r6typingr packagingrbasermodeling_utilsr integrationsr utilsr r r r utils.quantization_configrr" get_loggerr?r&rr2rrrQsP 03[[?   H %=k=r