L i ddlmZmZddlmZerddlmZddlmZddl m Z m Z m Z m Z ddlmZe rdd lZe j"eZGd d eZy ) ) TYPE_CHECKINGOptional) HfQuantizer)PreTrainedModel)replace_with_spqr_linear)is_accelerate_availableis_spqr_availableis_torch_availablelogging)QuantizationConfigMixinNczeZdZdZdZdeffd ZdZddZ dddd e e e fd Z dd Z ed Zdd ZxZS)SpQRHfQuantizerzS Quantizer of the SpQR method. Enables the loading of prequantized models. Tquantization_configc 4t||fi|||_yN)super__init__r)selfrkwargs __class__s l/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/transformers/quantizers/quantizer_spqr.pyrzSpQRHfQuantizer.__init__(s ,77#6 ctjjs tdt s t dt s t dy)Nz,GPU is required to run SpQR quantized model.zGUsing `spqr` quantization requires Accelerate: `pip install accelerate`zFUsing `spqr` quantization requires SpQR: `pip install spqr_quant[gpu]`)torchcuda is_available RuntimeErrorr ImportErrorr )rargsrs rvalidate_environmentz$SpQRHfQuantizer.validate_environment,sGzz&&(MN N&(gh h "fg g#rc|'tj}tjd|S|tjk7r t d|S)NzHAssuming SpQR inference on GPU and loading the model in `torch.float16`.z|You cannot use any type other than torch.float16 for SpQR. Please either leave it None or set it totorch.float16 explicitly.)rfloat16loggerinfo ValueError)rdtypes r update_dtypezSpQRHfQuantizer.update_dtype6sK =MME KKb c  emm #,  rmodelrkeep_in_fp32_modulesc |j||jj||_t||j|j|j|j_y)N)rmodules_to_not_convert)get_modules_to_not_convertrr-r config)rr*r+rs r$_process_model_before_weight_loadingz4SpQRHfQuantizer._process_model_before_weight_loadingAs^ '+&E&E 4++BBDX' # !  $ 8 8#'#>#> ,0+C+C (rc |Sr)rr*rs r#_process_model_after_weight_loadingz3SpQRHfQuantizer._process_model_after_weight_loadingRs rcy)NFr2)rs r is_trainablezSpQRHfQuantizer.is_trainableUsrcy)NTr2)rsafe_serializations ris_serializablezSpQRHfQuantizer.is_serializableYsr)r( torch.dtypereturnr9r)r*r)__name__ __module__ __qualname____doc__requires_calibrationrrr"r)rliststrr0r3propertyr5r8 __classcell__)rs@rrr!sn 7,C7h 59D D'tCy1D"rr)typingrrbasermodeling_utilsr integrationsr utilsr r r r utils.quantization_configrr get_loggerr;r%rr2rrrKsJ+03[[?   H %9k9r