K iddlZddlZddlmZmZmZddlZddlm Z m Z m Z m Z dejdefdZ ddeeejfdeeeefdefd Z ddeeejfd eeej$fdeeeefddfd Zd edeeejffd Zd eeej$fdeeejffdZej,ej.ej0ej2ej4ej6ej8ej:ej<ej>ej@e!ejDd Z#dedejHfdZ%deeejffdZ&dejde!fdZ'y)N)DictOptionalUnion) deserialize safe_open serializeserialize_filetensorreturnc\t|s|jd}|jS)NF)inplace)_is_little_endianbyteswaptobytes)r s W/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/safetensors/numpy.py_tobytesr s' V $/ >>  tensor_dictmetadatac |jDcic]3\}}||jj|jt |d5}}}t ||}t |}|Scc}}w)a& Saves a dictionary of tensors into raw bytes in safetensors format. Args: tensor_dict (`Dict[str, np.ndarray]`): The incoming tensors. Tensors need to be contiguous and dense. metadata (`Dict[str, str]`, *optional*, defaults to `None`): Optional text only metadata you might want to save in your header. For instance it can be useful to specify more about the underlying tensors. This is purely informative and does not affect tensor loading. Returns: `bytes`: The raw bytes representing the format Example: ```python from safetensors.numpy import save import numpy as np tensors = {"embedding": np.zeros((512, 1024)), "attention": np.zeros((256, 256))} byte_data = save(tensors) ``` dtypeshapedatar)itemsrnamerrrbytes)rrkv flattened serializedresults rsaver$sl: %%' Aq QWW\\AGGXa[ III9x8J : F M s8A)filenamec |jDcic]3\}}||jj|jt |d5}}}t |||ycc}}w)aj Saves a dictionary of tensors into raw bytes in safetensors format. Args: tensor_dict (`Dict[str, np.ndarray]`): The incoming tensors. Tensors need to be contiguous and dense. filename (`str`, or `os.PathLike`)): The filename we're saving into. metadata (`Dict[str, str]`, *optional*, defaults to `None`): Optional text only metadata you might want to save in your header. For instance it can be useful to specify more about the underlying tensors. This is purely informative and does not affect tensor loading. Returns: `None` Example: ```python from safetensors.numpy import save_file import numpy as np tensors = {"embedding": np.zeros((512, 1024)), "attention": np.zeros((256, 256))} save_file(tensors, "model.safetensors") ``` rrN)rrrrrr )rr%rrr r!s r save_filer'4s^B %%' Aq QWW\\AGGXa[ III9h: s8Arc.t|}t|S)a Loads a safetensors file into numpy format from pure bytes. Args: data (`bytes`): The content of a safetensors file Returns: `Dict[str, np.ndarray]`: dictionary that contains name as key, value as `np.ndarray` on cpu Example: ```python from safetensors.numpy import load file_path = "./my_folder/bert.safetensors" with open(file_path, "rb") as f: data = f.read() loaded = load(data) ``` )r_view2np)rflats rloadr+Zs. t D D>rci}t|d5}|jD]}|j|||< ddd|S#1swY|SxYw)a Loads a safetensors file into numpy format. Args: filename (`str`, or `os.PathLike`)): The name of the file which contains the tensors Returns: `Dict[str, np.ndarray]`: dictionary that contains name as key, value as `np.ndarray` Example: ```python from safetensors.numpy import load_file file_path = "./my_folder/bert.safetensors" loaded = load_file(file_path) ``` np) frameworkN)r offset_keys get_tensor)r%r#frs r load_filer2usY(F 8t ,( (A QF1I (( M( Ms *AA) F64F32F16I64U64I32U32I16U16I8U8BOOLC64 dtype_strct|SN)_TYPES)r@s r _getdtyperDs ) rci}|D]D\}}t|d}tj|d|j|d}|||<F|S)Nrr)rr)rDr- frombufferreshape)safeviewr#rr rarrs rr)r)s[ F1!G*%mmAfIU3;;AgJGq  Mrc|jj}|dk(rtjdk(ryy|dk(ry|dk(ry|dk(rytd|) N=littleTF|<>zUnexpected byte order )r byteordersys ValueError)r rPs rrrs_ &&IC ==H $ c  c  c  -i[9 ::rrB)(osrQtypingrrrnumpyr- safetensorsrrrr ndarrayrrstrr$PathLiker'r+r2float64float32float16int64uint64int32uint32int16uint16int8uint8bool complex64rCrrDr)rrrrhs ((IIRZZENR!c2::o&!2:4S>2J! !N*.#;c2::o&#;C$%#;tCH~&#; #;Luc2::o.6c2;;./Dbjj4I8 :: :: :: 88 99 88 99 88 99 '' ((  << "$sBJJ/ ;bjj ;T ;r