L i rddlmZmZddlmZddlmZmZmZeeddd Gd d eZ y ) )AnyUnion)add_end_docstrings) GenericTensorPipelinebuild_pipeline_init_argsTF) has_tokenizersupports_binary_outputa tokenize_kwargs (`dict`, *optional*): Additional dictionary of keyword arguments passed along to the tokenizer. return_tensors (`bool`, *optional*): If `True`, returns a tensor according to the specified framework, otherwise returns a list.c eZdZdZdZdZdZdZd dZde e e ffdZ dZ d dZd ee ee fd edeeeefffd ZxZS)FeatureExtractionPipelinea Feature extraction pipeline uses no model head. This pipeline extracts the hidden states from the base transformer, which can be used as features in downstream tasks. Example: ```python >>> from transformers import pipeline >>> extractor = pipeline(model="google-bert/bert-base-uncased", task="feature-extraction") >>> result = extractor("This is a simple test.", return_tensors=True) >>> result.shape # This is a tensor of shape [1, sequence_length, hidden_dimension] representing the input string. torch.Size([1, 8, 768]) ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This feature extraction pipeline can currently be loaded from [`pipeline`] using the task identifier: `"feature-extraction"`. All models may be used for this pipeline. See a list of all models, including community-contributed models on [huggingface.co/models](https://huggingface.co/models). FTc V|i}|d|vr td||d<|}i}|||d<|i|fS)N truncationz\truncation parameter defined twice (given as keyword argument as well as in tokenize_kwargs)return_tensors) ValueError)selfrtokenize_kwargsrkwargspreprocess_paramspostprocess_paramss o/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/transformers/pipelines/feature_extraction.py_sanitize_parametersz.FeatureExtractionPipeline._sanitize_parameters-sc  " O  !. r-7OL )+  %3A / 0 "&888returnc D|j|fd|ji|}|S)Nr) tokenizer framework)rinputsr model_inputss r preprocessz$FeatureExtractionPipeline.preprocess@s&%t~~f_T^^__ rc*|jdi|}|S)N)model)rr model_outputss r_forwardz"FeatureExtractionPipeline._forwardDs" 2\2 rc|r|dS|jdk(r|djS|jdk(r!|djjSy)Nrpttf)rtolistnumpy)rr%rs r postprocessz%FeatureExtractionPipeline.postprocessHs^  # # >>T ! #**, , ^^t # #))+224 4$rargsrc"t||i|S)a Extract the features of the input(s) text. Args: args (`str` or `list[str]`): One or several texts (or one list of texts) to get the features of. Return: A nested list of `float`: The features computed by the model. )super__call__)rr-r __class__s rr0z"FeatureExtractionPipeline.__call__Qsw000r)NNN)F)__name__ __module__ __qualname____doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerrdictstrrr!r&r,rlistrr0 __classcell__)r1s@rrrs0O!#O9&tCrAsD&CC4NkL1L1L1r