wL idZddlmZddlmZddlmZmZmZmZm Z m Z m Z e dZ gdZ ddgZd d d d d ee d e e eeefd e e eeefde ee gefdee d d ff dZdedefdZy )z6Contains utilities to handle paths in Huggingface Hub.fnmatch)Path)Callable GeneratorIterableListOptionalTypeVarUnionT).gitz.git/*z*/.gitz **/.git/**z.cache/huggingfacez.cache/huggingface/*z*/.cache/huggingfacez**/.cache/huggingface/**rz.cacheN)allow_patternsignore_patternskeyitemsrrrreturnc#Kt|tr|g}t|tr|g}||Dcgc] }t|}}||Dcgc] }t|}}|dtdtfd}|}|D]<}|||t fd|Ds"|t fd|Dr9|>ycc}wcc}ww)aFilter repo objects based on an allowlist and a denylist. Input must be a list of paths (`str` or `Path`) or a list of arbitrary objects. In the later case, `key` must be provided and specifies a function of one argument that is used to extract a path from each element in iterable. Patterns are Unix shell-style wildcards which are NOT regular expressions. See https://docs.python.org/3/library/fnmatch.html for more details. Args: items (`Iterable`): List of items to filter. allow_patterns (`str` or `List[str]`, *optional*): Patterns constituting the allowlist. If provided, item paths must match at least one pattern from the allowlist. ignore_patterns (`str` or `List[str]`, *optional*): Patterns constituting the denylist. If provided, item paths must not match any patterns from the denylist. key (`Callable[[T], str]`, *optional*): Single-argument function to extract a path from each item. If not provided, the `items` must already be `str` or `Path`. Returns: Filtered list of objects, as a generator. Raises: :class:`ValueError`: If `key` is not provided and items are not `str` or `Path`. Example usage with paths: ```python >>> # Filter only PDFs that are not hidden. >>> list(filter_repo_objects( ... ["aaa.PDF", "bbb.jpg", ".ccc.pdf", ".ddd.png"], ... allow_patterns=["*.pdf"], ... ignore_patterns=[".*"], ... )) ["aaa.pdf"] ``` Example usage with objects: ```python >>> list(filter_repo_objects( ... [ ... CommitOperationAdd(path_or_fileobj="/tmp/aaa.pdf", path_in_repo="aaa.pdf") ... CommitOperationAdd(path_or_fileobj="/tmp/bbb.jpg", path_in_repo="bbb.jpg") ... CommitOperationAdd(path_or_fileobj="/tmp/.ccc.pdf", path_in_repo=".ccc.pdf") ... CommitOperationAdd(path_or_fileobj="/tmp/.ddd.png", path_in_repo=".ddd.png") ... ], ... allow_patterns=["*.pdf"], ... ignore_patterns=[".*"], ... key=lambda x: x.repo_in_path ... )) [CommitOperationAdd(path_or_fileobj="/tmp/aaa.pdf", path_in_repo="aaa.pdf")] ``` Nitemrczt|tr|St|tr t|Std|d)Nz9Please provide `key` argument in `filter_repo_objects`: `z` is not a string.) isinstancestrr ValueError)rs b/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/huggingface_hub/utils/_paths.py _identityz&filter_repo_objects.._identityss=$$ $%4y XY]X^^pqr rc36K|]}t|ywNr.0rpaths r z&filter_repo_objects..s1[q'$2B1[c36K|]}t|ywrrrs rr#z&filter_repo_objects..s.YAwtQ/?.Yr$)rr_add_wildcard_to_directoriesr any)rrrrprrr"s @rfilter_repo_objectsr)'s~.#&()/3'*+!CQRa6q9RR"DSTq7:TT { sA s# s 4y  %c1[N1[.[   &3.Y.Y+Y   STs-B>B4B> B9A"B>patternc |ddk(r|dzS|S)N/*)r*s rr&r&sr{c} Nr)__doc__rpathlibrtypingrrrr r r r r DEFAULT_IGNORE_PATTERNSFORBIDDEN_FOLDERSrr)r&r/rrr5s=PPP CL X& 7;7;(, ` A;`U49c>23`eDIsN34 ` (A38$ % ` q$} `F##r