Mpjs"ddlmZmZddlZddlZddlZddlZddlZddlZddl Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZerddlmZddlmZddlmZddl m Z dd l!m"Z"dd l#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-dd l.m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3dd lm4Z4ddl5m6Z6m7Z7m8Z8m9Z9ddl:m;Z;mZ>m?Z?ddl@mAZAddl!mBZBmCZCmDZDmEZEddlFmGZG ddlHZHdZIGddeZKGdde(ZLhdZMdZNdZOe ddZPGdde(ZQy#eJ$rdZIY=wxYw) )ABCabstractmethodN)TypeVarTypeListDictIteratorCallableUnionOptionalSequenceTupleIterableIOAny TYPE_CHECKING Collection)InteractiveParser) ParseTree) Transformer)Literal)ParsingFrontend)ConfigurationError assert_configUnexpectedInput) SerializeSerializeMemoizerFSlogger TextOrSlice LarkInput) load_grammarFromPackageLoaderGrammarverify_used_filesPackageResource sha256_digest)Tree) LexerConf ParserConf_ParserArgType _LexerArgType)Lexer BasicLexer TerminalDef LexerThreadToken)ParseTreeBuilder)_validate_frontend_args_get_lexer_callbacks_deserialize_parsing_frontend_construct_parsing_frontend)RuleTFcHeZdZUedeedeefdZdZee e d<y)PostLexstreamreturnc|SN)selfr;s W/mnt/ssd/data/Dropbox/adrian/scripts/msg_venv/lib/python3.12/site-packages/lark/lark.pyprocesszPostLex.process)s r? always_acceptN) __name__ __module__ __qualname__rr r2rBrDrstr__annotations__r?rCrAr:r:(s:huo(5/$&M8C=%rCr:c eZdZUdZeeed<eed<eed<ded<eeefed<eed<eeefed <eed <eed <e ed <eed <e e eege fed<e ed<eed<ded<e eed<ded<eee egeffed<eed<eed<e e egefed<ded<e eed<dZereez Ziddddd dddd dd ddddd dd!dddddd!dd!d ddddidd"dd#dd"gdid$Zeee fed%<d&eee fd'dfd(Zd)ed'e fd*Zd)ed+ed'dfd,Zd1d'eee ffd-Zed.eee fd/ee eeeffd'dfd0Zy)2 LarkOptionsz$Specifies the options for Lark startdebugstrictzOptional[Transformer] transformerpropagate_positionsmaybe_placeholderscache cache_grammarregex g_regex_flagskeep_all_tokens tree_classparserlexerz0Literal["auto", "resolve", "explicit", "forest"] ambiguitypostlexz-Optional[Literal["auto", "normal", "invert"]]prioritylexer_callbacks use_bytes ordered_setsedit_terminalszUList[Union[str, Callable[[Union[None, str, PackageResource], str], Tuple[str, str]]]] import_paths source_patha? **=== General Options ===** start The start symbol. Either a string, or a list of strings for multiple possible starts (Default: "start") debug Display debug information and extra warnings. Use only when debugging (Default: ``False``) When used with Earley, it generates a forest graph as "sppf.png", if 'dot' is installed. strict Throw an exception on any potential ambiguity, including shift/reduce conflicts, and regex collisions. transformer Applies the transformer to every parse tree (equivalent to applying it after the parse, but faster) propagate_positions Propagates positional attributes into the 'meta' attribute of all tree branches. Sets attributes: (line, column, end_line, end_column, start_pos, end_pos, container_line, container_column, container_end_line, container_end_column) Accepts ``False``, ``True``, or a callable, which will filter which nodes to ignore when propagating. maybe_placeholders When ``True``, the ``[]`` operator returns ``None`` when not matched. When ``False``, ``[]`` behaves like the ``?`` operator, and returns no value at all. (default= ``True``) cache Cache the results of the Lark grammar analysis, for x2 to x3 faster loading. LALR only for now. - When ``False``, does nothing (default) - When ``True``, caches to a temporary file in the local directory - When given a string, caches to the path pointed by the string cache_grammar For use with ``cache`` option. When ``True``, the unanalyzed grammar is also included in the cache. Useful for classes that require the ``Lark.grammar`` to be present (e.g. Reconstructor). (default= ``False``) regex When True, uses the ``regex`` module instead of the stdlib ``re``. g_regex_flags Flags that are applied to all terminals (both regex and strings) keep_all_tokens Prevent the tree builder from automagically removing "punctuation" tokens (Default: ``False``) tree_class Lark will produce trees comprised of instances of this class instead of the default ``lark.Tree``. **=== Algorithm Options ===** parser Decides which parser engine to use. Accepts "earley" or "lalr". (Default: "earley"). (there is also a "cyk" option for legacy) lexer Decides whether or not to use a lexer stage - "auto" (default): Choose for me based on the parser - "basic": Use a basic lexer - "contextual": Stronger lexer (only works with parser="lalr") - "dynamic": Flexible and powerful (only with parser="earley") - "dynamic_complete": Same as dynamic, but tries *every* variation of tokenizing possible. ambiguity Decides how to handle ambiguity in the parse. Only relevant if parser="earley" - "resolve": The parser will automatically choose the simplest derivation (it chooses consistently: greedy for tokens, non-greedy for rules) - "explicit": The parser will return all derivations wrapped in "_ambig" tree nodes (i.e. a forest). - "forest": The parser will return the root of the shared packed parse forest. **=== Misc. / Domain Specific Options ===** postlex Lexer post-processing (Default: ``None``) Only works with the basic and contextual lexers. priority How priorities should be evaluated - "auto", ``None``, "normal", "invert" (Default: "auto") lexer_callbacks Dictionary of callbacks for the lexer. May alter tokens during lexing. Use with caution. use_bytes Accept an input of type ``bytes`` instead of ``str``. ordered_sets Should Earley use ordered-sets to achieve stable output (~10% slower than regular sets. Default: True) edit_terminals A callback for editing the terminals before parse. import_paths A List of either paths or loader functions to specify from where grammars are imported source_path Override the source of from where the grammar was loaded. Useful for relative imports and unconventional grammar loading **=== End of Options ===** FNearleyautoTr)r`rUr^r_rarb_plugins _defaults options_dictr<c*t|}i}|jjD]A\}}||vr1|j|}t |t r|dvrt |}n|}|||<Ct |dt r |dg|d<||jd<t|jd|jdk(r|jr td|jr|js td|rtd|jzy) N)rRr^rPrLoptions)rclalrcykNrczCannot specify an embedded transformer when using the Earley algorithm. Please use your transformer on the resulting parse tree, or use a different algorithm (i.e. LALR)z2cache_grammar cannot be set when cache is disabledzUnknown options: %s)dictrfitemspop isinstanceboolrH__dict__rrXrOrrSrRkeys)r@rgorinamedefaultvalues rA__init__zLarkOptions.__init__s  !^^113 "MD'qyd gt,=j1j KE!GDM " gg& , ' 01GG #* i  dkk#BC ;;( "t'7'7$&AB B   djj$%YZ Z $%:QVVX%EF F rCrtc` |jd|S#t$r}t|d}~wwxYw)Nri)rqKeyErrorAttributeError)r@rtes rA __getattr__zLarkOptions.__getattr__s6 $==+D1 1 $ # # $s - (-rvclt||jjd||j|<y)Nz,%r isn't a valid option. Expected one of: %s)rrirr)r@rtrvs rA __setattr__zLarkOptions.__setattr__s*dDLL--/1_`" TrCc|jSr>ri)r@memos rA serializezLarkOptions.serializes ||rCdatarc||Sr>r?)clsrrs rA deserializezLarkOptions.deserializes 4yrCr>)rErFrG__doc__rrHrIrpr intr r rr,r-r:rr2r0 OPTIONS_DOCrfrwr|r~r classmethodr8rr?rCrArKrK/s 9 K L((tSy)) s  K3+s"2344  AA g ==#x7788OX{m[&@ABBii#PKb;!!%! 5! d !  !  ! 4! (! ! t! ! F! V! ! u! 2!!" d#!$1!ItCH~6GT#s(^GG@$$$ ##C#D#S#XtCH~T#u[RVEV?W:W5X]jrCrK> rMrTr[rer^rWrOrUr]rP)rdnormalinvertN)rdresolveexplicitforest_TLark)boundc eZdZUdZeed<eed<ded<eed<eed<ded <ee ed <gd Z d4dZ eredejzz Zd5de d efdZd6dZd7dZd8deed d fdZedeed efdZdeeefdeeee effded efdZdeded efdZedZed9deeded e ed efd!Z!ed"gfdeed#ed$ed%d&d ef d'Z"d(Z#d5d)e$de d e%e&fd*Z'd+ed e fd,Z(d:d)e e)d-e ed d.fd/Z*d:d)e)d-e ed0d1d d2fd3Z+y );ra}Main interface for the library. It's mostly a thin wrapper for the many different parsers, and for the tree constructor. Parameters: grammar: a string or file-object containing the grammar spec (using Lark's ebnf syntax) options: a dictionary controlling various aspects of Lark. Example: >>> Lark(r'''start: "foo" ''') Lark(...) rbsource_grammarr%grammarrirYrrX terminals)rXrulesrir<Nc t||_|jjr|jdgz|_|jj}|rt rt}nt dt}|jj |j|_n|jj|_ |j}|}d}d}t|tr||_|jjr|j!s t#d|jj$r!|jj&dk7r t#dddj)fd |j+D}d d lm} ||z| ztt0j2dd z} t5| }t|jj$tr|jj$}n|jj$d ur t#d t7j8} t=j>d|jjrdnd| |gt0j2dd zz}|j} tAjB|d5} tEjFd|tI|tJz D]}||=| jMjOd}tQjR| }||jUdk(r=tW|r2tQjR| }|jX|fi| dddydddt_||j|jj`|jjb\|_2}nt|tfsJ||_2|jjhdk(r|jj&dk(rd|j_4n|jj&dk(rO|jjj'tEjldd|j_4nXd|j_4nF|jj&dk(rd|j_4nJ|jj&|jjh}t|tnrtq|trs3Jtu|d|jjjd|vr t#d |jjvdk(r+|jj&dk(r3d!|j_;n!tu|jj&d"d#|jjxdk(rd$|j_<|jjxtzvr)t#d%|jjxd&tz|jjvt|vr)t#d'|jjvd&t||jj&d(}nJ|jjj*tI|jjjj~}n tI}|jdj|jj|\|_B|_C|_D|jjr,|jD]}|jj||jDcic]}|j|c}|_F|jjxd)k(rr|jD]?}|jjx|jjx |j_<A|jD]}|jx |_<ng|jjxQ|jD]*}|jjxd|j_<,|jD] }d*|_< t|j||j|jjj|jj|jj|jj|jj+|_K|jj&r|j|_n|r|j|_4|rtEjFd,| tAjB|d-5} |J| j|jUddztQj| |j| tJdddyy#t$r d|_YwxYw#t$rYwxYw#t:$rd} YwxYw#1swY}xYw#tZ$rYt:$r!tEj\d|| |_YwxYwcc}w#1swYyxYw#t$r!}tEj\d.||Yd}~yd}~wwxYw)/Nrz?`regex` module must be installed if calling `Lark(regex=True)`.zz/Grammar must be ascii only, when use_bytes=Truerjz+cache only works with parser='lalr' for now)rOr[r]r`rec3JK|]\}}|vs |t|zywr>)rH).0kv unhashables rA z Lark.__init__..Fs%%b41aaWaNaaAh%bs ##r) __version__Tz"cache argument must be bool or strunknownz/.lark_%s_%s_%s_%s_%s.tmprSrRrbzLoading grammar from cache: %s utf8zN>NrPQ>R:SS,Q/ dll00#6#||11H||))501UVV-#*??#4 (2247R+/<<+E+E7T\^jVCmpm}m}A@AnBVC8C CH#ll /40 #A %ExP%(\4I%I.D ' .&'jjl&9&9%&@ ,2KKN)&,*=*=f*EEJ[\mJn17Q.&DJJ'9EWE" # # #.(4GT=M=Mt||OhOhjnjvjvkGkG(H $DL*gw/ //"DL <<   '||""f,%1 "$$0<<''3KK!XY)0DLL&)2DLL&$$-%, "1dll111u "" eT "eU+ ++ %!W X||##/I4F()rss << ! !V +||""h.)2 & $,,--/@CL M << F *$,DLL ! << (? ?$Y]YeYeYnYnqH&IJ J << ! !)A A$Z^ZfZfZpZpsK&LM M <<   & #  \\ ! ! - #DLL$8$8$F$F G  # :>9M9MdllN`N`bs9t6 D$6 << & &^^ / ++A. /48>>Ba B << H ,  C<<((4-1\\-B-B,BDLL) C /!%  / \\ " " *  1<<((4,0DLL) 1 " !  "$ 4+=+=t||?S?S ,,dll.H.HTXT`T`TjTjswsstGtG <<  ,,.DK **,DJ  LL6 A SWWXt,8'333GGL//7%?@KK A.IIa!67 88 A" .#-  .   >%-$- - # #) /$$%cemn$/DL /B CB88  S  !DhPQRR Ss f 8 f!f1<gB-g?gghh,Ah h ff! f.-f.1 g?gg g h%hh hhh i"h>>iz dont_ignorec^|j}|rddlm}||}d|_t|S)Nr)copyr?)rrignorer/)r@rrrs rArzLark._build_lexers.__  !j)J "J *%%rCcZi|_|jjdk7rt|j|jj xst |jj|jjdk7xr|jjdk(|jj|_ |jj|jj|_|jjt|jj|jy)Nrrjr) _callbacksrirZr3rrWr)rPrXrQ_parse_tree_buildercreate_callbackrOupdater5rr@s rA_prepare_callbackszLark._prepare_callbackss << ! !X -'7JJLL++3tLL44LL''61Zdll6L6LPZ6ZLL33 (D $#66FFt||G_G_`DO 3DLL4L4Ldnn]^rCc|jt|jj|jjt |j |j|jj}t|jj|jj|j||jS)Nr) rr4rirXrYr+rrrLr7r)r@ parser_confs rArzLark._build_parsers ! 3 3T\\5G5GH T__dll>P>PQ * LL   LL   OO LL   rCexclude_optionscH|jjdk7r td|jtt g\}}|r/|dj Dcic] \}}||vs ||c}}|d<tj||d|tjycc}}w)zgSaves the instance into the given file object Useful for caching and multiprocessing. rjz7Lark.save() is only implemented for the LALR(1) parser.rirr)protocolN) rirXNotImplementedErrormemo_serializer0r8rmrrHIGHEST_PROTOCOL)r@rrrmnrs rArz Lark.saves <<  & (%&_` `%%{D&9:a 04Y0E0E0Gd11TcKcq!tdDO T1-q6;R;RSes  B(BrcF|j|}|j|S)zfLoads an instance from the given file object Useful for caching and multiprocessing. __new__r)rrinsts rArz Lark.loads {{3zz!}rCrrctj|d|}|jxsi|_|jrtnt |_|j|_|j|_d|_ |j|_ |S)NrT) r*rr] callbacksrTrrr^rUskip_validationr[)r@rrrirs rA_deserialize_lexer_confzLark._deserialize_lexer_confsr**4 +=tD &66<" (/ u2 &00 #*#8#8  %) "$__ rCr@rc Zt|tr|}ntj|}|d}|d}|sJt j |t tdi}d|vrtj |d||_ t|d}t|tz ttjzr*tdjt|tz |j!|tj |||_|dDcgc]}t j ||c}|_d|_t)|j"j*|j"j,|j/|d ||j"|_|j0j2|_|j5|j2D cic]} | j6| c} |_t;|d ||j0|j<|j"|_|Scc}wcc} w) Nrr)r8r0rriz6Some options are not allowed when loading a Parser: {}rzrX)rorlrrrrr8r0r%rrrrKrfrformatrrirrbr4rXrYrrrrrtrr6r) r@rkwargsd memo_jsonrrrirrs rArz Lark._loads a A AAfI yy ,,YVa8bdfg  "..tIEDLtI' K/ /3{7L7L3M M$%]$fS[3H%HIK Kv"..w= 9=gGAd&&q$/G + 3 3T\\5G5GH66tH~tT\\Z22 !37>>Ba B3 N  OO OO LL    H Cs H#H(c P|j|}|j||dfi|S)Nrr)rrrrrs rA_load_from_dictzLark._load_from_dict=s+{{3tzz46A&AArCgrammar_filenamerel_toc |r?tjj|}tjj||}t |d5}||fi|cdddS#1swYyxYw)a&Create an instance of Lark with the grammar given by its filename If ``rel_to`` is provided, the function will find the grammar filename in relation to it. Example: >>> Lark.open("grammar_file.lark", rel_to=__file__, parser="lalr") Lark(...) r)encodingN)ospathdirnamerr)rr r ribasepathrs rArz Lark.openBs^ wwv.H!ww||H6FG  "V 4 %q$G$ % % %s  A""A+rpackage grammar_path search_pathsz Sequence[str]c t||}|d|\}}|jd||jdg|dj|||fi|S)akCreate an instance of Lark with the grammar loaded from within the package `package`. This allows grammar loading from zipapps. Imports in the grammar will use the `package` and `search_paths` provided, through `FromPackageLoader` Example: Lark.open_from_package(__name__, "example.lark", ("grammars",), parser=...) Nrbra)r$ setdefaultappend)rrrrripackage_loader full_pathtexts rAopen_from_packagezLark.open_from_packageTsc+7LA(|< 4=)4>2.&&~64#7##rCc~d|jd|jjd|jjdS)Nz Lark(open(z ), parser=z, lexer=z, ...))rbrirXrYrs rA__repr__z Lark.__repr__fs,=A=M=Mt||ObObdhdpdpdvdvwwrCrc"t|dr|r|j|}n |j}tj||}|j d}|j jr%|j jj|S|S)aOnly lex (and postlex) the text, without parsing it. Only relevant when lexer='basic' When dont_ignore=True, the lexer will return all tokens, even those marked for %ignore. :raises UnexpectedCharacters: In case the lexer cannot find a suitable match. rYN) hasattrrrYr1 from_textlexrir[rB)r@rrrY lexer_threadr;s rAr"zLark.lexjsxtW%%%k2EJJE",,UD9 !!$' <<  <<''//7 7 rCrtc |j|S)z Get information about a terminal)r)r@rts rA get_terminalzLark.get_terminal|s##D))rCrLrc<|jj||S)aRStart an interactive parsing session. Only works when parser='lalr'. Parameters: text (LarkInput, optional): Text to be parsed. Required for ``resume_parse()``. start (str, optional): Start symbol Returns: A new InteractiveParser instance. See Also: ``Lark.parse()`` )rL)rXparse_interactive)r@rrLs rAr'zLark.parse_interactives{{,,T,??rCon_errorz+Optional[Callable[[UnexpectedInput], bool]]rc|$|jjdk7r td|jj|||S)aParse the given text, according to the options provided. Parameters: text (LarkInput): Text to be parsed, as `str` or `bytes`. TextSlice may also be used, but only when lexer='basic' or 'contextual'. If Lark was created with a custom lexer, this may be an object of any type. start (str, optional): Required if Lark was given multiple possible start symbols (using the start option). on_error (function, optional): if provided, will be called on UnexpectedInput error, with the exception as its argument. Return true to resume parsing, or false to raise the exception. LALR only. See examples/advanced/error_handling.py for an example of how to use on_error. Returns: If a transformer is supplied to ``__init__``, returns whatever is the result of the transformation. Otherwise, returns a Tree instance. :raises UnexpectedInput: On a parse error, one of these sub-exceptions will rise: ``UnexpectedCharacters``, ``UnexpectedToken``, or ``UnexpectedEOF``. For convenience, these sub-exceptions also inherit from ``ParserError`` and ``LexerError``. rjz?The on_error option is only implemented for the LALR(1) parser.)rLr()rirXrparse)r@rrLr(s rAr*z Lark.parsesC*  DLL$7$76$A%&gh h{{  UX FFrC)rzUnion[Grammar, str, IO[str]]r<N)F)r<N)r<r)r?r>)NN),rErFrGrrHrIrKr.rr0rrwrrpr/rrrrrrrrrrrr r8r*rrr r rrrr!r r2r"r%r"r'r*r?rCrArrs?    L +&&9SB6K3333&&z& _   Tz# T T$r("DcN$sER]_cRcLdGdBep{AJ B 3 R DBB%$r(%c%8C=%]_%%"ikhl$tBx$#$S$Xg$|~$$"x $%$*** @hy&9 @xPS} @^q @G)GHSMGJwGBMGrC)Rabcrrrrrrrtypesrtypingrrrrr r r r r rrrrrrparsers.lalr_interactive_parserrtreervisitorsrrparser_frontendsr exceptionsrrrutilsrrrr r!r"r#r$r%r&r'r(r)commonr*r+r,r-rYr.r/r0r1r2parse_tree_builderr3r4r5r6r7rr8rTrrr:rKrrrrrr?rCrAr6s# B%1JJSSuuHHEE0HHJ&c&B)BNg<D T gG9gG{Js DD D