K i8dZddlmZddlmZmZmZmZmZm Z ddl m Z m Z edZ Gddee ZGdd ZGd d ee ZGd d ZdZdZdZdZdedefdZdZdZdZy)a This module defines the data structures used to represent a grammar. Specifying grammars in pgen is possible with this grammar:: grammar: (NEWLINE | rule)* ENDMARKER rule: NAME ':' rhs NEWLINE rhs: items ('|' items)* items: item+ item: '[' rhs ']' | atom ['+' | '*'] atom: '(' rhs ')' | NAME | STRING This grammar is self-referencing. This parser generator (pgen2) was created by Guido Rossum and used for lib2to3. Most of the code has been refactored to make it more Pythonic. Since this was a "copy" of the CPython Parser parser "pgen", there was some work needed to make it more readable. It should also be slightly faster than the original pgen2, because we made some optimizations. ) literal_eval)TypeVarGenericMappingSequenceSetUnion) GrammarParserNFAState _TokenTypeTc>eZdZdZdedeeedfdeedffdZy) Grammara Once initialized, this class supplies the grammar tables for the parsing engine implemented by parse.py. The parsing engine accesses the instance variables directly. The only important part in this parsers are dfas and transitions between dfas. start_nonterminal rule_to_dfaszDFAState[_TokenTypeT]reserved_syntax_stringsReservedStringc.||_||_||_yN)nonterminal_to_dfasrr)selfrrrs [/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/parso/pgen2/generator.py__init__zGrammar.__init__/s$0 '>$!2N)__name__ __module__ __qualname____doc__strrrrrrrr%sA3$'3&sH5L,M'MN3+2#7G2G*H3rrc0eZdZdZgfdddedfdZdZy)DFAPlanzj Plans are used for the parser to create stack nodes and do the proper DFA state transitions. next_dfaDFAState dfa_pushesc ||_||_yr)r"r$)rr"r$s rrzDFAPlan.__init__=s  $rch|jjd|jd|jdS)N(z, )) __class__rr"r$rs r__repr__zDFAPlan.__repr__As!#~~66 tWWrN)rrrrrrr+rrrr!r!8s,QS%%*9M%Xrr!cBeZdZdZdedeedefdZdZdZ dZ d Z y ) r#aa The DFAState object is the core class for pretty much anything. DFAState are the vertices of an ordered graph while arcs and transitions are the edges. Arcs are the initial edges, where most DFAStates are not connected and transitions are then calculated to connect the DFA state machines that have different nonterminals. from_rulenfa_setfinalct|tsJttt|tsJt|tsJ||_||_i|_i|_i|_ ||v|_ yr) isinstancesetnextiterr r-r.arcsnonterminal_arcs transitionsis_final)rr-r.r/s rrzDFAState.__init__Osr'3'''$tG}-x888%***" ,. 9;RT( rct|tsJ||jvsJt|tsJ||j|<yr)r1rr5r#)rnext_labels radd_arczDFAState.add_arc`sB%%%%DII%%%%***  %rcp|jjD]\}}||us ||j|<yr)r5items)roldnewr;r:s r unifystatezDFAState.unifystatefs5 IIOO- 'LE5|#& %  'rc6t|tsJ|j|jk7ryt|jt|jk7ry|jj D]$\}}||jj |us$yy)NFT)r1r#r8lenr5r>get)rotherr;r:s r__eq__zDFAState.__eq__ks}%*** ==ENN * tyy>S_ , IIOO- LE5EJJNN511 rcjd|jjd|jd|jdS)N)r)rr-r8r*s rr+zDFAState.__repr__ys% NN # #T^^T]]  rN) rrrrrrr rr<rArFr+rrrr#r#Es9)#)H )h)"! '  rr#c"eZdZdZdefdZdZy)rz Most grammars will have certain keywords and operators that are mentioned in the grammar as strings (e.g. "if") and not token types (e.g. NUMBER). This class basically is the former. valuec||_yr)rK)rrKs rrzReservedString.__init__s  rcN|jjd|jdS)Nr'r()r)rrKr*s rr+zReservedString.__repr__s>>22DJJ??rN)rrrrrrr+rrrrrs c@rrcd}|r`d}t|D]L\}}t|dzt|D],}||}||k(s||=|D]}|j||d}LN|r_yy)a This is not theoretically optimal, but works well enough. Algorithm: repeatedly look for two states that have the same set of arcs (same labels pointing to the same nodes) and unify them, until things stop changing. dfas is a list of DFAState instances TFN) enumeraterangerCrA)dfaschangesistate_ijstate_jstates r_simplify_dfasrYsG #D/ JAw1q5#d), q'g%Q!%;((':;"G   rcb t|tsJt|tsJ fd t} ||t|j||g}|D]}i}|j D]W}|j D]F}|j|j|jt} |j|HY|jD]W\} }|D]} | j |k(sn)t|j||} |j| |j| | Y|S)z Uses the powerset construction algorithm to create DFA states from sets of NFA states. Also does state reduction if some states are not needed. ct|tsJ||vry|j||jD]"}|j|j |$yr)r1r addr5nonterminal_or_stringr3) nfa_state base_nfa_setnfa_arc addclosures rraz_make_dfas..addclosuresY)X...  $ # ~~ 7G,,47<<6 7r) r1r r2r#r-r.r5r] setdefaultr3r>appendr<) startfinishr_statesrXr5r^r`r.r] nested_stateras @r _make_dfasrhs1 eX && & fh '' '75Lul#u f= >F? 6I$>> 600<"oog.K.KSUSGw||W5 6 6/3jjl ? * !7 & , ''72 , (&I  l+ MM,(= > ??, Mrctd|j|g}t|D]\}}td|||uxrdxsd|jD]m}|j|j }}||vr|j |}nt|}|j||td|z^td||fzoy)NzDump of NFA for State(final)z -> %d %s -> %d) printr-rPr5r]r3indexrCrc) rdretodorTrXarcr;r:rVs r _dump_nfarrs U__- 7DdO 35 iEVO9 ?R@:: 3C44chh5E}JJu%I E"}kAo&nqz12 3 3rc td|djt|D]e\}}td||jxrdxsd|jj D]$\}}td||j |fz&gy)NzDump of DFA forrrjrkrlrm)rnr-rPr8r5r>ro)rRrTrX nonterminalr:s r _dump_dfasrus T!W../dOE5 iENN8y>B?"'**"2"2"4 E K .KE1B#CC D EEr bnf_grammarreturnci}d}t|jD]:\}}t||}t||||j<|/|j}<i}|j D]g\}}|D]]} | j j D]>\} } | |vr| | j| <t||| } t| | j| <@_it|t|||S)a ``bnf_text`` is a grammar in extended BNF (using * for repetition, + for at-least-once repetition, [] for optional parts, | for alternatives and () for grouping). It's not EBNF according to ISO/IEC 14977. It's a dialect Python uses in its own parser. N) r parserhrYr-r>r5r6_make_transitionr!r7_calculate_tree_traversalr) rvtoken_namespacerrnfa_anfa_zrRreserved_stringsrt dfa_stateterminal_or_nonterminalr" transitions rgenerate_grammarrs!L%k288: 0 u%' t(, U__%  $ %  068)//1 J T JI5>^^5I5I5K J1'*l:JRI../FG!1'(/"J 9@8II))*5 J J Jl+ $l4D EErc|djr t||S|ddvsJ||jds|jdrJt|} ||S#t$rt |x}||<|cYSwxYw)z Creates a reserved string ("if", "for", "*", ...) or returns the token type (NUMBER, STRING, ...) for a given grammar terminal. r)"'z"""z''')isalphagetattr startswithrKeyErrorr)r|rr;rKrs rrzrzs  Qx..Qx:%,u,%##E*53C3CE3JJJU# *51 1 1?1F FA'.H sA##BBc i}t|j}|j|D]}||vst||||j D]}|D]}|j }|j jD]\}}||jD]\}} ||vr||} t| jr| jdjn| jj| r| djn |jg} td|j|ft| zzt|| ||<y)z By this point we know how dfas can move around within a stack node, but we don't know how we can add a new stack node (nonterminal transitions). rzZRule %s is ambiguous; given a %s token, we can't determine if we should evaluate %s or %s.N)listkeyssort_calculate_first_plansvaluesr7r6r>sortedr$r-r" ValueErrortupler!) r first_plans nonterminalsrtrRrr7r"rpushes prev_planchoicess rr{r{.s K+0023L#R k ) "#6 [ QR$**,H HI#//K)2)C)C)I)I)K H% X*5k*B*H*H*JH&J![0$/ $; #)$-#7#7!* 4 4Q 7 A A%.%7%7%A%A$*!'q 3 3/7/A/A * #)N%.$7$7$.!"%*'N!3  /6h.GK +5H H HHrc||}i}d||<|d}|jjD]\}}|jg||<|jjD]<\}} ||} | t d|z| jD]\} } |g| z|| <>|||<|S#t $rt |||} YBwxYw)z Calculates the first plan in the first_plans dictionary for every given nonterminal. This is going to be used to know when to create stack nodes. Nrzleft recursion for rule %r)r7r>r"r6rrr) rrrtrRnew_first_plansrXrr: nonterminal2 first_plans2trs rrr`s  { +DO#K  GE"..4467 E',~~&6 #7 %55;;= 2 e M&|4L# != !KLL%++- 2IAv"'6!1OA  2 2 /K   b12E{T`aL bs%B$$B=<B=N)rastrtypingrrrrrr parso.pgen2.grammar_parserr r r rr!r#rrYrhrrrurrrzr{rrrrrs*BB>m$ 3gk"3& X X7 w{#7 t @ @./d3$E'F#'F7'FT(/Hdr