Mpj/dZddlmZddlZddlmZddlmZddlm Z ddl m Z m ZmZd ZGd d ZGd d ZGddZGddZd"dZdZGddZGddeZdZdZdZdZdZdZdZdZ d Z!d!Z"y)#z$This module implements a CYK parser.) defaultdictN) ParseError)Token)Tree)Terminal NonTerminalSymbolcXt|tsJ|j|jk(SN) isinstanceTnametype)tss ^/mnt/ssd/data/Dropbox/adrian/scripts/msg_venv/lib/python3.12/site-packages/lark/parsers/cyk.pymatchrs% a   66QVV c@eZdZdZfdZdZdZdZdZdZ xZ S)RulezContext-free grammar rule.ctt| t|tsJ|t d|DsJ|||_||_||_||_ y)Nc3`K|]&}t|txst|t(ywr )r NTr.0xs r z Rule.__init__..s&FQ:a$8 1a(88Fs,.) superr__init__r ralllhsrhsweightalias)selfr"r#r$r% __class__s rr z Rule.__init__sZ dD"$#r"'C'"F#FFKKF  rcvt|jddjd|jDS)Nz ->  c32K|]}t|ywr strrs rrzRule.__str__.."s4NSV4N)r,r"joinr#r&s r__str__z Rule.__str__!s( ]CHH4NTXX4N,NOOrct|Sr r+r/s r__repr__z Rule.__repr__$ 4yrcVt|jt|jfSr )hashr"tupler#r/s r__hash__z Rule.__hash__'sTXXuTXX/00rcj|j|jk(xr|j|jk(Sr )r"r#r&others r__eq__z Rule.__eq__*s'xx599$>UYY)>>rc||k( Sr r9s r__ne__z Rule.__ne__-sEM""r) __name__ __module__ __qualname____doc__r r0r2r7r;r> __classcell__r's@rrrs%$P1?#rrc(eZdZdZdZdZdZdZy)GrammarzContext-free grammar.c$t||_yr ) frozensetrules)r&rIs rr zGrammar.__init__4su% rc4|j|jk(Sr )rIr9s rr;zGrammar.__eq__7szzU[[((rcdddjtd|jDzdzS)N c32K|]}t|ywr )reprrs rrz"Grammar.__str__..;s&C1tAw&Cr-)r.sortedrIr/s rr0zGrammar.__str__:s*dii&C &C CDDtKKrct|Sr r+r/s rr2zGrammar.__repr__=r3rN)r?r@rArBr r;r0r2r=rrrFrF1s&)LrrFceZdZdZddZdZy)RuleNodez@A node in the parse tree, which also contains the full rhs rule.c.||_||_||_yr )rulechildrenr$)r&rTrUr$s rr zRuleNode.__init__Es    rcdt|jjddjd|jDdS)Nz RuleNode(z, [z, c32K|]}t|ywr r+rs rrz$RuleNode.__repr__..KsEdQRc!fEdr-z]))rNrTr"r.rUr/s rr2zRuleNode.__repr__Js0'+DIIMM':DIIEdVZVcVcEd.[sFQ:a(Frr$r%)r originrr! expansionroptionspriority)r& lark_rules rr]zParser._to_ruleXs)**B///F)2E2EFFFF   i111:1B1B1K1K9$$-- QR rcsJtt||j\}}tfd|dt |dz fDr t d|dt |dz f}|j t|S)z(Parses input, which is a list of tokens.c3<K|]}|jk7ywr )r")rrstarts rrzParser.parse..hsF!quu~FsrzParsing failed.)r_parser_r!lenr_to_tree revert_cnf)r& tokenizedrltabletreesparses ` rruz Parser.parseas u5 i6 u FuaY!1C-D'EF F./ /q#i.1,-.u5}}Z.//rc||j|jj}g}|jD]j}t |t r!|j |j|4t |jtsJ|j |jlt|j|}||_|S)z.Converts a RuleNode parse tree to a lark Tree.) r\rTr%rUr rRappendrprrrrd)r& rule_node orig_rulerUchildrs rrpzParser._to_treemsOOINN$8$89 '' ,E%* e 45!%**e444 +  , !!8 ,r) r?r@rArBr r]rurprCrDs@rrZrZOs.  0 rrZct|trRtd|dzzt|jj z|j D]}t||dzytd|dzzt|jzy)Nr)rrm) r rRprintr,rTr"rU print_parser)nodeindentrzs rr}r}|sp$! cVaZ 3tyy}}#556]] +E vz * + cVaZ 3tvv;./rc tt}tt}t|D]\}}|jj D]\}}t ||s|D]}|||fj||j|||fvs,|j|||f|jjksXt|t|g|j|||f|j<tdt|dzD]z} tt|| z dzD]Y}t|dz|| zD]@} || dz f} | || zdz f} tj|| || D] \} }|j j#| j|jfgD]}|||| zdz fj||| | j}|| |j}|j|jz|jz}|j|||| zdz fvs(||||| zdz f|jjkst|||g||||| zdz f|j<C\}||fS)z*Parses sentence 's' using CNF grammar 'g'.r$rrm)rsetdict enumerateterminal_rulesitemsraddr"r$rRrrangero itertoolsproductnonterminal_rulesget)rgrsrtiwterminalrIrTlpspan1span2r1r2r1_treer2_treerule_total_weights rrnrns  E  E! ]1 //557 ]OHeXq!!]D1a&M%%d+q!f 5 eQFmDHH&=&D&DD2:4!A$PTP[P[2\q!f dhh/ ] ]]1c!fqj !{s1vzA~& {A1q5!a%( {AE AEAI'//e eElK{FB ! 3 3 7 78H" M{q!a%!)n-11$7"',rvv"6"',rvv"6,0KK'..,H7>>,Y) HHE1a!eai.,AA05!QUQY3H3R3Y3YY>FtgW^M_hy>zE1a!eai.1$((;{{ { {{ %<rc.eZdZdZfdZdZdZxZS) CnfWrapperzkCNF wrapper for grammar. Validates that the input grammar is CNF and provides helper data structures. ctt| ||_|j|_t t |_t t |_|jD]}t|jtsJ|t|jdvr tdt|jdk(rIt|jdtr,|j|jdj!|t|jdk(rOt#d|jDr3|jt%|jj!|J|y)N)rmrzCYK doesn't support empty rulesrmrrc3<K|]}t|tywr )r rrs rrz&CnfWrapper.__init__..s(JqAr):(Jrb)rrr r_rIrlistrrr r"rror#rrrwr!r6)r&r_rkr's rr zCnfWrapper.__init__s j$(* ]] )$/!,T!2 AaeeR( +! +(155z' !BCC155zQ:aeeAh#:##AEE!H-44Q7QUUqS(JAEE(J%J&&uQUU|4;;A>au rc4|j|jk(Sr )r_r9s rr;zCnfWrapper.__eq__s||u}},,rc,t|jSr )rNr_r/s rr2zCnfWrapper.__repr__sDLL!!r)r?r@rArBr r;r2rCrDs@rrrs  $-"rrc@eZdZdZfdZdZejZxZS) UnitSkipRulez@A rule that records NTs that were skipped during transformation.c@tt| ||||||_yr )rrr skipped_rules)r&r"r#rr$r%r's rr zUnitSkipRule.__init__s  lD*3VUC*rcbt|t|xr|j|jk(Sr )r rrr9s rr;zUnitSkipRule.__eq__s)%d,Z1C1CuGZGZ1ZZr) r?r@rArBr r;rr7rCrDs@rrrsJ+[}}Hrrc.g}t|tr||jz }|j|t|tr||jz }t|j|j ||j |j z|jS)Nrc)r rrrwr"r#r$r%) unit_rule target_rulers rbuild_unit_skiprulersM)\*000 %+|,222  { &-- 0B0BB)// [[rc|jD]<}t|jdk(st|jdts:|cSy)zDReturns a non-terminal unit rule from 'g', or None if there is none.rmrN)rIror#r r)rrTs rget_any_nt_unit_rulers@ txx=A *TXXa[""=K rc|jDcgc] }||k7s | }}|jDcgc]!}|j|jdk(s |#}}||Dcgc]}t||c}z }t |Scc}wcc}wcc}w)zFRemoves 'rule' from 'g' without changing the language produced by 'g'.r)rIr"r#rrF)rrTr new_rulesrefsrefs r_remove_unit_rulers|GG1qqDy1I1ww 7!!%%488A;"6A 7D 7 DAS%dC0AAI 9 2 7As A>A>!BBBc #jKt|jdzdjd|jDz}d|zdz}t |j|jdt |dzg|j |jtdt|jd z D]=}t t ||z|j|t ||dzzgdd ?t t |t|jd z z|jd d dd y w) z4Splits a rule whose len(rhs) > 2 into shorter rules.___c32K|]}t|ywr r+rs rrz_split..s.H!s1v.Hr-z__SP_%sz_%drrmrcrSplitN) r,r"r.r#rrr$r%rro)rTrule_str rule_namers r_splitrs488}t#chh.Htxx.H&HHHX&.I txx$((1+r)a-'89$++UYU_U_ `` 1c$((ma' (g2i!m$txx{ByAE7J4K&LUV^effg r)s488}q012DHHRSM!SZ [[sD1D3c |jDchc]&}|jD]}t|ts|(}}}|Dcic](}|t t dt |z|gdd*}}g}|jD]tjdkDrtdjDrjDcgc]#}t|tr||jn|%}}|jt j|jj|jfd|jD|jt|Scc}}wcc}wcc}w)z/Applies the TERM rule on 'g' (see top comment).z__T_%srTermrcrmc3<K|]}t|tywr )r rrs rrz_term..s$H!Z1%5$Hrbc3FK|]\}}|jvs|ywr )r#)rkvrTs rrz_term..sL41aa488mQLs!!)rIr#r rrrr,roanyr"rwr$r%extendrrF)rrTrall_trt_rulesrnew_rhss ` r_termr s577 I4DHH Iq 1a8HQ IQ IE ITYZqq$r(SV+,qc!6JJZGZI# txx=1 $Htxx$H!HJN((SQAq)9wqz~~q@SGS   T$((GDKKtzzZ [   L7==?L L   T " # 9  JZTs$E2E2-E8(E=cg}|jD]:}t|jdkDr|t|z }*|j |<t |S)z.Applies the BIN rule to 'g' (see top comment).r)rIror#rrwrF)rrrTs r_binrsSI# txx=1   %I   T " # 9 rcTt|}|rt||}t|}|r|S)z/Applies the UNIT rule to 'g' (see top comment).)rr)r nt_unit_rules r_unitr#s1'*L  a .+A.   HrcRttt|}t|S)z>Creates a CNF grammar from a general context-free grammar 'g'.)rrrr)rs rr^r^,s d58nA a=rc $|stt||||||S||djz }tt||djg||t |dj||dd||dj|dj g|S)Nrcrrrm)rRrr$r"unroll_unit_skipruler%)r"orig_rhsrrUr$r%s rrr2s S(6GZ`aa-*111 }Q'++,V5 I$]1%5%9%98 -ab 18 -a 0 7 7q9I9O9OQL  rct|tr|S|jjjj dr|j dSg}tt|j D]b}t|tr?|jjjj dr||j z }R|j|dt|jtrtt|jj|jj|jj||jj|jj St|j|S)zDReverts a parse tree (RuleNode) to its original non-CNF form (Node).__T_r__SP_)r rrTr"r startswithrUmaprqrRrwrrr#rr$r%)r~rUrzs rrqrq?s$  yy}}$$V,}}QT]]3 'E%*uzz~~/B/B/M/Mg/VENN*&  ' dii .' tyy}}$(II$;$;X$(II$4$4diiooG GDIIx0 0rrX)#rB collectionsrr exceptionsrlexerrtreerr_rrr rr rrrFrRrZr}rnrrrrrrrrrr^rrqr=rrrs*$#>> ##8  " f f**Z0"f""> 4 [\   1r