JL i92dZddlZddlmZGddeZy)u^ ISRI Arabic Stemmer The algorithm for this stemmer is described in: Taghva, K., Elkoury, R., and Coombs, J. 2005. Arabic Stemming without a root dictionary. Information Science Research Institute. University of Nevada, Las Vegas, USA. The Information Science Research Institute’s (ISRI) Arabic stemmer shares many features with the Khoja stemmer. However, the main difference is that ISRI stemmer does not use root dictionary. Also, if a root is not found, ISRI stemmer returned normalized form, rather than returning the original unmodified word. Additional adjustments were made to improve the algorithm: 1- Adding 60 stop words. 2- Adding the pattern (تفاعيل) to ISRI pattern set. 3- The step 2 in the original algorithm was normalizing all hamza. This step is discarded because it increases the word ambiguities and changes the original root. N)StemmerIcleZdZdZdZdZddZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZy) ISRIStemmera& ISRI Arabic stemmer based on algorithm: Arabic Stemming without a root dictionary. Information Science Research Institute. University of Nevada, Las Vegas, USA. A few minor modifications have been made to ISRI basic algorithm. See the source code of this module for more information. isri.stem(token) returns Arabic root for the given token. The ISRI Stemmer requires that all tokens have Unicode string types. If you use Python IDLE on Arabic Windows you have to decode text first using Arabic '1256' coding. czgd|_ddg|_gd|_gd|_gd|_gd|_dgd ggd d gd |_d d ggdgdgddd gd dgd dgd|_tjd|_ tjd|_ tjd|_ gd|_ y)N)uكالuبالuوللuوالuالuلل) uلuبuفuسويتنا)uتملuهملuتانuتينuكمل)uونuاتuانuينuتنuكمuهنuناuياuهاuتمuكنuنيuواuماuهم)ةuهruكr r r مr )r rrr )rr )r rr)r r r )r rr r)rrrrz[\u064B-\u0652]z[\u0621\u0624\u0626]z^[\u0622\u0623\u0625])1uيكونuوليسuوكانuكذلكuالتيuوبينu عليهاuمساءuالذيu وكانتuولكنu والتيuتكونu اليومu اللذينuعليهuكانتuلذلكuأمامuهناكuمنهاu مازالu لازالu لايزالu مايزالuاصبحuأصبحuأمسىuامسىuأضحىuاضحىu مابرحu مافتئu ماانفكu لاسيماuولايزالu الحاليu اليهاu الذينuفانهu والذيuوهذاuلهذاuفكانu ستكونuاليهuيمكنuبهذاuالذى)p3p2p1s3s2s1pr4pr53recompilere_short_vowelsre_hamzare_initial_hamza stop_words)selfs T/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/nltk/stem/isri.py__init__zISRIStemmer.__init__3s ">2    (Yzz-z  (#---(#(#(#   "zz*<= #:; " +C D2 c|j|d}||jvr|S|j|}|j|}|j |}|j|d}t |dk(r|j |}|St |dk(r$|j|}|j|}|St |dk(r$|j|}|j|}|St |dk(r`|j|}t |dk(r|j|}t |dk(r"|j|}|j|}|S)z? Stemming a word token using the ISRI stemmer. rrrrr) normr!pre32suf32wawlenpro_w4pro_w53end_w5pro_w6end_w6suf1pre1)r"tokens r#stemzISRIStemmer.stemsS 1  DOO #L       %# u:?KK&E Z1_LL'EKK&E Z1_KK&EKK&E Z1_IIe$E5zQ %(5zQ E* E* r%c |dk(r|jjd|}|S|dk(r|jjd|}|S|dk(r8|jjd|}|jjd|}|S)z normalization: num=1 normalize diacritics num=2 normalize initial hamza num=3 both 1&2 rrr r)rsubr )r"wordnums r#r(zISRIStemmer.norms !8''++B5D  AX((,,Xt $??4(8O $ t9> $??4(8O $ r%ct|dk\r)|jD]}|j|s|ddcSt|dk\r)|jD]}|j|s|ddcS|S)z9remove length three and length two suffixes in this orderrNr)r,rendswithr)r"r9suf3suf2s r#r*zISRIStemmer.suf32sw t9> %==&9$ % t9> %==&9$ % r%c<t|dk\r |dddk(r|dd}|S)uHremove connective ‘و’ if it precedes a word beginning with ‘و’rNruووr)r,r"r9s r#r+zISRIStemmer.waws* t9>d2Ah.88D r%cV|d|jdvr|dd}|S|d|jdvr |dd|ddz}|S|d|jdvr |dd|dz}|S|d|jdvr|dd}|S|j|}t|dk(r|j|}|S)z;process length four patterns and extract length three rootsrrNrrr)rr2r,r3rFs r#r-zISRIStemmer.pro_w4 s 7dhhqk !8D !W #8d12h&D !W #8d1g%D !W #9D  99T?D4yA~yy r%c\|d|jdvr|ddk(r |d|ddz}|S|d|jdvr|ddk(r |dd|dz}|S|d|jdvr|dd k(r|dd}|S|d|jdvr|dd k(r |d|ddz}|S|d|jdvr|ddk(r |d|ddz}|S|d|jd vr|dd k(r |dd|dz}|S|d|jd vr|dd k(r|dd}|S|ddk(r|ddk(r |dd|dz}|S|dd k(r|ddk(r|dd}|S|ddk(r|dd k(r |dd|dz}|S|ddk(r|ddk(r|d|dz|dz}|S|ddk(r|ddk(r |d|ddz}|S|ddk(r|ddk(r |dd|dz}|S|dd k(r|ddk(r |d|ddz}|S|ddk(r|ddk(r |dd|dz}|S|j|}t|d k(r|j|}|S)z;process length five patterns and extract length three rootsrrr rrNr rr r rrr rruئ)rr2r,r3rFs r#r.zISRIStemmer.pro_w53s^ 7diil "tAw(':7T!"X%DB A!W ! $aH)<!9tAw&D> =!W ! $aH)<!9D: 9!W ! $aH)<7T!"X%D6 5!W ! $aH)<7T!"X%D2 1!W ! $aH)<8d1g%D. -!W ! $aH)<8D* )!W T!W%8!9tAw&D& %!W T!W%88D" !!W T!W%8!9tAw&D !W T!W%87T!W$tAw.D !W T!W%87T!"X%D !W T!W%88d1g%D !W T!W%87T!AY&D !W T!W%88d1g%D  99T?D4yA~yy r%c|d|jdvr|dd}|S|ddk(r|dd}|S|ddk(r |dd|ddz}|S) z:process length five patterns and extract length four rootsrrrNrr r r)rrFs r#pro_w54zISRIStemmer.pro_w54@sq 7diil "8D  !W 8D !W 8d12h&D r%ct|dk(r|j|}|St|dk(r|j|}|S)z!ending step (word of length five)rr)r,r-rKrFs r#r/zISRIStemmer.end_w5JsB t9>;;t$D Y!^<<%D r%c|jds|jdr|dd}|S|ddk(r|ddk(r|dd k(r |d d|d z}|S|ddk(r#|d d k(r|d dk(r|d |dz|dz}|S|ddk(r |ddk(r|d |d k(r |d |d dz}|S|dd k(r#|d dk(r|d dk(r|d |dz|dz}|S|j|}t|dk(r|j|}|S)z:process length six patterns and extract length three rootsuاستuمستrNrr r rr rrrr rrr)r<r2r,r3rFs r#r0zISRIStemmer.pro_w6Rsk ??/ 0DOO 5 8D* ' Gx DGx$7DGx<<%D;;t$D Y!^<<%D r%cX|jD]}|j|s|ddcS|S)znormalize short sufixNrH)rrB)r"r9sf1s r#r2zISRIStemmer.suf1s477 !C}}S!CRy  ! r%cX|jD]}|j|s|ddcS|S)znormalize short prefixrN)rr<)r"r9sp1s r#r3zISRIStemmer.pre1s377 Cs#ABx  r%N)r)__name__ __module__ __qualname____doc__r$r5r(r)r*r+r-r.rKr/r0rOr1r2r3r%r#rr$sV B H#J    $L8r%r)rXr nltk.stem.apirrrYr%r#r[s * "g(gr%