U e@@s,dZddlZddlmZGdddeZdS)a[ ARLSTem2 Arabic Light Stemmer The details about the implementation of this algorithm are described in: K. Abainia and H. Rebbani, Comparing the Effectiveness of the Improved ARLSTem Algorithm with Existing Arabic Light Stemmers, International Conference on Theoretical and Applicative Aspects of Computer Science (ICTAACS'19), Skikda, Algeria, December 15-16, 2019. ARLSTem2 is an Arabic light stemmer based on removing the affixes from the words (i.e. prefixes, suffixes and infixes). It is an improvement of the previous Arabic light stemmer (ARLSTem). The new version was compared to the original algorithm and several existing Arabic light stemmers, where the results showed that the new version considerably improves the under-stemming errors that are common to light stemmers. Both ARLSTem and ARLSTem2 can be run online and do not use any dictionary. N)StemmerIc@seZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZddZddZd d!Zd"S)#ARLSTem2u Return a stemmed Arabic word after removing affixes. This an improved version of the previous algorithm, which reduces under-stemming errors. Typically used in Arabic search engine, information retrieval and NLP. >>> from nltk.stem import arlstem2 >>> stemmer = ARLSTem2() >>> word = stemmer.stem('يعمل') >>> print(word) عمل :param token: The input Arabic word (unicode) to be stemmed :type token: unicode :return: A unicode Arabic word cCstd|_td|_td|_ddddg|_dd d g|_d d g|_d ddg|_ddg|_ ddg|_ ddg|_ ddg|_ dddg|_ ddg|_ddg|_ddg|_dd g|_d!d"d#d$g|_d%d&g|_d'd(d)d*g|_d+d,d-g|_dS).Nz[\u0622\u0623\u0625]z[\u0649]z[\u064B-\u065F]uالuللuفلuفبuبالuكالuوالuفللuوللuفبالuوبالuفكالuكيuكمuهاuهمuكماuكنّuهماuهنّuانuينونuتانuتينuستuسيuساuسنuلنuلتuليuلأuتماuتنّناuتمuتاواتان)recompilere_hamzated_alifre_alifMaqsura re_diacriticspr2pr3pr32pr4su2su22su3su32pl_si2pl_si3verb_su2verb_pr2 verb_pr22 verb_pr33 verb_suf3 verb_suf2 verb_suf1)selfr!S/var/www/html/assets/scripts/venv/lib/python3.8/site-packages/nltk/stem/arlstem2.py__init__2s>               zARLSTem2.__init__c Csz|dkrtdd|_||}||}|dk r8|}||}|dk rP|WS||}|dk rh|WS||}||}|dkr|dkr||}|dk rd|_|WSn|WS|WStk r}z t |W5d}~XYnXdS)z: call this function to get the first stem NUThe word could not be stemmed, because it is empty !FT) ValueErrorZis_verbnormpreffem2masc adjectivesuff plur2singverbprint)r tokenprefmZadjZpsr,er!r!r"stem1ls8       zARLSTem2.stem1c Csz|dkrtd||}t|dkr|drX|ddkrX|dd|d}|WS|dr|dd kr|dd|d}|WSt|d kr|d s|dr|dd}|WS|d r|ddWS|WStk r}z t|W5d}~XYnXdS) Nr$rيموruل)r%r2len startswithendswithr-)r r.r1r!r!r"stems,     z ARLSTem2.stemcCsP|jd|}|jd|}|jd|}|drLt|dkrL|dd}|S)z normalize the word by removing diacritics, replace hamzated Alif with Alif bare, replace AlifMaqsura with Yaa and remove Waaw at the beginning. rr5r9r:r6N)rsubr r r<r;r r.r!r!r"r&s  z ARLSTem2.normcCst|dkr2|jD]}||r|ddSqt|dkrd|jD]}||rD|ddSqDt|dkr|jD]}||rv|ddSqvt|dkr|jD]}||r|ddSqdS)z< remove prefixes from the words' beginning. r:Nr3)r;rr<rrr)r r.Zp3Zp4p2r!r!r"r's             z ARLSTem2.prefcCsDt|dkr@|dr@|ddkr@|dr@|dd|dSdS)z4 remove the infixes from adjectives rBrr5Nr4r;r<r=rAr!r!r"r)s  zARLSTem2.adjectivecCs<|dr"t|dkr"|ddSt|dkrT|jD]}||r4|ddSq4t|dkr|jD]}||rf|ddSqf|d rt|dkr|dd}|St|dkr|jD]}||r|ddSqt|dkr|jD]}||r|ddSq|d r8t|dkr8|ddS|S) z= remove the suffixes from the word's ending. uكr:Nr7r3r4rBrFuهr)r=r;rrrr)r r.s2Zs3r!r!r"r*s0              z ARLSTem2.suffcCs t|dkrt|dr@|ddkr@|dr@|dd|dS|drt|ddkrt|drt|d d|dS|d rt|d kr|d d St|d kr|ddkr|dr|d|ddS|dr|d d S|drt|dkr|d dSd S)zR transform the word from the feminine form to the masculine form. rCrr5uيةr6rFrNuايةrBr4r3uةrrDr7r:rGrAr!r!r"r(s0       zARLSTem2.fem2masccCst|dkr,|dr,|dr,|ddSt|dkr^|jD]}||r>|ddSq>t|dkr|jD]}||rp|ddSqpt|dkr|d r|ddS|d r|d d kr|dd |d dS|d r|dd kr|dd|d SdS)zO transform the word from the plural form to the singular form. rBr8rr6r4r3NrFuاتrrDr:r7)r;r<r=rr)r r.Zps2Zps3r!r!r"r+)s$          zARLSTem2.plur2singcCs|||}|dk r|S||}|dk r,|S||}|dk rB|S||}|dk rX|S||}|dk rn|S||}|S)z= stem the verb prefixes and suffixes or both N)verb_t1verb_t2verb_t3verb_t4verb_t5verb_t6)r r.Zvbr!r!r"r,Ds"      z ARLSTem2.verbcCsZt|dkr<|dr<|jD]}||r|ddSqt|dkrx|drx|jD]}||rX|ddSqXt|dkr|drt|dkr|dr|ddS|dr|dd S|dr|dd S|d r|dd St|dkr$|dr$|d r$|dd St|dkrV|drV|d rV|dd Sd S) zJ stem the present tense co-occurred prefixes and suffixes rBrr6r4r5r3rrr7r N)r;r<rr=rr r.rHr!r!r"rJZs*           & &zARLSTem2.verb_t1cCst|dkr|jD].}||jdr||r|ddSq||jdrn||jdrn|ddS||jdr||jdr|ddSt|dkr||jdr|dr|ddSt|dkr||jdr|dr|ddSd S) zI stem the future tense co-occurred prefixes and suffixes rCrrDr4r6rBr r7N)r;rr<rr=rPr!r!r"rKzs,         zARLSTem2.verb_t2cCst|dkr2|jD]}||r|ddSqt|dkrd|jD]}||rD|ddSqDt|dkr|jD]}||rv|ddSqvdS)z1 stem the present tense suffixes rBNrFr3r4r:r7)r;rr=rr)r r.rrZsu1r!r!r"rLs         zARLSTem2.verb_t3cCsLt|dkrH|jD]}||r|ddSq|drH|ddSdS)z1 stem the present tense prefixes r:r6Nr5)r;rr<)r r.Zpr1r!r!r"rMs     zARLSTem2.verb_t4cCs\t|dkrX|jD]}||r|ddSq|jD]}||r8|ddSq8dS)z0 stem the future tense prefixes r3rDN)r;rr<r)r r.rr!r!r"rNs     zARLSTem2.verb_t5cCs6t|dkr2|jD]}||r|ddSq|S)z4 stem the imperative tense prefixes r3rDN)r;rr<)r r.rr!r!r"rOs    zARLSTem2.verb_t6N)__name__ __module__ __qualname____doc__r#r2r>r&r'r)r*r(r+r,rJrKrLrMrNrOr!r!r!r"r!s":*  !   r)rTr Z nltk.stem.apirrr!r!r!r" s