U eR@sdZddlZddlmZddlmZddlmZddlm Z m Z GdddeZ Gd d d eZ Gd d d e ej Z Gd dde ZGddde ZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdd d e ZGd!d"d"eZGd#d$d$eZGd%d&d&eZGd'd(d(eZGd)d*d*e ZGd+d,d,eZGd-d.d.eZd/d0ZdS)1z Snowball stemmers This module provides a port of the Snowball stemmers developed by Martin Porter. There is also a demo function: `snowball.demo()`. N) stopwords)porter)StemmerI)prefix_replacesuffix_replacec@s&eZdZdZdZd ddZddZdS) SnowballStemmera Snowball Stemmer The following languages are supported: Arabic, Danish, Dutch, English, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish and Swedish. The algorithm for English is documented here: Porter, M. "An algorithm for suffix stripping." Program 14.3 (1980): 130-137. The algorithms have been developed by Martin Porter. These stemmers are called Snowball, because Porter created a programming language with this name for creating new stemming algorithms. There is more information available at http://snowball.tartarus.org/ The stemmer is invoked as shown below: >>> from nltk.stem import SnowballStemmer # See which languages are supported >>> print(" ".join(SnowballStemmer.languages)) # doctest: +NORMALIZE_WHITESPACE arabic danish dutch english finnish french german hungarian italian norwegian porter portuguese romanian russian spanish swedish >>> stemmer = SnowballStemmer("german") # Choose a language >>> stemmer.stem("Autobahnen") # Stem a word 'autobahn' Invoking the stemmers that way is useful if you do not know the language to be stemmed at runtime. Alternatively, if you already know the language, then you can invoke the language specific stemmer directly: >>> from nltk.stem.snowball import GermanStemmer >>> stemmer = GermanStemmer() >>> stemmer.stem("Autobahnen") 'autobahn' :param language: The language whose subclass is instantiated. :type language: str or unicode :param ignore_stopwords: If set to True, stopwords are not stemmed and returned unchanged. Set to False by default. :type ignore_stopwords: bool :raise ValueError: If there is no stemmer for the specified language, a ValueError is raised. arabicdanishdutchenglishfinnishfrenchgerman hungarianitalian norwegianr portugueseromanianrussianspanishswedishFcCsN||jkrtd|dt|d}|||_|jj|_|jj|_dS)NzThe language 'z' is not supported.ZStemmer) languages ValueErrorglobals capitalizestemmerstemr)selflanguageignore_stopwordsZ stemmerclassr!S/var/www/html/assets/scripts/venv/lib/python3.8/site-packages/nltk/stem/snowball.py__init__hs    zSnowballStemmer.__init__cCs|j||SN)rrrtokenr!r!r"rpszSnowballStemmer.stemN)F)__name__ __module__ __qualname____doc__rr#rr!r!r!r"r"s1 rc@s"eZdZdZdddZddZdS) _LanguageSpecificStemmera This helper subclass offers the possibility to invoke a specific stemmer directly. This is useful if you already know the language to be stemmed at runtime. Create an instance of the Snowball stemmer. :param ignore_stopwords: If set to True, stopwords are not stemmed and returned unchanged. Set to False by default. :type ignore_stopwords: bool Fc Cst|j}|dr$|dd}t|_|rz t|D]}|j|q)r,r')rr!r!r"__repr__sz!_LanguageSpecificStemmer.__repr__N)F)r'r(r)r*r#r8r!r!r!r"r+ts r+c@seZdZdZdddZdS) PorterStemmeraF A word stemmer based on the original Porter stemming algorithm. Porter, M. "An algorithm for suffix stripping." Program 14.3 (1980): 130-137. A few minor modifications have been made to Porter's basic algorithm. See the source code of the module nltk.stem.porter for more information. FcCst||tj|dSr$)r+r#rr9)rr r!r!r"r#s zPorterStemmer.__init__N)F)r'r(r)r*r#r!r!r!r"r9s r9c@seZdZdZddZdS)_ScandinavianStemmerz This subclass encapsulates a method for defining the string region R1. It is used by the Danish, Norwegian, and Swedish stemmer. cCsd}tdt|D]}|||kr||d|krdt|d|dkrVdkrhnn|dd}n2t|d|ddkr||dd}n|Sqq|S)ar Return the region R1 that is used by the Scandinavian stemmers. R1 is the region after the first non-vowel following a vowel, or is the null region at the end of the word if there is no such non-vowel. But then R1 is adjusted so that the region before it contains at least three letters. :param word: The word whose region R1 is determined. :type word: str or unicode :param vowels: The vowels of the respective language that are used to determine the region R1. :type vowels: unicode :return: the region R1 for the respective word. :rtype: unicode :note: This helper method is invoked by the respective stem method of the subclasses DanishStemmer, NorwegianStemmer, and SwedishStemmer. It is not to be invoked directly! Nrrangelen)rr4vowelsr1ir!r!r"_r1_scandinavians(z%_ScandinavianStemmer._r1_scandinavianN)r'r(r)r*rDr!r!r!r"r:sr:c@s eZdZdZddZddZdS)_StandardStemmerz~ This subclass encapsulates two methods for defining the standard versions of the string regions R1, R2, and RV. cCsd}d}tdt|D]4}|||kr||d|kr||dd}qLqtdt|D]4}|||krZ||d|krZ||dd}qqZ||fS)a Return the standard interpretations of the string regions R1 and R2. R1 is the region after the first non-vowel following a vowel, or is the null region at the end of the word if there is no such non-vowel. R2 is the region after the first non-vowel following a vowel in R1, or is the null region at the end of the word if there is no such non-vowel. :param word: The word whose regions R1 and R2 are determined. :type word: str or unicode :param vowels: The vowels of the respective language that are used to determine the regions R1 and R2. :type vowels: unicode :return: (r1,r2), the regions R1 and R2 for the respective word. :rtype: tuple :note: This helper method is invoked by the respective stem method of the subclasses DutchStemmer, FinnishStemmer, FrenchStemmer, GermanStemmer, ItalianStemmer, PortugueseStemmer, RomanianStemmer, and SpanishStemmer. It is not to be invoked directly! :note: A detailed description of how to define R1 and R2 can be found at http://snowball.tartarus.org/texts/r1r2.html r;r<Nr>)rr4rArBr2rCr!r!r"_r1r2_standardsz_StandardStemmer._r1r2_standardcCsd}t|dkr|d|krRtdt|D]$}|||kr*||dd}qq*nZ|d|kr|d|krtdt|D]$}|||krx||dd}qqxn |dd}|S)a Return the standard interpretation of the string region RV. If the second letter is a consonant, RV is the region after the next following vowel. If the first two letters are vowels, RV is the region after the next following consonant. Otherwise, RV is the region after the third letter. :param word: The word whose region RV is determined. :type word: str or unicode :param vowels: The vowels of the respective language that are used to determine the region RV. :type vowels: unicode :return: the region RV for the respective word. :rtype: unicode :note: This helper method is invoked by the respective stem method of the subclasses ItalianStemmer, PortugueseStemmer, RomanianStemmer, and SpanishStemmer. It is not to be invoked directly! r;r<Nrr=)r@r?rr4rArvrCr!r!r" _rv_standard s     z_StandardStemmer._rv_standardN)r'r(r)r*rGrKr!r!r!r"rEs*rEc@seZdZdZedZedZedZdZ edZ edZ edZ ed Z d Zd Zd Zd ZdZdZdZdZdZdZdZdZdZdZdZdZdZdZdZ dZ!dZ"dZ#d Z$d!Z%d"Z&d#Z'd$Z(d%Z)d&Z*d'Z+d(Z,d)Z-d*Z.d+Z/d,Z0d-Z1d-Z2d.Z3d.Z4d.Z5d.Z6d.Z7d.Z8d.Z9d.Z:d.Z;d.Zd/d0Z?d1d2Z@d3d4ZAd5d6ZBd7d8ZCd9d:ZDd;d<ZEd=d>ZFd?d@ZGdAdBZHdCdDZIdEdFZJdGdHZKdIdJZLdKdLZMdMdNZNdOdPZOdQdRZPdSdTZQdUdVZRdWdXZSdYdZZTd[d\ZUd]d^ZVd_S)` ArabicStemmera https://github.com/snowballstem/snowball/blob/master/algorithms/arabic/stem_Unicode.sbl (Original Algorithm) The Snowball Arabic light Stemmer Algorithm: - Assem Chelli - Abdelkrim Aries - Lakhdar Benzahia NLTK Version Author: - Lakhdar Benzahia z9[\u064b-\u064c-\u064d-\u064e-\u064f-\u0650-\u0651-\u0652]z[\u0640]z[\u060C-\u061B-\u061F])أإآؤuئz^[\u0622\u0623\u0625]z[\u0624]z[\u0626]z[\u0623\u0622\u0625])كالبالاللل)ةات) يكهناكمهاهنهمكماهمان)اrWوrVتrUrW) rYrXنيrZr\r^r]r[كنr`r_كمو) rdrbrarWrZتاتنانونينتما)واuتم)rcتموuى)rMأأأآأؤأاأإ)uفالuوال)uفrc)rSrTrQrR)بrXلببكك)uسيuستuسنuسأ)uيستuنستuتست)rYrX)rerZr\r^r]r[rf)r`r_rg)rbrarW)rZrhri)rjrkrl)rWrXrY)rZr[r\r]r^)r_r`)rnuفا)rQrR)rSrT)rXrv)rwrxTFcCs.|jd|}|jd|}|jd|}|S)zT :param token: string :return: normalized token type string r;)_ArabicStemmer__vocalizationsub_ArabicStemmer__kasheeda(_ArabicStemmer__arabic_punctuation_marksr%r!r!r"Z__normalize_preszArabicStemmer.__normalize_precCsb|jD]}||rt||d}q&q|jd|}|jd|}|jd|}|jd|}|S)NuءrbrcrW)_ArabicStemmer__last_hamzatr.r_ArabicStemmer__initial_hamzatrz_ArabicStemmer__waw_hamza_ArabicStemmer__yeh_hamza_ArabicStemmer__alefat)rr&Zhamzar!r!r"Z__normalize_post!s   zArabicStemmer.__normalize_postcCsr|jD]f}||r||jkr@t|dkr@d|_d|_d|_qn||jkrt|dkrd|_d|_d|_qnqdS)NTFr=)_ArabicStemmer__checks1 startswith_ArabicStemmer__articles_3lenr@is_nounis_verb is_defined_ArabicStemmer__articles_2lenrr&prefixr!r!r"Z __checks_1.s  zArabicStemmer.__checks_1cCsb|jD]V}||r|dkr8t|dkr8d|_d|_q^|dkrt|dkrd|_d|_q^qdS)NrUrHTFrVr=)_ArabicStemmer__checks2r.r@rrrr&suffixr!r!r"Z __checks_2=s  zArabicStemmer.__checks_2cCs|jD]}||r||jkr@t|dkr@|dd}d|_q||jkrlt|dkrl|dd}d|_q||jkrt|dkr|dd}d|_qq|SNrT)!_ArabicStemmer__suffix_verb_step1r.)_ArabicStemmer__conjugation_suffix_verb_1r@suffixes_verb_step1_success)_ArabicStemmer__conjugation_suffix_verb_2)_ArabicStemmer__conjugation_suffix_verb_3rr!r!r"Z__Suffix_Verb_Step1Js     z!ArabicStemmer.__Suffix_Verb_Step1cCs|jD]}||rt|dkr|dkrJt|dkrJ|dd}d|_q||jkrvt|dkrv|dd}d|_q||jkrt|dkr|dd}d|_q||jkrt|dkr|dd}d|_q|dkrt|d kr|dd }d|_qq|S) Nr=rdrrTrrrmrr)"_ArabicStemmer__suffix_verb_step2ar.r@suffix_verb_step2a_success)_ArabicStemmer__conjugation_suffix_verb_4,_ArabicStemmer__conjugation_suffix_verb_past/_ArabicStemmer__conjugation_suffix_verb_presentrr!r!r"Z__Suffix_Verb_Step2a]s.      z"ArabicStemmer.__Suffix_Verb_Step2acCsb|jD]V}||r|dkr8t|dkr8|dd}q^|dkrt|dkr|dd}q^q|S)Nrorrrcrr)"_ArabicStemmer__suffix_verb_step2cr.r@rr!r!r"Z__Suffix_Verb_Step2czs    z"ArabicStemmer.__Suffix_Verb_Step2ccCs<|jD]0}||rt|dkr|dd}d|_q8q|SNrrT)"_ArabicStemmer__suffix_verb_step2br.r@suffix_verb_step2b_successrr!r!r"Z__Suffix_Verb_Step2bs   z"ArabicStemmer.__Suffix_Verb_Step2bcCs<|jD]0}||rt|dkr|dd}d|_q8q|S)Nr=rT)#_ArabicStemmer__suffix_noun_step2c2r.r@suffix_noun_step2c2_successrr!r!r"Z__Suffix_Noun_Step2c2s   z#ArabicStemmer.__Suffix_Noun_Step2c2cCs|jD]}||r||jkr@t|dkr@|dd}d|_q||jkrlt|dkrl|dd}d|_q||jkrt|dkr|dd}d|_qq|Sr)"_ArabicStemmer__suffix_noun_step1ar.)_ArabicStemmer__conjugation_suffix_noun_1r@suffix_noun_step1a_success)_ArabicStemmer__conjugation_suffix_noun_2)_ArabicStemmer__conjugation_suffix_noun_3rr!r!r"Z__Suffix_Noun_Step1as     z"ArabicStemmer.__Suffix_Noun_Step1acCs<|jD]0}||rt|dkr|dd}d|_q8q|S)NrrT)"_ArabicStemmer__suffix_noun_step2ar.r@suffix_noun_step2a_successrr!r!r"Z__Suffix_Noun_Step2as   z"ArabicStemmer.__Suffix_Noun_Step2acCs<|jD]0}||rt|dkr|dd}d|_q8q|Sr)"_ArabicStemmer__suffix_noun_step2br.r@suffix_noun_step2b_successrr!r!r"Z__Suffix_Noun_Step2bs   z"ArabicStemmer.__Suffix_Noun_Step2bcCs6|jD]*}||rt|dkr|dd}q2q|S)Nrr)#_ArabicStemmer__suffix_noun_step2c1r.r@rr!r!r"Z__Suffix_Noun_Step2c1s   z#ArabicStemmer.__Suffix_Noun_Step2c1cCs<|jD]0}||rt|dkr|dd}d|_q8q|S)NrrT)"_ArabicStemmer__suffix_noun_step1br.r@suffixe_noun_step1b_successrr!r!r"Z__Suffix_Noun_Step1bs   z"ArabicStemmer.__Suffix_Noun_Step1bcCs6|jD]*}||rt|dkr|dd}q2q|S)Nr=r)!_ArabicStemmer__suffix_noun_step3r.r@rr!r!r"Z__Suffix_Noun_Step3s   z!ArabicStemmer.__Suffix_Noun_Step3cCs&|jD]}||rt||d}q|S)NrW)'_ArabicStemmer__suffix_all_alef_maqsurar.rrr!r!r"Z__Suffix_All_alef_maqsuras  z'ArabicStemmer.__Suffix_All_alef_maqsuracCs|jD]}||rt|dkr|dkr:t||d}qq|dkrTt||d}qq|dkrnt||d}qq|dkrt||d }qq|d krt||d }qq|S) Nr=rprMrqrOrrrPrsrbrtrN)_ArabicStemmer__prefix_step1rr@rrr!r!r"Z__Prefix_Step1s$      zArabicStemmer.__Prefix_Step1cCs@|jD]4}||rt|dkr|t|d}d|_qq|S)NruاستTF)!_ArabicStemmer__prefix_step4_verbrr@rrrrr!r!r"Z__Prefix_Step4_Verb$s  z!ArabicStemmer.__Prefix_Step4_VerbcCsFd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ d|_ d|_ |}| |||||}||jkst|dkr|S|jr||}|jr||}|js||}n||}|js||}|jr||}|js|js:||}||}|js||}|js|js||}nf||}|j r||}|jsh||}|js|js||}n|js||}||}||}|js|jr||}||}| |}|j s|!|}|"|}|j s|jr|#|}n$|j s4|jr4|$|}|%|}|&|}|}|S)zw Stem an Arabic word and return the stemmed form. :param word: string :return: string TFrH)'rrrrrrrrrrrrr_ArabicStemmer__checks_1_ArabicStemmer__checks_2_ArabicStemmer__normalize_prerr@!_ArabicStemmer__Suffix_Verb_Step1r"_ArabicStemmer__Suffix_Verb_Step2a"_ArabicStemmer__Suffix_Verb_Step2c"_ArabicStemmer__Suffix_Verb_Step2b#_ArabicStemmer__Suffix_Noun_Step2c2"_ArabicStemmer__Suffix_Noun_Step1a"_ArabicStemmer__Suffix_Noun_Step2a"_ArabicStemmer__Suffix_Noun_Step2b#_ArabicStemmer__Suffix_Noun_Step2c1"_ArabicStemmer__Suffix_Noun_Step1b!_ArabicStemmer__Suffix_Noun_Step3'_ArabicStemmer__Suffix_All_alef_maqsura_ArabicStemmer__Prefix_Step1_ArabicStemmer__Prefix_Step2a_ArabicStemmer__Prefix_Step2b"_ArabicStemmer__Prefix_Step3a_Noun"_ArabicStemmer__Prefix_Step3b_Noun!_ArabicStemmer__Prefix_Step3_Verb!_ArabicStemmer__Prefix_Step4_Verb_ArabicStemmer__normalize_post)rr4Z modified_wordZ stemmed_wordr!r!r"r-s                             zArabicStemmer.stemN)Wr'r(r)r*recompileryr{r|r}r~rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr!r!r!r"rL4s               rLc@s4eZdZdZdZdZdZdZdZdZ dZ d d Z d S) DanishStemmera The Danish Snowball stemmer. :cvar __vowels: The Danish vowels. :type __vowels: unicode :cvar __consonants: The Danish consonants. :type __consonants: unicode :cvar __double_consonants: The Danish double consonants. :type __double_consonants: tuple :cvar __s_ending: Letters that may directly appear before a word final 's'. :type __s_ending: unicode :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :note: A detailed description of the Danish stemming algorithm can be found under http://snowball.tartarus.org/algorithms/danish/stemmer.html aeiouyæåøbcdfghjklmnpqrstvwxzbbccddffgghhjjkkllmmnnppZqqrrssttvvZwwZxxzzuabcdfghjklmnoprtvyzå) ZerendesZerendeZhedensZethedZeredehedenZhederendesZernesZerensZeretsZeredendeZerneZerenZererZhedseneseresZeretZhedeneereensersetseneresetr5s)gddtgtkt)eligløstligelsigcCs|}||jkr|S|||j}|jD]f}||r*|dkrh|d|jkr|dd}|dd}n$|dt| }|dt| }qq*|jD]*}||r|dd}|dd}qq|dr|dd}|dd}|j D]}||r|dkr|dd}|dd}nJ|dt| }|dt| }||jrh|dd}|dd}qpq|j D]2}||rvt|dkrv|dd}qqv|S)z Stem a Danish word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode rrNrZigstrr=) r-rrD_DanishStemmer__vowels_DanishStemmer__step1_suffixesr._DanishStemmer__s_endingr@_DanishStemmer__step2_suffixes_DanishStemmer__step3_suffixes!_DanishStemmer__double_consonants)rr4rBr double_consr!r!r"rsJ                     zDanishStemmer.stemN) r'r(r)r*rZ_DanishStemmer__consonantsr rrr r rr!r!r!r"rs"rc@s$eZdZdZdZdZdZddZdS) DutchStemmera The Dutch Snowball stemmer. :cvar __vowels: The Dutch vowels. :type __vowels: unicode :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step3b_suffixes: Suffixes to be deleted in step 3b of the algorithm. :type __step3b_suffixes: tuple :note: A detailed description of the Dutch stemming algorithm can be found under http://snowball.tartarus.org/algorithms/dutch/stemmer.html uaeiouyè)rrrser)baarlijkbarendingrcCs|}||jkr|Sd}|dddddddddd d d d d d d dddd}|drdd|ddf}tdt|D]F}||d|jkr||dkrd|d|d||ddf}qtdt|dD]X}||d|jkr||d kr||d|jkrd|d|d||ddf}q|||j\}}tdt|D]}|||jkrp||d|jkrpdt|d|dkrdkrnn|dd}n"t|d|ddkr|Sqqp|j D]}| |r|dkrTt ||d}t ||d}| drt ||d}n,|dkr| ds|t| d|jkr|t| dt| dkr|dt| }|dt| }|dt| }| dr|dd}|dd}|dd}nr|dkr|t| d|jkr|t| dd kr|dt| }|dt| }|dt| }qq| dr|d!|jkrd"}|dd}|dd}|dd}| dr|dd}|dd}|dd}| dr|d#d$kr|dd%}|dd%}|dd%}| d&r|d'|jkr|d#d!dkr|dd!}|dd!}|dd!}| dr|dd}|dd}|dd}|j D]:}| |r|d(kr:|dd'}|dd'}| d)r |d'dkr |dd!}n| dr|dd}n|d)kr`|d'dkr`|dd!}n|d*kr|dd%}|dd%}| dr|d!|jkr|dd}| dr|dd}n4|d+kr|dd%}n|d,kr|r|dd'}qqt|d-krp|d|jkrp|ddkrp|d'dd.krp|d%|jkrpd|dd'|d'|df}|dd dd}|S)/z Stem a Dutch word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode Fäaáër5éírCïöoóüuúyr;Yr<NIr=rrZheid)rrgem)rrrr)rrjrTcrr)rrrrrrr)aaeeoouu) r-rreplacerjoinr?r@_DutchStemmer__vowelsrG_DutchStemmer__step1_suffixesr.r_DutchStemmer__step3b_suffixes)rr4 step2_successrCrBrFrr!r!r"rGs     & &$,                                            zDutchStemmer.stemN)r'r(r)r*r/r0r1rr!r!r!r"r 2s r c)@seZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d d dddddddddd ddddddddddddd d d!d!d"d"d"d"d#d#d#d#d$d$d$d$d%(Zd&d'Zd(S))EnglishStemmera The English Snowball stemmer. :cvar __vowels: The English vowels. :type __vowels: unicode :cvar __double_consonants: The English double consonants. :type __double_consonants: tuple :cvar __li_ending: Letters that may directly appear before a word final 'li'. :type __li_ending: unicode :cvar __step0_suffixes: Suffixes to be deleted in step 0 of the algorithm. :type __step0_suffixes: tuple :cvar __step1a_suffixes: Suffixes to be deleted in step 1a of the algorithm. :type __step1a_suffixes: tuple :cvar __step1b_suffixes: Suffixes to be deleted in step 1b of the algorithm. :type __step1b_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :cvar __step4_suffixes: Suffixes to be deleted in step 4 of the algorithm. :type __step4_suffixes: tuple :cvar __step5_suffixes: Suffixes to be deleted in step 5 of the algorithm. :type __step5_suffixes: tuple :cvar __special_words: A dictionary containing words which have to be stemmed specially. :type __special_words: dict :note: A detailed description of the English stemming algorithm can be found under http://snowball.tartarus.org/algorithms/english/stemmer.html Zaeiouy) rrrrrrrrrZ cdeghkmnrt)z's'z's')ssesiediesusrr)eedlyZinglyZedlyeedred)izationationalfulnessousnessivenesstionalbilitilesslientliationalismalitiousliivitifulliencianciabliizeratorallibliogili) r=rAalizeicateicitiativeicalnessful)ementanceenceableiblementantentismateitiousiveizeionalric)r5lskiskyZdieZlieZtieidlZgentlZugliZearliZonliZsinglnewshoweatlascosmosbiasandesinningoutingcanningherringearringproceedexceedsucceed)(skisZskiesZdyingZlyingZtyingZidlyZgentlyZuglyZearlyonlyZsinglyrnrprqrrrsrtrurvZinningsrwZoutingsrxZcanningsryZherringsrzZearringsr{ZproceedsZ proceededZ proceedingr|ZexceedsZexceededZ exceedingr}ZsucceedsZ succeededZ succeedingc CsB |}||jkst|dkr"|S||jkr6|j|S|dddddd}|drh|dd}|drd d |ddf}tdt|D]F}||d|jkr||dkrd |d|d ||ddf}qd }d }d }d }|d rr|d r|dd}n |dd}tdt|D]@}|||jkr.||d|jkr.||dd}qq.n| ||j\}}|j D]N}| |r|dt| }|dt| }|dt| }qڐq|j D]}| |r|dkr"|dd}|dd}|dd}n|dkrt|dt| dkrn|dd}|dd}|dd}n$|dd}|dd}|dd}n^|dkr|ddD]}||jkrd}qȐq|r|dd}|dd}|dd}qq|j D]P}| |r|dkr| |rLt||d}t|t|krVt||d}nd }t|t|krzt||d}nd }n|dt| D]}||jkrd}qq|rL|dt| }|dt| }|dt| }| drDd |df}d |df}t|dks2t|dkrLd |df}n| |jrx|dd}|dd}|dd}n|d krt|dkr|d|jkr|ddkr|d|jkr|d|jks|d krLt|dkrL|d|jkrL|d|jkrLd |df}t|dkr0d |df}t|dkrLd |df}qVqt|dkr|ddkr|d|jkrd |dddf}t|dkrd |dddf}nd }t|dkrd |dddf}nd }|jD](}| |r| | r|d krB|dd}|dd}|dd}n|d!krd |dddf}t|dkrd |dddf}nd }t|dkrd |dddf}nd }nZ|d"kr|dd}|dd}|dd}n(|d#krNt||d$}t|t|kr"t||d$}nd }t|t|krFt||d$}nd }n|d%krt||d&}t|t|krt||d&}nd }t|t|krt||d&}nd}nd|d'krt||d(}t|t|krt||d(}nd }t|t|kr t||d(}nd }n|d)krD|dd*}|dd*}|dd*}n|d+krt||d,}t|t|krzt||d,}nd }t|t|krt||d,}nd }nn|d-k rt||d.}t|t|krt||d.}nd }t|t|k rt||d.}nd}n |d/k rht||d0}t|t|k r>t||d0}nd }t|t|k rbt||d0}nd }n|d1k r|d*d2k r|dd}|dd}|dd}nn|d3k r|dd}|dd}|dd}n>|d4k r|d|jk r|dd}|dd}|dd} qq|jD]}| | r$| | r|d k rt|dd}|dd}|dd}nn|d5k rt||d&}t|t|k rt||d&}nd }t|t|k rt||d&}nd }n |d6k r|dd}|dd}|dd}n|d7k rft||d8}t|t|k rk r2|d|jk s&|ddk s&|d|jk s&|d*|jk r2|dd}|d d}|S)?z Stem an English word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode rHu’r4u‘u‛r<Nr!r;r"F)generZcommunarsen)rrrrr5r)r6r7rrT)r:r9r*)atblZizr5r=ZwxYrrZyYrCrA)rKrLrMrD)rNr<rh)r=rErOrd)rFrGrPrjr>r()rHr?rf)r@rIrg)rBrQZblerRrl)rJrCrSr=rT)rUrVrXrk)rZrYrWr&ristr)r-rr@_EnglishStemmer__special_wordsr-rr.r?_EnglishStemmer__vowelsrG_EnglishStemmer__step0_suffixesr. _EnglishStemmer__step1a_suffixes _EnglishStemmer__step1b_suffixesr"_EnglishStemmer__double_consonants_EnglishStemmer__step2_suffixes_EnglishStemmer__li_ending_EnglishStemmer__step3_suffixes_EnglishStemmer__step4_suffixes) rr4rCZstep1a_vowel_foundZstep1b_vowel_foundrBrFrletterr!r!r"r~s$       &   $                                        ,                                                                   zEnglishStemmer.stemN)r'r(r)r*rrrrrrrrrZ_EnglishStemmer__step5_suffixesrrr!r!r!r"r3sj +r3c@s<eZdZdZdZdZdZdZdZdZ dZ d Z d Z d d Z d S)FinnishStemmeraS The Finnish Snowball stemmer. :cvar __vowels: The Finnish vowels. :type __vowels: unicode :cvar __restricted_vowels: A subset of the Finnish vowels. :type __restricted_vowels: unicode :cvar __long_vowels: The Finnish vowels in their long forms. :type __long_vowels: tuple :cvar __consonants: The Finnish consonants. :type __consonants: unicode :cvar __double_consonants: The Finnish double consonants. :type __double_consonants: tuple :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :cvar __step4_suffixes: Suffixes to be deleted in step 4 of the algorithm. :type __step4_suffixes: tuple :note: A detailed description of the Finnish stemming algorithm can be found under http://snowball.tartarus.org/algorithms/finnish/stemmer.html u aeiouyäöu aeiouäö)r)r*iir+r,uääuöörr) ZkaanukäänstiZkinhanhänkouköpaupä) ZnsaunsäZmmeZnnesinianänr)siinttenseenrhenhinhonrhöndenttattässassästaställallältaltälleksiinetatänanärrn)ZimpiZimpauimpäZimmiimmauimmämpimpampämmimmammäZejauejäcCs|}||jkr|Sd}|||j\}}|jD]}||r2|dkrv||kr|dd}|dd}|dd}nL|t| ddkr|dt| }|dt| }|dt| }qq2|jD]}||r|dkr|ddkr|dd }|dd }|dd }n|d kr|dd }|dd }|dd }|d rdt|d d }|d r|t|d d }|d rt|d d }n|d kr|dd dks|dd dkr|dd }|dd }|dd }n|dkr@|dd dks|dd dkr|dd }|dd }|dd }nf|dkr|dd dkr|dd }|dd }|dd }n$|dd}|dd}|dd}qq|j D]}||r|dkr|dkr|ddks`|dkr|ddks`|dkr|ddks`|dkr0|ddks`|d krH|dd!ks`|d"kr|dd#kr|dd}|dd}|dd}d$}n|d%kr |t| ddkr|t| d&|j kr|dt| }|dt| }|dt| }d$}nqn|d'kr\|d(d|j kr|dd}|dd}|dd}d$}nqnN|d)kr|d |jkr|d|j kr|dd*}|dd*}|dd*}d$}n|d+kr|ddkr|dd}|dd}|dd}d$}n|d,krp|dd*}|dd*}|dd*}d$}|d dd-ksJ|d d|j kr|dd*}|dd*}|dd*}n:|dt| }|dt| }|dt| }d$}qq|j D]}||r|d.kr |ddd/krB|dd}|dd}|dd}n6|dt| }|dt| }|dt| }qLq|rt|dkr|d*d0kr|dd*}|dd*}n|s:t|d&kr:|d*d1kr:|d |jkr:|dd*}|dd*}|dd*}|d2r|dd}|dd}n6|d3r:|ddd/kr:|dd}|dd}|d d|j krf|dd*}|dd*}t|d&kr|d |j kr|d*d4kr|dd*}|dd*}|d5r|dd*}|dd*}|d6r|dd*}|dd*}tdt|D]}|| |jkrqn~|dkrN|| dd|jkr|dd*}nF|| d| d|jkrd7|d| || ddf}qq|S)8z Stem a Finnish word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode FrNrr<u ntaeiouyäörkrrZkserrr()rrr&)rrrrr)rr)rrrrr)rr)rrrrrrrrrr5rrCrrrrrrT)rrrrHr)rrr)rrrie)rrrrrrpoZijtrruaäei)ZojZujZjor;)r-rrG_FinnishStemmer__vowels_FinnishStemmer__step1_suffixesr.r@_FinnishStemmer__step2_suffixesr_FinnishStemmer__step3_suffixes"_FinnishStemmer__restricted_vowels_FinnishStemmer__long_vowels_FinnishStemmer__consonants_FinnishStemmer__step4_suffixesr?"_FinnishStemmer__double_consonantsr.)rr4Z step3_successrBrFrrCr!r!r"rosh                     $   $                                      &         "             ,          & zFinnishStemmer.stemN)r'r(r)r*rrrrrrrrrrr!r!r!r"rs  rc@s4eZdZdZdZdZdZdZdZddZ d d Z d S) FrenchStemmera The French Snowball stemmer. :cvar __vowels: The French vowels. :type __vowels: unicode :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2a_suffixes: Suffixes to be deleted in step 2a of the algorithm. :type __step2a_suffixes: tuple :cvar __step2b_suffixes: Suffixes to be deleted in step 2b of the algorithm. :type __step2b_suffixes: tuple :cvar __step4_suffixes: Suffixes to be deleted in step 4 of the algorithm. :type __step4_suffixes: tuple :note: A detailed description of the French stemming algorithm can be found under http://snowball.tartarus.org/algorithms/french/stemmer.html uaeiouyâàëéêèïîôûù)+ issementsissementatricesatriceateursationslogiesusionsutionsementsammentemmentancesiqUesismesablesistesateurrElogieusionutionencesr[eusesmentsr\iqUeismer^ister]itésiveseauxeuser`euxitérgifsauxif)#ZissaIentZissantesZiraIentZissanteZissantsZissionsZirionsZissaisZissaitZissantZissentZissiezZissonsZiraisZiraitZirentZiriezZironsZirontissesZissezuîmesuîtesiraiirasZirezisser7irauîtririsitrC)&eraIentassionserionsassentassiezèrenteraiseraiteriezeronserontaIentantesassesionseraieraserezâmesâtesanteantsasseéeseraiezaisaitraéeésrezâtaiasrr)ièreIèreriierIerr5rc Cs< |}||jkr|Sd}d}d}d}tdt|D]D}||ddkr4||dkr4d|d|d||ddf}q4tdt|dD]}||d|jkr||d|jkr||dkrd|d|d||ddf}n2||dkrd|d|d ||ddf}||d|jksB||d|jkr||d krd|d|d ||ddf}q|||j\}}|||j} |jD]v} | | r| d kr|dd }d}n@| dkr| |kr|dt|  }d}n| |kr t || d}d}n| dkrP| | krP|dt|  }d}|dddkrd|kr|dd}|dddkrLd|krL|dd}n|dddkrd|kr|dd}n d|krLd|dd df}np|dddkrd|ksd|krL|dd}n<|dddkr d| ks6d| kr d|dddf}n| dkr| | krt |dd}t | dd} d}n| d kr| | krt |d d!}d}n^| d"kr| | kr| | s| | | d|jkr|dt|  }| dt|  } d}n| d#krD| |krDd|ddd$f}d}n| d%kr| |kr|t|  d|jkr|dt|  }d}n~| d&kr| |kr|dt|  }d}nP| d'kr(| |kr(|dt|  }d}|ddd(kr d(|kr|dd}nd|dddf}n| d)krP| |krPt || d*}d}n| d+krx| |krxt || d}d}n| d,kr| |krt || d!}d}nj| d-krx| |krx|dt|  }d}|d.dd/kr d/|kr|dd.}nd|ddd$f}nj|ddd(krNd(|kr6|dd}nd|dddf}n(|dddkr d|kr |dd}n| d0kr | |kr |dt|  }d}|dddkr d|kr |dd}|ddd(kr d(|kr|dd}nd|dddf}qq|r |rD|j D]d} | | r&| | krt| t| kr| | | d|jkr|dt|  }d}qq&|sD|jD]} | | r| d1krd1|kr|dd.}d}nl| d2kr|dt|  }d}nJ| d3kr:|dt|  }| dt|  } d}| d4r:|dd }qDq|sV|sV|r|d d kr|d|dd df}n$|d d5k rd|dd d6f}nt|d7kr|d d8kr|dd9kr|dd }|jD]} | | r| | kr| d:k r(| |k r(| d.d;k r(|dd}nX| dk r|dd } qq| d? r|dd }tdt|D]j}|| |jk r|d7}nH|dk r|| d@k rd|d| d4|| ddf} q q|d dddd d }|S)Az Stem a French word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode Fr<qrr;NUrCr#r!r"rrT)rrr)r[rrivrrZeusx)abliqUr!r")ièrIèrr#r$rrarrb)r`rrrl)rr) r\rrr^rrrrrrr)rrrErrrrk)rrlog)rrrr)r]r)rrr(abil)rrgrrr)rrrrrrrrrrrrrrrrrrr)rrrrrrr r r r r rrrarrrrr5çr'rHruaiouèsrir)rrrrrgu)ZennZonnZettZellZeill)rè)r-rr?r@r._FrenchStemmer__vowelsrG_FrenchStemmer__rv_french_FrenchStemmer__step1_suffixesr.rrrindex_FrenchStemmer__step2a_suffixes_FrenchStemmer__step2b_suffixes_FrenchStemmer__step4_suffixesr-) rr4 step1_successZrv_ending_foundZstep2a_successZstep2b_successrCrBrFrJrr!r!r"rs|  &( &$& &                                    *    "       ( zFrenchStemmer.stemcCsxd}t|dkrt|ds2|d|kr@|d|kr@|dd}n4tdt|D]$}|||krN||dd}qtqN|S)a Return the region RV that is used by the French stemmer. If the word begins with two vowels, RV is the region after the third letter. Otherwise, it is the region after the first vowel not at the beginning of the word, or the end of the word if these positions cannot be found. (Exceptionally, u'par', u'col' or u'tap' at the beginning of a word is also taken to define RV as the region to their right.) :param word: The French word whose region RV is determined. :type word: str or unicode :param vowels: The French vowels that are used to determine the region RV. :type vowels: unicode :return: the region RV for the respective French word. :rtype: unicode :note: This helper method is invoked by the stem method of the subclass FrenchStemmer. It is not to be invoked directly! r;rH)parcoltaprr<r=N)r@rr?rIr!r!r"Z __rv_french s     zFrenchStemmer.__rv_frenchN) r'r(r)r*r*r,r.r/r0rr+r!r!r!r"rRs-%(:rc@s0eZdZdZdZdZdZdZdZdZ dd Z d S) GermanStemmeraD The German Snowball stemmer. :cvar __vowels: The German vowels. :type __vowels: unicode :cvar __s_ending: Letters that may directly appear before a word final 's'. :type __s_ending: unicode :cvar __st_ending: Letter that may directly appear before a word final 'st'. :type __st_ending: unicode :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :note: A detailed description of the German stemming algorithm can be found under http://snowball.tartarus.org/algorithms/german/stemmer.html u aeiouyäöüZ bdfghklmnrtZ bdfghklmnt)ernemrrrr5r)Zestrrr)ischlichheitkeitrungrikcCs8|}||jkr|S|dd}tdt|dD]}||d|jkr4||d|jkr4||dkrd|d|d||ddf}q4||dkr4d|d|d ||ddf}q4|||j\}}tdt|D]}|||jkr||d|jkrd t|d|dkr,d kr>nn|d d}n"t|d|dd kr`|Sqhq|jD]}| |rn|d kr|t| d t| dkr|dt| d}|dt| d}|dt| d}nv|dkr.|d|j krd|dd}|dd}|dd}n6|dt| }|dt| }|dt| }qnqn|j D]}| |rt|dkr|d|j krt|ddd kr|dd}|dd}|dd}n6|dt| }|dt| }|dt| }qqt|j D]}| |r |dkrd|t| dt| krd|t| d t| dkr|dt| d}n|dt| }nL|dkrd|t| dt| kr|dt| }n |dkrnd|t| dt| ksBd|t| dt| krZ|dt| d}n|dt| }n|dkrd|t| d t| kr|dt| d }nLd|t| dt| kr|dt| d}n|dt| }qq |ddd d!d"dddd d}|S)#z Stem a German word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode ßrr<rr;Nrr!r"r=r)rrr5rZnissrrrrr)rr<rrHr5)rr=r8)r9r:rrr;r9rrrrr)r-rr-r?r@_GermanStemmer__vowelsr.rG_GermanStemmer__step1_suffixesr._GermanStemmer__s_ending_GermanStemmer__step2_suffixes_GermanStemmer__st_ending_GermanStemmer__step3_suffixes)rr4rCrBrFrr!r!r"r_ s   $ & & ,         &      "  ""  zGermanStemmer.stemN) r'r(r)r*r?rArCr@rBrDrr!r!r!r"r5@ sr5c@sPeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZddZddZdS)HungarianStemmera The Hungarian Snowball stemmer. :cvar __vowels: The Hungarian vowels. :type __vowels: unicode :cvar __digraphs: The Hungarian digraphs. :type __digraphs: tuple :cvar __double_consonants: The Hungarian double consonants. :type __double_consonants: tuple :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :cvar __step4_suffixes: Suffixes to be deleted in step 4 of the algorithm. :type __step4_suffixes: tuple :cvar __step5_suffixes: Suffixes to be deleted in step 5 of the algorithm. :type __step5_suffixes: tuple :cvar __step6_suffixes: Suffixes to be deleted in step 6 of the algorithm. :type __step6_suffixes: tuple :cvar __step7_suffixes: Suffixes to be deleted in step 7 of the algorithm. :type __step7_suffixes: tuple :cvar __step8_suffixes: Suffixes to be deleted in step 8 of the algorithm. :type __step8_suffixes: tuple :cvar __step9_suffixes: Suffixes to be deleted in step 9 of the algorithm. :type __step9_suffixes: tuple :note: A detailed description of the Hungarian stemming algorithm can be found under http://snowball.tartarus.org/algorithms/hungarian/stemmer.html uaeiouöüáéíóõúû)csZdzZdzsgyZlynytyZzs)rrZccsrrrZggyrrrZllyrrZnnyrrrZsszrttyrrZzzs)rjel),uképpenuonkéntuenkéntuankéntuképpukéntZbanZbenZnakZnekvalZvelutólutõlurólurõlubólubõlZhozZhezuhözunálunéluértZkorbaberarrrrZotuötuluüluváuvéronruönrr)uánkéntuánén)ZastuluestülástuléstülZstulustül)rr) uokéuökéuakéuekéákéáéiékéééiukééiéér)ájukéjükuünkZunkZjukujükánkénkZnkukuükr7omamZodr;aduödjaZjeámádémédmdrr5rrr)*ZjaitokZjeitekZjainkZjeinkZaitokZeitekáitokéitekZjaimZjeimZjaidZjeidZeinkZainkZitekZjeikZjaikáinkéinkZaimZeimZaidZeidZjaiZjeiZinkZaikZeikáimáidáikéiméidéikimidreir=áirYrC)ákékuökokZekZakrcCs|}||jkr|S|||j|j}||jr|jD]l}|dt|d|kr:d |dd|df}|dt|d|krd |dd|df}qq:|j D]}||r||rJ|dt| }|dt| }|drd |dddf}t |dd}n.|d rJd |ddd f}t |d d }qRq|j D]T}||rX|d krt ||d }t ||d }nt ||d}t ||d}qqX|j D]}||r|d krt ||d}t ||d}nH|d kr t ||d }t ||d }n$|dt| }|dt| }q:q|jD]}||r@|jD]t}|dt|d|krVd |dd|df}|dt|d|krd |dd|df}q@qVq@|jD]}||r|dkr t ||d}t ||d}nH|dkr.t ||d }t ||d }n$|dt| }|dt| }q\q|jD]}||rb||r|dkrt ||d}t ||d}nH|dkrt ||d }t ||d }n$|dt| }|dt| }qqb|jD]}||r||r|dkr:t ||d}t ||d}nH|dkr^t ||d }t ||d }n$|dt| }|dt| }qq|jD]f}||r||r|dkrt ||d}n*|dkrt ||d }n|dt| }qq|S)z Stem an Hungarian word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode rr;Nr(rrrrrr5rRrSrT)rUrV)rWrXrZ)r]r[rdrer)r^r\rfrgr)rnrorwrlrjrp)rqrrrYrmrkrsrxry)r-r_HungarianStemmer__r1_hungarian_HungarianStemmer__vowels_HungarianStemmer__digraphsr.!_HungarianStemmer__step1_suffixes$_HungarianStemmer__double_consonantsr@r.!_HungarianStemmer__step2_suffixesr!_HungarianStemmer__step3_suffixes!_HungarianStemmer__step4_suffixes!_HungarianStemmer__step5_suffixes!_HungarianStemmer__step6_suffixes!_HungarianStemmer__step7_suffixes!_HungarianStemmer__step8_suffixes!_HungarianStemmer__step9_suffixes)rr4rBr rr!r!r"r s                                                        zHungarianStemmer.stemcCsd}|d|kr|D]6}||ddkr|||ddd}|Sqtdt|D]$}|||krZ||dd}qqZn4tdt|D]$}|||kr||dd}qq|S)a? Return the region R1 that is used by the Hungarian stemmer. If the word begins with a vowel, R1 is defined as the region after the first consonant or digraph (= two letters stand for one phoneme) in the word. If the word begins with a consonant, it is defined as the region after the first vowel in the word. If the word does not contain both a vowel and consonant, R1 is the null region at the end of the word. :param word: The Hungarian word whose region R1 is determined. :type word: str or unicode :param vowels: The Hungarian vowels that are used to determine the region R1. :type vowels: unicode :param digraphs: The digraphs that are used to determine the region R1. :type digraphs: tuple :return: the region R1 for the respective word. :rtype: unicode :note: This helper method is invoked by the stem method of the subclass HungarianStemmer. It is not to be invoked directly! r;rr<Nr)indexr?r@)rr4rAZdigraphsrBZdigraphrCr!r!r"Z__r1_hungarianL s    zHungarianStemmer.__r1_hungarianN)r'r(r)r*r|r}rr~rrrrrrrrrr{r!r!r!r"rE s !.!,rEc@s(eZdZdZdZdZdZdZddZdS) ItalianStemmeraj The Italian Snowball stemmer. :cvar __vowels: The Italian vowels. :type __vowels: unicode :cvar __step0_suffixes: Suffixes to be deleted in step 0 of the algorithm. :type __step0_suffixes: tuple :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :note: A detailed description of the Italian stemming algorithm can be found under http://snowball.tartarus.org/algorithms/italian/stemmer.html uaeiouàèìòù)%ZglielaZglieleZglieliZglieloZglieneZseneZmelaZmeleZmeliZmeloZmeneZtelaZteleZteliZteloZteneZcelaZceleZceliZceloZceneZvelaZveleZveliZveloZveneZglicilalerSlominertivi)3rZatriciazioneazioniuzioneuzioniusioneusioniamentoamentiimentoimentiamenteabileabiliibileibilimenteatoreatorilogiaranzaZanzeZicheZichiismoZismiistaristiuistàuistèuistìr antienzaenzeicoiciicaiceosoosiosaZoseitàivoiviivarg)WZerebberoZirebberoZasseroZassimoZerannoZerebbeZeremmoZeresteZerestiZesseroZirannoZirebbeZiremmoZiresteZirestiZiscanoZisconoZisseroZaronoZavamoZavanoZavateZeremoZereteZeronoZevamoZevanoZevateZiremoZireteZironoZivamoZivanoZivateZammoandor ZassiZemmoZendarZendiendorereiZYamoZiamoZimmorireiZiscaZisceZisciZiscoZanoareatardatiZatoavaZaviZavoueràrueròeteZevaZeviZevouiràireuiròitaitereZitorrrZonoutauteutiZutoarrcCs@|}||jkr|Sd}|dddddddd d d }td t|D]D}||d d krT||dkrTd|d|d||d df}qTtd t|d D]}||d |jkr||d |jkr||dkrd|d|d||d df}q||dkrd|d|d||d df}q|||j\}}|||j}|j D]}| |r`|t| dt| dkr|dt| }|dt| }|dt| }|dt| }nR|t| dt| dkr.t ||d}t ||d}t ||d}t ||d}q8q`|j D]<}| |r>|dkr2| |r2d}|dd}|dd}|dd}| dr|dd}|dd}|dd}| dr.|dd}|dd}nJ| dr |dd}|dd}n$| d rt|dd!}|dd!}nB|d"krh| |rhd}|dd}|dd}n | |rtd}|d#kr|dt| }|dt| }|dt| }| d$rt|dd}|dd}n|d%kr|dd}|dd}nn|d&kr,|dd'}|dd'}nH|d(krRt ||d)}t ||d)}n"|d*kr|dd+}|dd+}|dd+}| d,r|dd}|dd}n$| d rt|dd!}|dd!}n|d-krP|dd+}|dd+}|dd+}| drt|dd}|dd}|dd}| d$rt|dd}|dd}n$|dt| }|dt| }q~q>|s|j D]<}| |r|dt| }|dt| }qȐq| d.r|dd/}|dd/}| dr|dd/}|dd/}| d0r(|dd/}|dddd}|S)1z Stem an Italian word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode Fràrr)rìròr ùr<rrr;NrrCr#r)rrrH)rrrr5rTrrrr)osrkr&r()rrrr)rrrrrk)rr)rrrrr&)rrterrrkr)rrrrg)rr5rCrrr)rrr)chgh)r-rr-r?r@r._ItalianStemmer__vowelsrGrK_ItalianStemmer__step0_suffixesr.r_ItalianStemmer__step1_suffixes_ItalianStemmer__step2_suffixes)rr4r1rCrBrFrJrr!r!r"rC s    &$& &  ""                                                             zItalianStemmer.stemN) r'r(r)r*rrrrrr!r!r!r"ry s '5Zrc@s,eZdZdZdZdZdZdZdZddZ d S) NorwegianStemmera The Norwegian Snowball stemmer. :cvar __vowels: The Norwegian vowels. :type __vowels: unicode :cvar __s_ending: Letters that may directly appear before a word final 's'. :type __s_ending: unicode :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :note: A detailed description of the Norwegian stemming algorithm can be found under http://snowball.tartarus.org/algorithms/norwegian/stemmer.html rZbcdfghjlmnoprtvyz)ZhetenesZhetenehetensheterhetenranderZedesrerteZedeZanerrrrhetastertrrrrrrrr5r)rvt) ZhetslovZelegrZelovZslovlegZeigrrZlovrcCs8|}||jkr|S|||j}|jD]}||r*|dkrZt||d}t||d}nn|dkr|d|jks|ddkr|d|jkr|dd}|dd}n$|dt| }|dt| }qq*|j D],}||r|dd}|dd}qq|j D]*}||r|dt| }q4q|S) z Stem a Norwegian word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode )rrrrrrrNr) r-rrD_NorwegianStemmer__vowels!_NorwegianStemmer__step1_suffixesr.r_NorwegianStemmer__s_endingr@!_NorwegianStemmer__step2_suffixes!_NorwegianStemmer__step3_suffixesrr4rBrr!r!r"r8s<               zNorwegianStemmer.stemN) r'r(r)r*rrrrrrr!r!r!r"r s rc@s(eZdZdZdZdZdZdZddZdS) PortugueseStemmerav The Portuguese Snowball stemmer. :cvar __vowels: The Portuguese vowels. :type __vowels: unicode :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step4_suffixes: Suffixes to be deleted in step 4 of the algorithm. :type __step4_suffixes: tuple :note: A detailed description of the Portuguese stemming algorithm can be found under http://snowball.tartarus.org/algorithms/portuguese/stemmer.html uaeiouáéíóúâêô)/ZamentosZimentosuço~esrradorasadoresuaço~eslogiasênciasridadesuançasismosistasadorauaça~oruânciaruça~oênciaridadeuançaZezasicosicasruáveluívelrosososasadorr ivasivosrZezarrrrrrr)xaríamoseríamosiríamosuássemosuêssemosuíssemosuaríeisueríeisuiríeisuásseisuésseisuísseisáramosuéramosuíramosuávamosaremoseremosiremosZariamZeriamZiriamZassemZessemZissemzara~ozera~ozira~oZariasZeriasZiriasZardesZerdesZirdesrZessesrZastesZestesruáreisZareisuéreisZereisuíreisZireisuáveisuíamosZarmosZermosZirmosZariaZeriaZiriar ZesserasteZesterZareirrZaramZeramZiramZavamZaremZeremZiremrrZindoadasidasarásaraserásrirásZavasZaresrZiresuíeisadosidosuámosamosemosimosradaidaaráaraeráriráriamadoidoZiasrZeisriarvrar7rrrrrreuiuZou)rrrCrrrrcCs|}||jkr|Sd}d}|dddddddd }|||j\}}|||j}|jD]}||rh|d kr4||r4d }|d d }|d d }|d d }|dr |d d}|d d}|d d}|dr0|d d}|d d}n$|drz|d d}|d d}nF|dkr||r|t| dt| dkrd }t ||d}t ||d}n||rzd }|dkrt ||d}t ||d}n|dkrt ||d}t ||d}n|dkrt ||d}t ||d}nl|dkrd|d d}|d d}|d d}|drz|d d}|d d}n|d kr|d t| }|d t| }|d t| }|d!r|d d}|d d}n$|d"rz|d d}|d d}n|d#krV|d t| }|d t| }|d t| }|drz|d d}|d d}n$|d t| }|d t| }qqh|s|j D]@}||rd }|d t| }|d t| }qАq|s|r|d$r|dd%kr|d d&}|d d&}|s^|s^|j D]<}||r |d t| }|d t| }q^q |d'r|d d&}|d d&}|d r|ds|d(r|d$r|d d&}n|d)rt |d)d%}|dddd}|S)*z Stem a Portuguese word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode Fãza~õzo~uqüZquugür(rTNrrrrrrkrb)rrr<r5r)rrr%)rrr)rrenterr&)r ZavelZivelr()rrrr&)rrrrrCr'r)r5rêrr') r-rr-rG_PortugueseStemmer__vowelsrK"_PortugueseStemmer__step1_suffixesr.r@r"_PortugueseStemmer__step2_suffixes"_PortugueseStemmer__step4_suffixes)rr4r1r2rBrFrJrr!r!r"r,s                                                     zPortugueseStemmer.stemN) r'r(r)r*rrrr rr!r!r!r"rks 1zrc@s,eZdZdZdZdZdZdZdZddZ d S) RomanianStemmera The Romanian Snowball stemmer. :cvar __vowels: The Romanian vowels. :type __vowels: unicode :cvar __step0_suffixes: Suffixes to be deleted in step 0 of the algorithm. :type __step0_suffixes: tuple :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :note: A detailed description of the Romanian stemming algorithm can be found under http://snowball.tartarus.org/algorithms/romanian/stemmer.html u aeiouăâî)iiloruluieloriileilorateiaţieaţiaauaeleiuaieiilerPear). abilitate abilitati abilităţi ibilitate abilităiivitateivitati ivităţiicitateicitati icităţiicatoriivităiicităiicatoraţiuneatoareătoareiţiuneitoareicivaiciveiciviicivăicalaicaleicaliicalăativarWativiativărătoriitivaitiveitiviitivăitoriicivrXativrOătoritivitor)>ZabilarruabilăZibilarruibilărZitateZitatiuităţir&ZibilZoasauoasăZoaseZantar ruantărOuităiiuneiunirrrristăiştiruatărrdruutărrruitărerrrruicăruoşirarrgruivărcistrutrrkrr)^ seserăţiu aserăţiu iserăţiu âserăţiu userăţiseserămuaserămuiserămuâserămuuserămserăţiseseşiseserăueascăuarăţiuurăţiuirăţiuârăţiuaseşiuaserăuiseşiuiserăuâseşiuâserăuuseşiuuserăserămsesemZinduuânduueazăueştiueşteuăştiuăşteueaţiuiaţiuarămuurămuirămuârămZasemZisemuâsemZusemseşiserăseserrruâreinduândZezeZeziescuăscZeamZeaiZeaurZiaiZiauuaşiuarăuuşiuurăuişiuirăuâşiuârăaseZiseuâseZuseaţieţiiţiâţiseirraraur/rZuiuâiămr7rtâmrc Csz|}||jkr|Sd}d}tdt|dD]}||d|jkr0||d|jkr0||dkrd|d|d||ddf}q0||dkr0d|d|d||ddf}q0|||j\}}|||j}|jD]}| |r||kr|d kr<|dt| }||kr6|dt| }nd}n|d ksl|d ksl|d krz|d ddkrz|dd}n~|dkrt ||d}||krt ||d}nd}nJ|dkrt ||d}||krt ||d}nd}n|dkr|dd}qqd} |j D]}| |r ||krd}d} |dkrDt ||d}n|dkr\|dd }n|dkrtt ||d}nr|dkrt ||d}nZ|dkrt ||d}||krt ||d}n,|d krt ||d!}||krt ||d!}nd}qq | sqq|j D]}| |r ||krd}|d"krX|d d#krd|dd d$f}n*|d%krpt ||d&}n|dt| }qq |s(|s(|j D]}| |r||kr|d'kr|dt| }|dt| }n6||s|||dd(kr|dt| }q(qd)D]4}| |r,||krX|dt| }qbq,|dddd}|S)*z Stem a Romanian word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode Fr<rr;NrrCr#)rPr#r*r'r.r&rabr)r/r+r$r5)rr,r-r%r"r&)r(r)rT)r0r1r4r2r&r3)r5r6r<r7r)r8r9r=r:r>r;rUrDrErFrGrXrHrIrJrKrk) rVrLrWrMrNr?r@rOrrArWrOr) rXrPrQrRrSrBrCrYrTr)rZr[uţr)rcrr^rrrr\r]r^)r`rarbrcrdrerfrgrhrirmrnrorprqrsr7rtrtru aeioăâî)rrr5rCuă)r-rr?r@_RomanianStemmer__vowelsr.rGrK _RomanianStemmer__step0_suffixesr.r _RomanianStemmer__step1_suffixes _RomanianStemmer__step2_suffixes _RomanianStemmer__step3_suffixesrrr-) rr4r1r2rCrBrFrJrZreplacement_doner!r!r"rs  $ & &                                         zRomanianStemmer.stemN) r'r(r)r*rvrwrxryrzrr!r!r!r"r!s0@ar!c@sLeZdZdZdZdZdZdZdZdZ dZ d d Z d d Z d dZ ddZdS)RussianStemmeram The Russian Snowball stemmer. :cvar __perfective_gerund_suffixes: Suffixes to be deleted. :type __perfective_gerund_suffixes: tuple :cvar __adjectival_suffixes: Suffixes to be deleted. :type __adjectival_suffixes: tuple :cvar __reflexive_suffixes: Suffixes to be deleted. :type __reflexive_suffixes: tuple :cvar __verb_suffixes: Suffixes to be deleted. :type __verb_suffixes: tuple :cvar __noun_suffixes: Suffixes to be deleted. :type __noun_suffixes: tuple :cvar __superlative_suffixes: Suffixes to be deleted. :type __superlative_suffixes: tuple :cvar __derivational_suffixes: Suffixes to be deleted. :type __derivational_suffixes: tuple :note: A detailed description of the Russian stemming algorithm can be found under http://snowball.tartarus.org/algorithms/russian/stemmer.html ) zivshis'zyvshis'vshis'ZivshiZyvshivshirZyvv)zui^ushchi^ui^uzui^ushchi^ai^az ui^ushchimiz ui^ushchymiz ui^ushchegoz ui^ushchogoz ui^ushchemuz ui^ushchomuz ui^ushchikhz ui^ushchykhz ui^ushchui^uz ui^ushchaiaz ui^ushchoi^uz ui^ushchei^u i^ushchi^ui^u i^ushchi^ai^az ui^ushcheez ui^ushchiez ui^ushchyez ui^ushchoez ui^ushchei`z ui^ushchii`z ui^ushchyi`z ui^ushchoi`z ui^ushchemz ui^ushchimz ui^ushchymz ui^ushchom i^ushchimi i^ushchymi i^ushchego i^ushchogo i^ushchemu i^ushchomu i^ushchikh i^ushchykh i^ushchui^u i^ushchai^a i^ushchoi^u i^ushchei^u i^ushchee i^ushchie i^ushchye i^ushchoe i^ushchei` i^ushchii` i^ushchyi` i^ushchoi` i^ushchem i^ushchim i^ushchym i^ushchom shchi^ui^u shchi^ai^az ivshi^ui^uz ivshi^ai^az yvshi^ui^uz yvshi^ai^ashchimishchymishchegoshchogoshchemushchomushchikhshchykhshchui^ushchai^ashchoi^ushchei^uZivshimiZivshymiZivshegoZivshogoZivshemuZivshomuZivshikhZivshykhzivshui^uzivshai^azivshoi^uzivshei^uZyvshimiZyvshymiZyvshegoZyvshogoZyvshemuZyvshomuZyvshikhZyvshykhzyvshui^uzyvshai^azyvshoi^uzyvshei^u vshi^ui^u vshi^ai^ashcheeshchieshchyeshchoeshchei`shchii`shchyi`shchoi`shchemshchimshchymshchomZivsheeZivshieZivshyeZivshoezivshei`zivshii`zivshyi`zivshoi`ZivshemZivshimZivshymZivshomZyvsheeZyvshieZyvshyeZyvshoezyvshei`zyvshii`zyvshyi`zyvshoi`ZyvshemZyvshimZyvshymZyvshomvshimivshymivshegovshogovshemuvshomuvshikhvshykhvshui^uvshai^avshoi^uvshei^uemi^ui^uemi^ai^anni^ui^unni^ai^avsheevshievshyevshoevshei`vshii`vshyi`vshoi`vshemvshimvshymvshomemimiemymiemegoemogoememuemomuemikhemykhemui^uemai^aemoi^uemei^unniminnyminnegonnogonnemunnomunnikhnnykhnnui^unnai^annoi^unnei^uemeeemieemyeemoeemei`emii`emyi`emoi`emememimemymemomnneenniennyennoennei`nnii`nnyi`nnoi`nnemnnimnnymnnomzi^ui^uzi^ai^aZimiZymiZegoZogoZemuZomuZikhZykhui^uzai^azoi^uzei^ur*ryeZoeei`ii`zyi`oi`r7rtymr`)zsi^azs').esh'zei`tezui`tezui^utzish'ri`tei^utnnoZilaZylaZenarZiliZyliZiloZyloZenozi^atZuetZenyzit'zyt'rrrrSr7rnorrHt'rzui`ilZylrtrrrZyti^ui`rlr)$zii^amizii^akhzi^amizii^amzi^akhZamiziei`zi^amZiemZakhzii^uz'i^uzii^az'i^aZevovrz'ervrrrrr7rar`ri^arr5rCrrrr!r4)zei`shezei`sh)zost'Zostc Cs||jkr|Sd}tt|D]}t||dkrd}qd=d?d@dAd@dBdCdDdCdEdFdGdFdHdIdJdIdKdLdMdLdNdOdPdOdQdRdSdRdTdUdVdUdWdXdYdXdZd[d\d[d]d^d_d^d`dadbda}|S)ca# Transliterate a Russian word into the Roman alphabet. A Russian word whose letters consist of the Cyrillic alphabet are transliterated into the Roman alphabet in order to ease the forthcoming stemming process. :param word: The word that is transliterated. :type word: unicode :return: the transliterated word. :rtype: unicode :note: This helper method is invoked by the stem method of the subclass RussianStemmer. It is not to be invoked directly! uАrаuБbбuВr~вuГgгuДriдuЕr5еuЁuёuЖzhжuЗzзuИrCиuЙrйuКrкuЛrlлuМrhмuНrнuОrоuПpпuРrрuСrсuТrтuУrуuФfфuХkhхuЦt^sцuЧrчuШshшuЩshchщuЪ''ъuЫr!ыuЬr4ьuЭr эuЮrюuЯrяr-rr4r!r!r"Z__cyrillic_to_romancs       !"#$%&'()*+,-./0123456789:;<=>?@AEz"RussianStemmer.__cyrillic_to_romancCs|ddddddddd d d d d ddddddddddddddddddd d!d"ddd#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;d<d=d>d?d@}|S)AaH Transliterate a Russian word back into the Cyrillic alphabet. A Russian word formerly transliterated into the Roman alphabet in order to ease the stemming process, is transliterated back into the Cyrillic alphabet, its original form. :param word: The word that is transliterated. :type word: str or unicode :return: word, the transliterated word. :rtype: unicode :note: This helper method is invoked by the stem method of the subclass RussianStemmer. It is not to be invoked directly! rrKrrLrDrEr=r>r?r@rrAr rJrr.rBrCrr/r5r(r)r*rr!r"r#r~r$r%r&rir'r+r,rCr-rlr0rhr1rr2rr3r4r5r6r7rr8rr9rr:r;r<rFrGr!rHr4rIrMrNr!r!r"Z__roman_to_cyrillics       $z"RussianStemmer.__roman_to_cyrillicN)r'r(r)r*rrrrrrrrrrrr!r!r!r"r{s m0&#0Wr{c@s8eZdZdZdZdZdZdZdZdZ dd Z d d Z d S) SpanishStemmeraR The Spanish Snowball stemmer. :cvar __vowels: The Spanish vowels. :type __vowels: unicode :cvar __step0_suffixes: Suffixes to be deleted in step 0 of the algorithm. :type __step0_suffixes: tuple :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2a_suffixes: Suffixes to be deleted in step 2a of the algorithm. :type __step2a_suffixes: tuple :cvar __step2b_suffixes: Suffixes to be deleted in step 2b of the algorithm. :type __step2b_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :note: A detailed description of the Spanish stemming algorithm can be found under http://snowball.tartarus.org/algorithms/spanish/stemmer.html uaeiouáéíóúü) ZselasZselosZselaZseloZlasZlesZlosZnosmerrrr)/ZamientosZimientosZamientoZimientoacionacionesucionesrranciaslogíasenciasrrZanzasrrZiblesrraciónrancialogíauciónenciarrrrrr^r_rrrrr idadrrrrrrrr) ZyeronyendoZyamosZyaisZyanyenZyasyesZyarZyouyó)`rrruiéramosuiésemosuaríaisrueríaisruiríaisrZieraisZieseisZasteisZisteisuábamosruásemosuaríanuaríasuaréisueríanueríasueréisuiríanuiríasuiréisZieranZiesenZieroniendoZierasZiesesZabaisZaraisZaseisuéamosuaránruaríaueránrueríauiránruiríaZieraZieserrZabanZaranZasenZaronrZabasrrrZasesuíaisrrr r r ruarérueréruiréZabar r rrluíanrruíasuáiséisuíarbr;ruruiórrrruísrr)rrr5rrrrrcCs|}||jkr|Sd}|||j\}}|||j}|jD]}||r@||sZq@|dt| ds|dt| dr|dt| dr||dt| }||dt| }||dt| }||dt| }qq@|j D]2}||s*q|dkr||rd}|dd}|dd}|dd}|d r|dd }|dd }|dd }|d r|dd }|dd }n$|d r@|dd }|dd }nZ||r@d}|d kr^|dt| }|dt| }|dt| }|dr@|dd }|dd }n|dkrt ||d}t ||d}n|dkrt ||d}t ||d}n|dkrt ||d}t ||d}np|dkr8|dt| }|dt| }|dt| }|dr@|dd}|dd}n|dkr|dt| }|dt| }|dt| }dD]6}||r||dt| }|dt| }q|n|dkr|dt| }|dt| }|dt| }|d r@|dd }|dd }n$|dt| }|dt| }qJq|s4|j D]^}||rV|t| dt| dkrV|dt| }|dt| }qqV|j D]v}||r|dt| }|dt| }|dkr*|dr|dd}|dr*|dd}q4q|j D]p}||r:|dt| }|dkr|dt| }|d ddkr|dr|dd}qq:||}|S) z Stem a Spanish word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode FN) ruándoruárruérr`uiéndoruírr]ZuyendorTrrrrr) rrrWrrrQrRr rrXrTrk)rYrUr%)rZrSr)r[rVrr)r r^r_r()r\r)r&rkr)rrrrr<)rrrar r(r)r5r)r-rrG_SpanishStemmer__vowelsrK_SpanishStemmer__step0_suffixesr.r@!_SpanishStemmer__replace_accented_SpanishStemmer__step1_suffixesr _SpanishStemmer__step2a_suffixes _SpanishStemmer__step2b_suffixes_SpanishStemmer__step3_suffixes)rr4r1rBrFrJrZpre_suffr!r!r"rs                                    .               zSpanishStemmer.stemcCs,|ddddddddd d S) u} Replaces all accented letters on a word with their non-accented counterparts. :param word: A spanish word, with or without accents :type word: str or unicode :return: a word with the accented letters (á, é, í, ó, ú) replaced with their non-accented counterparts (a, e, i, o, u) :rtype: str or unicode rrrr5rrCrrr rrMrNr!r!r"Z__replace_accentedls z!SpanishStemmer.__replace_accentedN) r'r(r)r*rbrcrerfrgrhrrdr!r!r!r"rOs1b2rOc@s,eZdZdZdZdZdZdZdZddZ d S) SwedishStemmera The Swedish Snowball stemmer. :cvar __vowels: The Swedish vowels. :type __vowels: unicode :cvar __s_ending: Letters that may directly appear before a word final 's'. :type __s_ending: unicode :cvar __step1_suffixes: Suffixes to be deleted in step 1 of the algorithm. :type __step1_suffixes: tuple :cvar __step2_suffixes: Suffixes to be deleted in step 2 of the algorithm. :type __step2_suffixes: tuple :cvar __step3_suffixes: Suffixes to be deleted in step 3 of the algorithm. :type __step3_suffixes: tuple :note: A detailed description of the Swedish stemming algorithm can be found under http://snowball.tartarus.org/algorithms/swedish/stemmer.html u aeiouyäåöZbcdfghjklmnoprtvy)%ZheternarrrZandenZarnasZernasZornasruZandetZarensZarnaZernaZornarZarnerZarenZadesZernsZaderr6rrrrbrrrorrrrrr5r)rrrrrrr)fulltlöstrrrcCs|}||jkr|S|||j}|jD]f}||r*|dkrh|d|jkr|dd}|dd}n$|dt| }|dt| }qq*|jD]*}||r|dd}|dd}qq|j D]F}||r|dkr|dt| }n|dkr |dd}qq|S)z Stem a Swedish word and return the stemmed form. :param word: The word that is stemmed. :type word: str or unicode :return: The stemmed form. :rtype: unicode rrNr)rrr)rkrl) r-rrD_SwedishStemmer__vowels_SwedishStemmer__step1_suffixesr._SwedishStemmer__s_endingr@_SwedishStemmer__step2_suffixes_SwedishStemmer__step3_suffixesrr!r!r"rs4             zSwedishStemmer.stemN) r'r(r)r*rmrornrprqrr!r!r!r"ris'ricsTddlm}ddddddd d d d dd ddddd}tdtdtdtdtddtjdd}|dkrzqP|tjkrtdqRt||||dd}dfdd|D}t d d!|d }d|}t d d!|d }tdtd"td# d$t|td%td& d$t|td"tdqRdS)'a< This function provides a demonstration of the Snowball stemmers. After invoking this function and specifying a language, it stems an excerpt of the Universal Declaration of Human Rights (which is a part of the NLTK corpus collection) and then prints out the original and the stemmed text. r)udhrzArabic_Alarabia-ArabiczDanish_Dansk-Latin1zDutch_Nederlands-Latin1zEnglish-Latin1zFinnish_Suomi-Latin1zFrench_Francais-Latin1zGerman_Deutsch-Latin1zHungarian_Magyar-UTF8zItalian_Italiano-Latin1zNorwegian-Latin1zPortuguese_Portugues-Latin1zRomanian_Romana-Latin2z Russian-UTF8zSpanish-Latin1zSwedish_Svenska-Latin1r z******************************zDemo for the Snowball stemmersz9Please enter the name of the language to be demonstrated /z"(enter 'exit' in order to leave): exitz@ Oops, there is no stemmer for this language. Please try again. Ni, c3s|]}|VqdSr$)r).0r4rr!r" -szdemo..z (.{,70})\sz\1\nzF----------------------------------------------------------------------ZORIGINALFz zSTEMMED RESULTS) nltk.corpusrrprintinputr.rrr0rrzrstripcenter)rrZ udhr_corpusrZexcerptZstemmedr!rxr"demosj    r) r*rr{rZ nltk.stemrZ nltk.stem.apirZnltk.stem.utilrrrr+r9r:rErLrr r3rrr5rErrrr!r{rOrirr!r!r!r"sz    R(+[`#A Yq zz\fOq