gensim logo

gensim
gensim tagline

Get Expert Help

• machine learning, NLP, data mining

• custom SW design, development, optimizations

• corporate trainings & IT consulting

parsing.porter – Porter Stemming Algorithm

parsing.porter – Porter Stemming Algorithm

Porter Stemming Algorithm This is the Porter stemming algorithm, ported to Python from the version coded up in ANSI C by the author. It may be be regarded as canonical, in that it follows the algorithm presented in

Porter, 1980, An algorithm for suffix stripping, Program, Vol. 14, no. 3, pp 130-137,

only differing from it at the points maked –DEPARTURE– below.

See also http://www.tartarus.org/~martin/PorterStemmer

The algorithm as described in the paper could be exactly replicated by adjusting the points of DEPARTURE, but this is barely necessary, because (a) the points of DEPARTURE are definitely improvements, and (b) no encoding of the Porter stemmer I have seen is anything like as exact as this version, even with the points of DEPARTURE!

Vivake Gupta (v@nano.com)

Release 1: January 2001

Further adjustments by Santiago Bruno (bananabruno@gmail.com) to allow word input not restricted to one word per line, leading to:

Release 2: July 2008

Optimizations and cleanup of the code by Lars Buitinck, July 2012.

class gensim.parsing.porter.PorterStemmer

Bases: object

The main part of the stemming algorithm starts here. b is a buffer holding a word to be stemmed. The letters are in b[0], b[1] ... ending at b[k]. k is readjusted downwards as the stemming progresses.

Note that only lower case sequences are stemmed. Forcing to lower case should be done before stem(...) is called.

stem(w)

Stem the word w, return the stemmed form.

stem_documents(docs)
stem_sentence(txt)