gensim logo

gensim
gensim tagline

Get Expert Help From The Gensim Authors

Consulting in Machine Learning & NLP

• Commercial document similarity engine: ScaleText.ai

Corporate trainings in Python Data Science and Deep Learning

models.coherencemodel – Topic coherence pipeline

models.coherencemodel – Topic coherence pipeline

Calculate topic coherence for topic models. This is the implementation of the four stage topic coherence pipeline from the paper Michael Roeder, Andreas Both and Alexander Hinneburg: “Exploring the space of topic coherence measures”. Typically, CoherenceModel used for evaluation of topic models.

The four stage pipeline is basically:

  • Segmentation
  • Probability Estimation
  • Confirmation Measure
  • Aggregation

Implementation of this pipeline allows for the user to in essence “make” a coherence measure of his/her choice by choosing a method in each of the pipelines.

See also

gensim.topic_coherence
Internal functions for pipelines.
class gensim.models.coherencemodel.CoherenceModel(model=None, topics=None, texts=None, corpus=None, dictionary=None, window_size=None, keyed_vectors=None, coherence='c_v', topn=20, processes=-1)

Bases: gensim.interfaces.TransformationABC

Objects of this class allow for building and maintaining a model for topic coherence.

Examples

One way of using this feature is through providing a trained topic model. A dictionary has to be explicitly provided if the model does not contain a dictionary already

>>> from gensim.test.utils import common_corpus, common_dictionary
>>> from gensim.models.ldamodel import LdaModel
>>> from gensim.models.coherencemodel import CoherenceModel
>>>
>>> model = LdaModel(common_corpus, 5, common_dictionary)
>>>
>>> cm = CoherenceModel(model=model, corpus=common_corpus, coherence='u_mass')
>>> coherence = cm.get_coherence()  # get coherence value

Another way of using this feature is through providing tokenized topics such as

>>> from gensim.test.utils import common_corpus, common_dictionary
>>> from gensim.models.coherencemodel import CoherenceModel
>>> topics = [
...    ['human', 'computer', 'system', 'interface'],
...    ['graph', 'minors', 'trees', 'eps']
... ]
>>>
>>> cm = CoherenceModel(topics=topics, corpus=common_corpus, dictionary=common_dictionary, coherence='u_mass')
>>> coherence = cm.get_coherence()  # get coherence value
Parameters:
  • model (BaseTopicModel, optional) – Pre-trained topic model, should be provided if topics is not provided. Currently supports LdaModel, LdaMulticore, LdaMallet and LdaVowpalWabbit. Use topics parameter to plug in an as yet unsupported model.
  • topics (list of list of str, optional) – List of tokenized topics, if this is preferred over model - dictionary should be provided.
  • texts (list of list of str, optional) – Tokenized texts, needed for coherence models that use sliding window based (i.e. coherence=`c_something`) probability estimator .
  • corpus (iterable of list of (int, number), optional) – Corpus in BoW format.
  • dictionary (Dictionary, optional) – Gensim dictionary mapping of id word to create corpus. If model.id2word is present, this is not needed. If both are provided, passed dictionary will be used.
  • window_size (int, optional) – Is the size of the window to be used for coherence measures using boolean sliding window as their probability estimator. For ‘u_mass’ this doesn’t matter. If None - the default window sizes are used which are: ‘c_v’ - 110, ‘c_uci’ - 10, ‘c_npmi’ - 10.
  • coherence ({'u_mass', 'c_v', 'c_uci', 'c_npmi'}, optional) – Coherence measure to be used. Fastest method - ‘u_mass’, ‘c_uci’ also known as c_pmi. For ‘u_mass’ corpus should be provided, if texts is provided, it will be converted to corpus using the dictionary. For ‘c_v’, ‘c_uci’ and ‘c_npmi’ texts should be provided (corpus isn’t needed)
  • topn (int, optional) – Integer corresponding to the number of top words to be extracted from each topic.
  • processes (int, optional) – Number of processes to use for probability estimation phase, any value less than 1 will be interpreted as num_cpus - 1.
aggregate_measures(topic_coherences)

Aggregate the individual topic coherence measures using the pipeline’s aggregation function. Use self.measure.aggr(topic_coherences).

Parameters:topic_coherences (list of float) – List of calculated confirmation measure on each set in the segmented topics.
Returns:Arithmetic mean of all the values contained in confirmation measures.
Return type:float
compare_model_topics(model_topics)

Perform the coherence evaluation for each of the models.

Parameters:model_topics (list of list of str) – list of list of words for the model trained with that number of topics.
Returns:Sequence of pairs of average topic coherence and average coherence.
Return type:list of (float, float)

Notes

This first precomputes the probabilities once, then evaluates coherence for each model.

Since we have already precomputed the probabilities, this simply involves using the accumulated stats in the CoherenceModel to perform the evaluations, which should be pretty quick.

compare_models(models)

Compare topic models by coherence value.

Parameters:models (BaseTopicModel) – Sequence of topic models.
Returns:Sequence of pairs of average topic coherence and average coherence.
Return type:list of (float, float)
estimate_probabilities(segmented_topics=None)

Accumulate word occurrences and co-occurrences from texts or corpus using the optimal method for the chosen coherence metric.

Notes

This operation may take quite some time for the sliding window based coherence methods.

Parameters:segmented_topics (list of list of pair, optional) – Segmented topics, typically produced by segment_topics().
Returns:Corpus accumulator.
Return type:CorpusAccumulator
classmethod for_models(models, dictionary, topn=20, **kwargs)

Initialize a CoherenceModel with estimated probabilities for all of the given models. Use for_topics() method.

Parameters:
  • models (list of BaseTopicModel) – List of models to evaluate coherence of, each of it should implements get_topics() method.
  • dictionary (Dictionary) – Gensim dictionary mapping of id word.
  • topn (int, optional) – Integer corresponding to the number of top words to be extracted from each topic.
  • kwargs (object) – Sequence of arguments, see for_topics().
Returns:

CoherenceModel with estimated probabilities for all of the given models.

Return type:

CoherenceModel

Example

>>> from gensim.test.utils import common_corpus, common_dictionary
>>> from gensim.models.ldamodel import LdaModel
>>> from gensim.models.coherencemodel import CoherenceModel
>>>
>>> m1 = LdaModel(common_corpus, 3, common_dictionary)
>>> m2 = LdaModel(common_corpus, 5, common_dictionary)
>>>
>>> cm = CoherenceModel.for_models([m1, m2], common_dictionary, corpus=common_corpus, coherence='u_mass')
classmethod for_topics(topics_as_topn_terms, **kwargs)

Initialize a CoherenceModel with estimated probabilities for all of the given topics.

Parameters:topics_as_topn_terms (list of list of str) – Each element in the top-level list should be the list of topics for a model. The topics for the model should be a list of top-N words, one per topic.
Returns:CoherenceModel with estimated probabilities for all of the given models.
Return type:CoherenceModel
get_coherence()

Get coherence value based on pipeline parameters.

Returns:Value of coherence.
Return type:float
get_coherence_per_topic(segmented_topics=None, with_std=False, with_support=False)

Get list of coherence values for each topic based on pipeline parameters.

Parameters:
  • segmented_topics (list of list of (int, number)) – Topics.
  • with_std (bool, optional) – True to also include standard deviation across topic segment sets in addition to the mean coherence for each topic.
  • with_support (bool, optional) – True to also include support across topic segments. The support is defined as the number of pairwise similarity comparisons were used to compute the overall topic coherence.
Returns:

Sequence of similarity measure for each topic.

Return type:

list of float

classmethod load(fname, mmap=None)

Load an object previously saved using save() from a file.

Parameters:
  • fname (str) – Path to file that contains needed object.
  • mmap (str, optional) – Memory-map option. If the object was saved with large arrays stored separately, you can load these arrays via mmap (shared memory) using mmap=’r’. If the file being loaded is compressed (either ‘.gz’ or ‘.bz2’), then `mmap=None must be set.

See also

save()
Save object to file.
Returns:Object loaded from fname.
Return type:object
Raises:AttributeError – When called on an object instance instead of class (this is a class method).
measure

Make pipeline, according to coherence parameter value.

Returns:Pipeline that contains needed functions/method for calculated coherence.
Return type:namedtuple
model

Get self._model field.

Returns:Used model.
Return type:BaseTopicModel
save(fname_or_handle, separately=None, sep_limit=10485760, ignore=frozenset([]), pickle_protocol=2)

Save the object to a file.

Parameters:
  • fname_or_handle (str or file-like) – Path to output file or already opened file-like object. If the object is a file handle, no special array handling will be performed, all attributes will be saved to the same file.
  • separately (list of str or None, optional) –

    If None, automatically detect large numpy/scipy.sparse arrays in the object being stored, and store them into separate files. This prevent memory errors for large objects, and also allows memory-mapping the large arrays for efficient loading and sharing the large arrays in RAM between multiple processes.

    If list of str: store these attributes into separate files. The automated size check is not performed in this case.

  • sep_limit (int, optional) – Don’t store arrays smaller than this separately. In bytes.
  • ignore (frozenset of str, optional) – Attributes that shouldn’t be stored at all.
  • pickle_protocol (int, optional) – Protocol number for pickle.

See also

load()
Load object from file.
segment_topics()

Segment topic, alias for self.measure.seg(self.topics).

Returns:Segmented topics.
Return type:list of list of pair
static top_topics_as_word_lists(model, dictionary, topn=20)

Get topn topics as list of words.

Parameters:
  • model (BaseTopicModel) – Pre-trained topic model.
  • dictionary (Dictionary) – Gensim dictionary mapping of id word.
  • topn (int, optional) – Integer corresponding to the number of top words to be extracted from each topic.
Returns:

Top topics in list-of-list-of-words format.

Return type:

list of list of str

topics

Get topics self._topics.

Returns:Topics as list of tokens.
Return type:list of list of str
topn

Get number of top words self._topn.

Returns:Integer corresponding to the number of top words.
Return type:int