gensim logo

gensim
gensim tagline

Get Expert Help From The Gensim Authors

Consulting in Machine Learning & NLP

• Commercial document similarity engine: ScaleText.ai

Corporate trainings in Python Data Science and Deep Learning

corpora._mmreader – Reader for corpus in the Matrix Market format.

corpora._mmreader – Reader for corpus in the Matrix Market format.

Reader for corpus in the Matrix Market format.

class gensim.corpora._mmreader.MmReader(input, transposed=True)

Bases: object

Matrix market file reader (fast Cython version), used internally in MmCorpus.

Wrap a term-document matrix on disk (in matrix-market format), and present it as an object which supports iteration over the rows (~documents).

num_docs

int – Number of documents in market matrix file.

num_terms

int – Number of terms.

num_nnz

int – Number of non-zero terms.

Notes

Note that the file is read into memory one document at a time, not the whole matrix at once (unlike e.g. scipy.io.mmread and other implementations). This allows us to process corpora which are larger than the available RAM.

Parameters:
  • input ({str, file-like object}) – Path to the input file in MM format or a file-like object that supports seek() (e.g. smart_open objects).
  • transposed (bool, optional) – Do lines represent doc_id, term_id, value, instead of term_id, doc_id, value?
docbyoffset(self, offset)

Get the document at file offset offset (in bytes).

Parameters:offset (int) – File offset, in bytes, of the desired document.
Returns:Document in sparse bag-of-words format.
Return type:list of (int, str)
input

input – object

num_docs

num_docs – ‘long long’

num_nnz

num_nnz – ‘long long’

num_terms

num_terms – ‘long long’

skip_headers(self, input_file)

Skip file headers that appear before the first document.

Parameters:input_file (iterable of str) – Iterable taken from file in MM format.
transposed

transposed – ‘bool’