gensim logo

gensim tagline

Get Expert Help

• machine learning, NLP, data mining

• custom SW design, development, optimizations

• corporate trainings & IT consulting

corpora.csvcorpus – Corpus in CSV format

corpora.csvcorpus – Corpus in CSV format

Corpus in CSV format.

class gensim.corpora.csvcorpus.CsvCorpus(fname, labels)

Bases: gensim.interfaces.CorpusABC

Corpus in CSV format. The CSV delimiter, headers etc. are guessed automatically based on the file content.

All row values are expected to be ints/floats.

Initialize the corpus from a file. labels = are class labels present in the input file? => skip the first column

load(fname, mmap=None)

Load a previously saved object from file (also see save).

If the object was saved with large arrays stored separately, you can load these arrays via mmap (shared memory) using mmap=’r’. Default: don’t use mmap, load large arrays as normal objects.

If the file being loaded is compressed (either ‘.gz’ or ‘.bz2’), then mmap=None must be set. Load will raise an IOError if this condition is encountered.

save(*args, **kwargs)
save_corpus(fname, corpus, id2word=None, metadata=False)

Save an existing corpus to disk.

Some formats also support saving the dictionary (feature_id->word mapping), which can in this case be provided by the optional id2word parameter.

>>> MmCorpus.save_corpus('', corpus)

Some corpora also support an index of where each document begins, so that the documents on disk can be accessed in O(1) time (see the corpora.IndexedCorpus base class). In this case, save_corpus is automatically called internally by serialize, which does save_corpus plus saves the index at the same time, so you want to store the corpus with:

>>> MmCorpus.serialize('', corpus) # stores index as well, allowing random access to individual documents

Calling serialize() is preferred to calling save_corpus().