Calculate perplexity of word2vec model

Question

I trained Gensim W2V model on 500K sentences (around 60K) words and I want to calculate the perplexity.

What will be the best way to do so?
for 60K words, how can I check what will be a proper amount of data?

Thanks

score 2 · Answer 1 · answered Dec 13 '18 at 16:00

2

If you want to calculate the perplexity, you have first to retrieve the loss. On the gensim.models.word2vec.Word2Vec constructor, pass the compute_loss=True parameter - this way, gensim will store the loss for you while training. Once trained, you can call the get_latest_training_loss() method to retrieve the loss.

Since the loss in the cross-entropy loss of the skip-gram model, 2 to the power of the loss will give you the preplexity. (2**loss)

answered Dec 13 '18 at 16:00

bluesummers

11,365
8
72
108

I get loss of 36672808.0 does it make sense? – oren_isp Dec 16 '18 at 11:52
Did you run any training? Tell me what is the perplexity and the number of total words in your vocabulary, it will be easier to determine if it makes sense – bluesummers Dec 30 '18 at 10:38

Calculate perplexity of word2vec model

1 Answers1

Linked