Stack Overflow
Stack Overflow

Questions tagged [perplexity]

Perplexity is a measurement of how well a probability distribution or probability model predicts a sample.

From Wikipedia

In information theory, perplexity is a measurement of how well a probability distribution or probability model predicts a sample. It may be used to compare probability models. A low perplexity indicates the probability distribution is good at predicting the sample.

40 questions
1
vote
1 answer

Elbow/knee in a curve in R

I've got this data processing: library(text2vec) ##Using perplexity for hold out set t1 <- Sys.time() perplex <- c() for (i in 3:25){ set.seed(17) lda_model2 <- LDA$new(n_topics = i) doc_topic_distr2 <- lda_model2$fit_transform(x = dtm, …
MelaniaCB
  • 427
  • 5
  • 16
1
vote
2 answers

Gensim Topic Modeling with Mallet Perplexity

I am topic modelling Harvard Library book title and subjects. I use Gensim Mallet Wrapper to model with Mallet's LDA. When I try to get Coherence and Perplexity values to see how good the model is, perplexity fails to calculate with below…
Tolga
  • 116
  • 2
  • 12
1
vote
2 answers

Python: handling large numbers

I need to count perplexity and I try to do it with def get_perplexity(test_set, model): perplexity = 1 n = 0 for word in test_set: n += 1 perplexity = perplexity * 1 / get_prob(model, word) perplexity =…
Petr Petrov
  • 4,090
  • 10
  • 31
  • 68
1
vote
1 answer

Check perplexity of a Language Model

I created a language model with Keras LSTM and now I want to assess wether it's good so I want to calculate perplexity. What is the best way to calc perplexity of a model in Python?
Cranjis
  • 1,590
  • 8
  • 31
  • 64
1
vote
0 answers

Sk-learn LDA for topic extraction, perplexity and score

Hello all! As apart of a project, I need to build a text classifier with the labeled data I have. A data point is composed of a single sentences and one of 3 categories for each sentence. I have extracted 5 topics from this database with LDA. What…
day_dreamer
  • 11
  • 2
1
vote
1 answer

Perplexity calculations rise between each significantly drop

I am training a conversational agent using LSTM and tensorflow's translation model. I use batchwise training, resulting in a significant drop in the training data perplexity after each epoch start. This drop can be explained by the way I read data…
simejo
  • 103
  • 1
  • 8
0
votes
1 answer

Challenges when calculating perplexity: using bidirectional models, and dealing with large text size and values, are my approaches reasonable?

Challenges when calculating perplexity: is my approach reasonable? I am trying to find a pre-trained language model that will work best for my text. The text is pretty specific in its language and content but there's no test data avaiable or budget…
Agnes
  • 19
  • 3
0
votes
0 answers

Laplace Smoothing - Greater perplexity of model language when increase the N of N-Gram Model

I'm training a Language Model using NLTK library of Python. To obtain a better result, I use the Laplace smoothing technique. But when I increase the N of N-gram model, my perplexity increases too, and I was expecting that the perplexity would…
Leticia
  • 1
  • 1
0
votes
0 answers

Negative Perplexity while using gensim LDA

I am using gensim's LDA and trying to see the perplexity for a certain number of topics. Perplexity for 1 : -7.903370624873305 Coherence Score for 1 : 0.8044880331838007 Perplexity for 2 : -8.269065851934347 Coherence Score for 2 : …
Stayne
  • 21
  • 5
0
votes
0 answers

Relation between perplexity and number of training samples

I'm trying to calculate the perplexity of some English language texts using NLTK. I'm trying to figure out how a simple n-gram model will perform with less training samples. The thing I don't understand is why does perplexity get lower if I decrease…
mrpostman889
  • 1
  • 1
0
votes
0 answers

How to choose the best LDA model when coherence and perplexy show opposed trends?

I have a corpus with around 1,500,000 documents of titles and abstracts from scientific research projects within STEM. I used Mallet https://mimno.github.io/Mallet/transforms to fit models from 10 to 790 topics in 10 topics increments (I allow for…
fcbt
  • 1
0
votes
1 answer

What is the held-out probability in Mallet LDA? How can we calculate Perplexity by the held-out probability?

I am new to mallet. Now I would like to get the perplexity scores for 10-100 topics in my lda model so I run the held-our probability, it gives me the value of -8926490.73103205 for topic=100, which seems a little bit off. Is that the perplexity…
May3514
  • 1
0
votes
0 answers

Comparing Perplexities of different N-gram Models

In my problem, I'm trying to compare the perplexity values of different N-gram models, say till N=4. However, I'm confused with the other results obtained using other methods. Here is my first implementation: - import nltk …
kartikeya saraswat
  • 23
  • 5
0
votes
1 answer

How to calculate perplexity of BERTopic?

Is there a way to calculate the perplexity of BERTopic? I am unable to find any such thing in the BERTopic library and in other places.
Inaam Ilahi
  • 105
  • 2
  • 9
0
votes
2 answers

How to find perplexity of bigram if probability of given bigram is 0

Given the formula to calculate the perplexity of a bigram (and probability with add-1 smoothing), Probability How does one proceed when one of the probabilities of the word per in the sentence to predict is 0? # just examples, don't mind the…
axelmukwena
  • 779
  • 7
  • 24
1
2
3