0

I tried several things to calculate the coherence score for a sklearn LDA model, but it does not work out. What is a way to calculate the Coherence score for a sklearn LDA model?

When I use the standard gensim code to calculate the coherence score, I receive the following error: ValueError: This topic model is not currently supported. Supported topic models should implement the get_topics method.```

Here is part of my code:

count_vectorizer = CountVectorizer(stop_words='english')

  # Fit and transform the processed titles

count_data = count_vectorizer.fit_transform(training_data_preprocessed['Input'])
tf = count_data




number_topics = 5
number_words = 5

# Create and fit the LDA model
lda = LDA(n_components=number_topics)
lda.fit(tf)

# Print the topics found by the LDA model
print("Topics found via LDA:")
print_topics(lda, count_vectorizer, number_words)
Selena
  • 17
  • 6
  • I used gensim in order to calculate it – Nir Elbaz Apr 04 '22 at 12:47
  • @NirElbaz. yes, however, I used a sklearn model, so when I tried to calculate the coherence score using the gensim function, I received the following error: ValueError: This topic model is not currently supported. Supported topic models should implement the `get_topics` method.``` – Selena Apr 04 '22 at 12:56
  • If you will share the code I might be able to help you, but having said that , LDA is one of more challenging goal in NLP, I spent a lot of time on "real data" and couldn't make a break trough – Nir Elbaz Apr 04 '22 at 13:59
  • Does this answer your question? [How do I calculate the coherence score of an sklearn LDA model?](https://stackoverflow.com/questions/60613532/how-do-i-calculate-the-coherence-score-of-an-sklearn-lda-model) – Ailurophile Apr 06 '22 at 11:58

1 Answers1

0

I think you can use this code below for coherence model in LDA:

# import library from gensim  
from gensim.models import CoherenceModel

# instantiate topic coherence model
cm = CoherenceModel(model=lda_model_15, corpus=bow_corpus, texts=docs, coherence='c_v')

# get topic coherence score
coherence_lda = cm.get_coherence() 
print(coherence_lda)
Hansen Idden
  • 31
  • 4