0

I was working on Rasa NLU for intent classification, in link how shall I improve the confidence score for a given intent.

I have tried to give more training data but still the confidence score isn't increasing. Can anyone please let me know which parameters \ hyperparameters I can tune in order to get good confidence score. I did tried to all possible combinations provided in this link but still there was hardly any improvement.

I did checked the suggestion provided over here, but I am looking for granular tuning of the model such that it can perform better.

Thanks.

Edit 1: Please provide a valid reason for down-voting the question.

Aman Dalmia
  • 101
  • 1
  • 8
  • Is your training data proprietary or can you share it? How many intents/training examples per intent? which pipeline? How similar are your intents? Do you have entities? I don't feel like you've provided enough information to get more of an answer than just: provide more training data. – Caleb Keller Aug 31 '18 at 19:38
  • Sorry couldn't able to share the training data, Intents is around 44 and training distinct examples are around 1k per intent. – Aman Dalmia Sep 03 '18 at 06:16
  • Sounds way over fit. As in way too many training examples per intent. – Caleb Keller Sep 03 '18 at 20:23

2 Answers2

0

You can use tensorflow_embedding which gives confidence score near to .9, rather than using spacy_sklearn which provides neat to .3

Abhijeet
  • 1
0

Depending whether spaCy provides a good language model for your language, you should either use the spaCy pipeline (as it comes with pretrained models) or the tensorflow_embedding pipeline which works with any language but requires more training examples.

I think that your problems might be caused by overlapping training examples. An example to clarify:

## intent:ask_bot_name
- Tell me your name
- What is your name
- name please

## intent:ask_location_name
- Tell me the name
- What's the name
- name please

So I would suggest to go through your training data and have a look whether different intents have the same or very very similar examples.

Tobias
  • 1,880
  • 11
  • 17