There's a crucial part in the process that says, the best place for the chatbot to learn is from real users, what if I already have that data, and would like to test the model on it.
Think of Interactive Learning, but in scale and possibly automated. Does such a feature already exist within RASA?
Think of Interactive Learning, but in scale and possibly automated.
Are you referring to something like reinforcement learning? Unfortunately something like that currently doesn't exist. Measuring the success of conversations is a tough problem (e.g. some users might give you positive feedback when the bot solved their problem, while others would simply leave the conversation). Something like external business metrics could do the trick (e.g. whether the user turned out buying something from you within the next 24h), but it's still hard. Another problem is that you probably want to have some degree of control over how your chatbot interacts with your users. Training the bot on user conversations without any double checking could potentially lead to problems (e.g. Microsoft once had an AI trained on Twitter data, which didn't turn out well).
Rasa is offering Rasa X for learning from real conversations. The community edition is a free, closed source product which helps you monitor and annotate real user conversations quickly.
Disclaimer: I am a software engineer working at Rasa.
