5

My question is related to the project I've just started working on, and it's a ChatBot.

The bot I want to build has a pretty simple task. It has to automatize the process of purchasing movie tickets. This is pretty close domain and the bot has all the required access to the cinema database. Of course it is okay for the bot to answer like “I don’t know” if user message is not related to the process of ordering movie tickets.

I already created a simple demo just to show it to a few people and see if they are interested in such a product. The demo uses simple DFA approach and some easy text matching with stemming. I hacked it in a day and it turned out that users were impressed that they are able to successfully order tickets they want. (The demo uses a connection to the cinema database to provide users all needed information to order tickets they desire).

My current goal is to create the next version, a more advanced one, especially in terms of Natural Language Understanding. For example, the demo version asks users to provide only one information in a single message, and doesn’t recognize if they provided more relevant information (movie title and time for example). I read that an useful technique here is called "Frame and slot semantics", and it seems to be promising, but I haven’t found any details about how to use this approach.

Moreover, I don’t know which approach is the best for improving Natural Language Understanding. For the most part, I consider:

  1. Using “standard” NLP techniques in order to understand user messages better. For example, synonym databases, spelling correction, part of speech tags, train some statistical based classifiers to capture similarities and other relations between words (or between the whole sentences if it’s possible?) etc.
  2. Use AIML to model the conversation flow. I’m not sure if it’s a good idea to use AIML in such a closed domain. I’ve never used it, so that’s the reason I’m asking.
  3. Use a more “modern” approach and use neural networks to train a classifier for user messages classification. It might, however, require a lot of labeled data
  4. Any other method I don’t know about?

Which approach is the most suitable for my goal?

Do you know where I can find more resources about how does “Frame and slot semantics” work in details? I'm referring to this PDF from Stanford when talking about frame and slot approach.

pkacprzak
  • 5,537
  • 1
  • 17
  • 37

1 Answers1

0

The question is pretty broad, but here are some thoughts and practical advice, based on experience with NLP and text-based machine learning in similar problem domains.

I'm assuming that although this is a "more advanced" version of your chatbot, the scope of work which can feasibly go into it is quite limited. In my opinion this is a very important factor as different methods widely differ in the amount and type of manual effort needed to make them work, and state-of-the-art techniques might be largely out of reach here.

Generally the two main approaches to consider would be rule-based and statistical. The first is traditionally more focused around pattern matching, and in the setting you describe (limited effort can be invested), would involve manually dealing with rules and/or patterns. An example for this approach would be using a closed- (but large) set of templates to match against user input (e.g. using regular expressions). This approach often has a "glass ceiling" in terms of performance, but can lead to pretty good results relatively quickly.

The statistical approach is more about giving some ML algorithm a bunch of data and letting it extract regularities from it, focusing the manual effort in collecting and labeling a good training set. In my opinion, in order to get "good enough" results the amount of data you'll need might be prohibitively large, unless you can come up with a way to easily collect large amounts of at least partially labeled data.

Practically I would suggest considering a hybrid approach here. Use some ML-based statistical general tools to extract information from user input, then apply manually built rules/ templates. For instance, you could use Google's Parsey McParseface to do syntactic parsing, then apply some rule engine on the results, e.g. match the verb against a list of possible actions like "buy", use the extracted grammatical relationships to find candidates for movie names, etc. This should get you to pretty good results quickly, as the strength of the syntactic parser would allow "understanding" even elaborate and potentially confusing sentences.

I would also suggest postponing some of the elements you think about doing, like spell-correction, and even stemming and synonyms DB - since the problem is relatively closed, you'll probably have better ROI from investing in a rule/template-framework and manual rule creation. This advice also applies to explicit modeling of conversation flow.

etov
  • 2,972
  • 2
  • 22
  • 36
  • Thanks for the answer. Unfortunately, I cannot use Parsey McParsefase, because the bot is targeted not for English language. However, since I expect the user messages to be quite short, I'm think that considering all possible partitions of a message into phrases and for each such phrase trying to classify it as a "movie title", "datetime", "movie type", and other intents can be useful. Moreover, I consider some sort of classifying verbs into classes corresponding to actions to perform. – pkacprzak Jun 08 '16 at 17:14
  • There are syntactic parsers for most languages, e.g. Stanford's parser (http://nlp.stanford.edu/software/lex-parser.shtml) has models for Chinese, German, Arabic, Italian and more, so the approach in general should still be feasible (what language(s) are you targeting?). From my experience, user input, even in closed domain, can be extremely variable, I would think twice about going with the unstructured approach you consider – etov Jun 09 '16 at 13:05