How do identify if the reply is user input or suggest action click - Bot Framework

Question

We've built a QnA bot in our organization for internal use, using BotFramework V4 and C# (.Net Core). We also want to use Bing spell check to fix typos in questions.

But the "correction" makes troubles in some multi-turn conversation: Bing spell check may replace abbervation and special terms, then QnA service may not catch the right question and answer.

We took the example from the tutorial, that use CardAction to prompt related questions:

foreach (var prompt in result.Context.Prompts) {
    buttonList.Add(
        new CardAction()
        {
            Value = prompt.DisplayText,
            Type = ActionTypes.ImBack,
            Title = prompt.DisplayText,
        });
}

Following an example (in French) when using the bot on Microsoft Teams:

My question is: how can I determine if the message is from user click on suggest button (for example, get the object CardAction and its property Type) ? I had look in turnContext parameter provided in OnMessageActivityAsync method. but nothing found.

I would like to apply Bing spell check for directly user input, but not for suggest actions.

Thank you. (p.s.) i found a similar question to botframework v3 without accepted answer, i'm using v4.

Answer 1

In channels like Teams where there is a specific imBack action type that's distinct from postBack and messageBack, there is usually no way to distinguish an incoming imBack action from a typed message.

To get your desired behavior, you should use postBack or messageBack. They work the same way as each other in Teams, but messageBack will communicate your intentions better. The idea is that you will be sending visible text along with invisible metadata in the same message from the user to the bot.

foreach (var prompt in result.Context.Prompts) {
    buttonList.Add(
        new CardAction()
        {
            Type = ActionTypes.MessageBack,
            Value = new { source = "button" },
            Title = prompt.DisplayText,
            Text = prompt.DisplayText,
            DisplayText = prompt.DisplayText,
        });
}

Why are we using prompt.DisplayText three times here? The title is displayed as the label on the button, the text populates the activity's Text property and the display text is what's rendered in the conversation.

In Teams, the Value property of a postBack or messageBack has to be an object and not a string. if you tried to say Value = "button" then you'd get an empty object in the incoming activity. Since the Value property of the action populates the Value property of the activity, you can tell if the incoming activity is from a button like this:

if (turnContext.Activity.Value != null
    && JToken.FromObject(turnContext.Activity.Value) is JObject value
    && value["source"] == "button")

Since my tests have so far indicated that typed messages never populate the Value property, you could simplify that check to this:

if (turnContext.Activity.Value != null)

Note that this answer only applies to Teams. There are different rules for determining if an activity came from a button on different channels, and those rules are not currently documented anywhere. If you're interested, I have written a method for my cards library that accomplishes the task of determining if an activity came from a button on any channel. Once the library is released, you can call that method and see if it returns null.