Resume Bot Framework dialog when triggered by external service

Question

The Scenario

I have a bot built using the Bot Framework with a series of dialogs. One of these dialogs gives the user the option of inputting some complex data via a web page by presenting a button to them. Clicking the button they are then taken to the site, fill out the data, save and are then directed back to the bot.

I want my bot to pause the dialog until it receives an event from my web page telling me the user has saved the data and then continue asking the user questions.

Before

I had a version implemented whereby I would store a ConversationReference before the user clicked the button and then when the external event happened I would send the cards and next messages I wanted to show (not in a dialog) from a webhook, that was fine but it got quite complicated/messy - I'd rather keep the whole app in one continuous dialog.

Idea 1: Use DirectLine API

I did some research and many people were suggesting using the DirectLine API. So I implemented this:

public async Task SendEventAsync(InternalEventMessage message, ConversationReference reference) {
        var client = new DirectLineClient(!String.IsNullOrEmpty(_settings.DirectLineSecret) ? _settings.DirectLineSecret : null);
        if (_settings.SiteUrl.Contains("localhost")) {
            client.BaseUri = new Uri(_settings.DirectLineServiceUrl);
        }

        var eventMessage = Activity.CreateEventActivity();
        //Wrong way round?!?
        eventMessage.From = reference.Bot;
        eventMessage.Type = ActivityTypes.Event;
        eventMessage.Value = message;
        var conversation = await client.Conversations.PostActivityAsync(reference.Conversation.Id, eventMessage as Activity);
    }

This uses the DirectLine client to send an event message to the serviceUrl using a stored ConversationReference, basically imitating a user (bot and user seem to be the wrong way round in the SDK). Checking for localhost was so that the DirectLine library pointed at the emulator server rather than https://directline.botframework.com.

In my dialog I call:

//method above shows input button and links to web page
context.Wait(WaitForAddressInput);
}

private async Task WaitForAddressInput(IDialogContext context, IAwaitable<IActivity> result) {
    var message = await result;
    switch (message.Type) {
        case ActivityTypes.Message:
            //TODO: Add response
            break;

        case ActivityTypes.Event:
            var eventMessage = message as IEventActivity;
            if (((JObject)eventMessage.Value).ToObject<InternalEventMessage>().Type == EventType.AddressInputComplete) {
                _addressResult = (await _tableService.ReadOrderById(Order.OrderId)).Address;
                await context.PostAsync($"Great}");
                context.Done(_addressResult);
            }
            break;
    }
}

This waits for any message from the user after the button has been shown and if our event matches then we proceed with the dialog.

This works locally using the emulator but, frustratingly, doesn't live. It fails to recognise channels created via webchat or Messenger. That is explained here: Microsoft Bot Framework DirectLine Can't Access Conversations

For security reasons, you can't use DirectLine to spy on messages from another conversation.

So I can't access a channel that I haven't created using DirectLine.

Idea 2: BotConnector

So I thought I'd try the BotConnector using similar code:

public async Task SendEventAsync(InternalEventMessage message, Microsoft.Bot.Connector.DirectLine.ConversationReference reference) {
    var botAccount = new ChannelAccount(reference.User.Id, reference.User.Name);
    var userAccount = new ChannelAccount(reference.Bot.Id, reference.Bot.Name);

    MicrosoftAppCredentials.TrustServiceUrl(reference.ServiceUrl);
    var connector = new ConnectorClient(new Uri(reference.ServiceUrl), new MicrosoftAppCredentials("xxxxxxxxxxxxxxxxxxxxxxxx", "xxxxxxxxxxxxxxxxxxxxxxxx"));

    connector.Credentials.InitializeServiceClient();

    var eventMessage = Activity.CreateMessageActivity();
    eventMessage.Recipient = botAccount;
    eventMessage.From = userAccount;
    eventMessage.Type = ActivityTypes.Event;
    eventMessage.Conversation = new ConversationAccount(id: reference.Conversation.Id);
    eventMessage.ServiceUrl = reference.ServiceUrl;
    eventMessage.Timestamp = DateTimeOffset.UtcNow;
    eventMessage.LocalTimestamp = DateTime.Now;
    eventMessage.ChannelId = reference.ChannelId;

    var result = await connector.Conversations.SendToConversationAsync(eventMessage as Microsoft.Bot.Connector.Activity);
}

This doesn't crash and I can see the event appear in the emulator request console but nothing happens, it seems to be ignored!

Idea 3: Try to imitate the bot service calling my bot

I haven't tried this yet because I think it might be the most time consuming but I was reading here about the service authentication and wondered if it would be possible to imitate the hosted bot service sending a message and send my event that way with the required data?

This seems like a fairly common scenario so I'm surprised I haven't come across a way to do this yet. If anyone has any other ideas on how I can send an event message to my bot from an external service then I'd love to hear it.

Update: See my answer below Eric's to see what I did.

Answer 1

Idea 1:

DirectLine is a channel, not a library to use in order to connect to channels. (For instance: you would not use Facebook Messenger to connect to Skype) DirectLineClient is useful for creating a client application that connects to the DirectLine channel through the Direct Line connector service.

Idea 2:

This method should work. In fact, the BotAuth library uses this method for the MagicNumber login flow within the CallbackController: https://github.com/MicrosoftDX/botauth/blob/9a0a9f1b665f4aa95b6d60d09346dda90d8b314e/CSharp/BotAuth/Controllers/CallbackController.cs

For your scenario, you should be able to construct a CardAction of type ActionTypes.OpenUrl that contains a value with the ConversationReference encoded in the url. Clicking the button will call an mvc controller that displays a page (saving the ConversationReference in a cookie or something) and when the user finishes adding the address on the page, use the ConversationReference to send an event to the bot (similar to how BotAuth resumes the conversation in the CallbackController).

Idea 3:

This would bypass the connector services, and is not a supported scenario. The link you shared explains the details of how authentication works in the Bot Framework, not how to bypass the connector services.

Answer 2

Eric's answer led me to solve the issue using the BotAuth example but, for completeness, here is what I did using Idea 2.

I created a CallbackController on my Bot Framework endpoint and then used the following code to send an event back to the awaiting dialog:

MicrosoftAppCredentials.TrustServiceUrl(reference.ServiceUrl);
            var message = reference.GetPostToBotMessage();
            message.Value = new InternalEventMessage(type);
            message.Type = ActivityTypes.Event;

            await Conversation.ResumeAsync(reference, message);

The dialog awaits with this code and continues:

context.Wait(WaitForAddressInput);
        }

        private async Task WaitForAddressInput(IDialogContext context,
                                      IAwaitable<IActivity> result)
        {
            var message = await result;
            switch (message.Type)
            {
                case ActivityTypes.Message:
                    //TODO: Add response
                    break;

                case ActivityTypes.Event:
                    //Process event and continue!
                    break;
            }
        }

This is the most complicated issue I've had with the Bot Framework and I found the docs a little lacking. Hope this helps someone!