QnAMaker just terminates and throws me out of all dialogs in our bot

Question

We have started to use QnAMaker in our Bot more during the recent test period. I have noticed that sometimes when we've called QnAMakerDialog a lot (5 - 10 calls per minute), everything just terminates and and throws me back to bot framework main loop. I've traced to see what happens and I can see that we call base(new QnAMakerService(new QnAMakerAttribute(..),...) but after the call we're back in the man loop again. No exception, no errors no nothing.

We might have reached the upper limit for the free QnAMaker server (we have applied to increase the limit) but even if we have, I still need to trap this and signal this either to the user or log the fact in the error log.

Anyone else seen this behavior?

// Tommy

(I'll put the same question in the QnAMaker github)

You could trap this by not using QnAMakerDialog but doing your QnAMaker call directly on your code — Nicolas R, Apr 04 '18 at 11:38
When you call your QnAMaker app via CURL or Postman, are you receiving 403s? If so, you've hit the monthly quota and are stuck until it refreshes. 5-10 calls a minute shouldn't have you reach throttling, which would result in a 429 error code. I recommend following Nicolas R's instructions to try and drill down/capture the behavior. — Steven G., Apr 04 '18 at 18:24
@StevenG. My problem is that our code is based on the QnAMakerDialog class from Microsoft so we don't get any error codes from the class, it just "dies". According to the Qna FAQ, Current transaction limits are 10 calls per minute. I'm OK if we're reaching the limit, we have applied for more, the problem is that we don't get any message about it :-( I'll try Postman tomorrow and see what message I get back. // Tommy — Tommy Einarsson, Apr 04 '18 at 19:03
@StevenG. Tried Postan when the error occurred: "error": { "code": "RateLimitExceeded", "message": "Rate limit is exceeded. Try again later." } Bummer :-( — Tommy Einarsson, Apr 05 '18 at 14:47
Hi @TommyEinarsson, in my FAQ bot with QnA Maker, if Rate Limit is exceeded, the bot application would throw exception. You said No exception, no errors with your bot, could you share the code of your bot application and SDK version you are using now? — Fei Han, Apr 11 '18 at 08:46
@FeiHan: We're using Microsoft.Bot.Builder.CognitiveServices v1.1.2 and Microsoft.Bot.Builder v.3.14.1.1 Since we got our Premium subscription of QnAMaker, we haven't seen the problem. — Tommy Einarsson, Apr 13 '18 at 13:28

score 0 · Answer 1 · answered Apr 24 '18 at 06:20

I can see that we call base(new QnAMakerService(new QnAMakerAttribute(..),...) but after the call we're back in the man loop again. No exception, no errors no nothing.

You said No exception, no errors with your bot, to detect if Rate limit is exceeded, you can try to turn on Application Insights with your bot application, then you can trace the dependency calls to check the status of the requests to QnAMaker service.

Request to QnAMaker service with result code 429:

Besides, I do a test with QnAMaker bot using Microsoft.Bot.Builder v3.14.1.1 and Microsoft.Bot.Builder.CognitiveServices v1.1.2. and I find that the bot would throw error when Rate limit is exceeded.

QnaDialog:

[Serializable]
public class QnaDialog : QnAMakerDialog
{
    public QnaDialog() : base(new QnAMakerService(new QnAMakerAttribute("xxxxx", "xxxxx", "Sorry, I couldn't find an answer for that", 0.5)))
    {
    }

    protected override async Task RespondFromQnAMakerResultAsync(IDialogContext context, IMessageActivity message, QnAMakerResults result)
    {
        await context.PostAsync($"I found {result.Answers.Count} answer(s) that might help...{result.Answers.First().Answer}.");
        context.Done(true);
    }
}

packages.config:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Autofac" version="3.5.2" targetFramework="net46" />
  <package id="Chronic.Signed" version="0.3.2" targetFramework="net46" />
  <package id="Microsoft.AspNet.WebApi" version="5.2.3" targetFramework="net46" />
  <package id="Microsoft.AspNet.WebApi.Client" version="5.2.3" targetFramework="net46" />
  <package id="Microsoft.AspNet.WebApi.Core" version="5.2.3" targetFramework="net46" />
  <package id="Microsoft.AspNet.WebApi.WebHost" version="5.2.3" targetFramework="net46" />
  <package id="Microsoft.Bot.Builder" version="3.14.1.1" targetFramework="net46" />
  <package id="Microsoft.Bot.Builder.CognitiveServices" version="1.1.2" targetFramework="net46" />
  <package id="Microsoft.Bot.Connector" version="3.14.1.1" targetFramework="net46" />
  <package id="Microsoft.IdentityModel.Logging" version="1.1.4" targetFramework="net46" />
  <package id="Microsoft.IdentityModel.Protocol.Extensions" version="1.0.4.403061554" targetFramework="net46" />
  <package id="Microsoft.IdentityModel.Protocols" version="2.1.4" targetFramework="net46" />
  <package id="Microsoft.IdentityModel.Protocols.OpenIdConnect" version="2.1.4" targetFramework="net46" />
  <package id="Microsoft.IdentityModel.Tokens" version="5.1.4" targetFramework="net46" />
  <package id="Microsoft.Rest.ClientRuntime" version="2.3.2" targetFramework="net46" />
  <package id="Microsoft.WindowsAzure.ConfigurationManager" version="3.1.0" targetFramework="net46" />
  <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net46" />
  <package id="System.IdentityModel.Tokens.Jwt" version="5.1.4" targetFramework="net46" />
</packages>

Test result:

QnAMaker just terminates and throws me out of all dialogs in our bot

1 Answers1