1

How do I prevent the Azure cognitive search service from becoming cold after a period of no load?

I use QnaMaker in chatbots (Managed QnA Maker Service version).

QnaMaker seems to idle after not being used for a while. The first query takes 7 seconds to complete. Every query after the first one completes within a second.

The chatbots themselves (developed with Microsoft Bot Framework V4 nodejs) show no latency. Even after not being used for a while (thanks to the "Always on" feature in the bots appservice)

I use the stable GA version of QnaMaker and the managed version (in preview). In the first version, the appservice for QnaMaker has the feature "Always On" enabled. For the preview version, there is no appservice I can check.

Chatbotcode initialization QnaMaker

const { QnAMaker } = require('botbuilder-ai');
        const endpoint_fr = {
            knowledgeBaseId: process.env.QnAKbId_fr,
            endpointKey: process.env.QnaEndpointKey_fr,
            host: process.env.QnaHostName_fr
        };
        try {
            this.qnaMaker_fr = new QnAMaker(endpoint_fr, {});
        } catch (err) {
            console.warn(`QnAMaker Exception: ${err} Check your QnAMaker configuration in .env`);
        }

the actual call to QnAMaker service

   qnaResults = await this.qnaMaker_fr.getAnswers(stepContext.context);

The QnAMaker stack is linked to a Azure Search-resource. The one I am using has pricing tier Basic (1 replica, partition and search unit) 10 indexes are in use (15 are allowed) to store 7 knowledge bases. location is West Europe.

How do I prevent the Azure cognitive search service from becoming cold after a period of no load?

[update] Did some more digging and came to the conclusion that this cold start only occured for knowledge bases in the (preview) managed service. I decided to move all KB's to the stable version and the cold start problems stopped. This could also have to do with the fact that I am in Western Europe and the managed version is available in Northern Europe only

billoverton
  • 2,705
  • 2
  • 9
  • 32
Hessel
  • 613
  • 1
  • 6
  • 15
  • Could you please elaborate a bit more on your architecture? I'm hearing that you are using QnA Maker service, the bot service, and Azure Cognitive Search, but not sure how each is being called and from where, and that will affect how to fix cold-start problems. – Jennifer Marsman - MSFT Apr 09 '21 at 14:21
  • hi @JenniferMarsman-MSFT. I did add some additional information. Hope this is what you were looking for. – Hessel Apr 09 '21 at 15:18

3 Answers3

2

I'm glad that you are using "Always On". The QnA Maker team sometimes recommends adding quick availability/web test using Azure app insights monitor (see https://learn.microsoft.com/azure/azure-monitor/app/monitor-web-app-availability). This is essentially a probing service every few seconds that can resolve the cold-start problem.

Jennifer Marsman - MSFT
  • 5,167
  • 1
  • 25
  • 24
  • Hi Jennifer, Thanks for your answer. Wrt the teams recommendation: do they mean a simple call/ping to the service (via a url call)? My question wrt that solution is that I expect it will not trigger the QnAMaker since I am not able to send a question via a url call to qnamaker service. – Hessel Apr 12 '21 at 07:39
2

I'm not sure about App Insights, but I do have a regular test I run against the QnA Maker service, primarily to monitor uptime, but it would double as keeping the service "warm". I set up an Azure Function (linked to the same App Service Plan as the QnA Maker app service) and just have it make a standard REST query at a defined interval (I use 15 minutes, you can use whatever and especially if you are using standard service tier there is no limit or extra costs on number of queries). You will just need to get the appropriate keys from the resource to make the request and store in your configuration or Key Vault, which I assume you are already familiar with from your bot itself. Should not create any incremental costs for you.

If it is possible to keep the service warm with just a ping, Jennifer's suggestion may work, but I can tell you that I have had 0 issues with the automated tests I've been running via Azure Functions.

billoverton
  • 2,705
  • 2
  • 9
  • 32
  • Thanks Bill. I narrowed this issue down to the managed version of the QnA service (which is in Preview). I moved all KB's back to the stable version and the cold start issues stopped – Hessel Apr 14 '21 at 08:35
  • Glad you got it resolved. To close this question out your can submit your own answer with the information you provided in this comment and accept it. That will also help others coming to this page with similar issues. – billoverton May 03 '21 at 13:55
0

I moved all my QNA knowledge bases back to the stable version. The managed version seemed to cause the cold start. Not sure why but my bots are performing again.

Hessel
  • 613
  • 1
  • 6
  • 15