Unable to create KB in QnA Maker when deployed via ARM (No Endpoint keys found)

Question

I am having an issue where I cannot create KBs in QnA Maker for services which I have deployed via ARM template/DevOps. There are a number of issues here and on Github, but the main suggestions (create all the resources in the same region, don't put anything else on the app service plan, delete and redeploy) have not worked for me. As noted the resources HAVE been created and deleted multiple times with the same names, so I don't know if that's part of the issue. The resources create just fine (cognitive service, app service, app service plan, azure search, and app insights), all in WestUS, but then I am unable to create a knowledge base either through the API or directly at qnamaker.ai. In both cases I get the error message:

No Endpoint keys found.

I can get the keys through Azure CLI, plus they are showing in the portal, so that's not the issue. It may perhaps be an issue with the Authorization EndpointKey which is generated/shown after publishing a new KB, but as I cannot create or publish one, I cannot find this key. Not sure if that is the key the error message is referring to.

Here is the ARM template I am using the set up the resources.

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "sites_etn_qnamaker_name": {
            "defaultValue": "etn-qnamaker",
            "type": "string"
        },
        "serverfarms_etn_qnamaker_name": {
            "defaultValue": "etn-qnamaker",
            "type": "string"
        },
        "components_etn_qnamaker_ai_name": {
            "defaultValue": "etn-qnamaker-ai",
            "type": "string"
        },
        "accounts_etn_qnamaker_name": {
            "defaultValue": "etn-qnamaker",
            "type": "string"
        },
        "searchServices_etnqnamaker_azsearch_name": {
            "defaultValue": "etnqnamaker-azsearch",
            "type": "string"
        },
        "smartdetectoralertrules_failure_anomalies___etn_qnamaker_ai_name": {
            "defaultValue": "failure anomalies - etn-qnamaker-ai",
            "type": "string"
        },
        "actiongroups_application_20insights_20smart_20detection_externalid": {
            "defaultValue": "/subscriptions/REDACTED/resourceGroups/avcnc-chatbot-rg/providers/microsoft.insights/actiongroups/application%20insights%20smart%20detection",
            "type": "string"
        }
    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.CognitiveServices/accounts",
            "apiVersion": "2017-04-18",
            "name": "[parameters('accounts_etn_qnamaker_name')]",
            "location": "westus",
            "sku": {
                "name": "S0"
            },
            "kind": "QnAMaker",
            "properties": {
                "apiProperties": {
                    "qnaRuntimeEndpoint": "[concat('https://', parameters('accounts_etn_qnamaker_name'), '.azurewebsites.net')]"
                },
                "customSubDomainName": "[parameters('accounts_etn_qnamaker_name')]"
            }
        },
        {
            "type": "Microsoft.Insights/components",
            "apiVersion": "2015-05-01",
            "name": "[parameters('components_etn_qnamaker_ai_name')]",
            "location": "westus",
            "tags": {
                "hidden-link:/subscriptions/REDACTED/resourceGroups/ENTP-Chatbot-rg/providers/Microsoft.Web/sites/etn-qnamaker": "Resource"
            },
            "kind": "web",
            "properties": {
                "Application_Type": "web"
            }
        },
        {
            "type": "Microsoft.Search/searchServices",
            "apiVersion": "2015-08-19",
            "name": "[parameters('searchServices_etnqnamaker_azsearch_name')]",
            "location": "West US",
            "sku": {
                "name": "basic"
            },
            "properties": {
                "replicaCount": 1,
                "partitionCount": 1,
                "hostingMode": "default"
            }
        },
        {
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "2018-02-01",
            "name": "[parameters('serverfarms_etn_qnamaker_name')]",
            "location": "West US",
            "sku": {
                "name": "S1",
                "tier": "Standard",
                "size": "S1",
                "family": "S",
                "capacity": 1
            },
            "kind": "app",
            "properties": {
                "perSiteScaling": false,
                "maximumElasticWorkerCount": 1,
                "isSpot": false,
                "reserved": false,
                "isXenon": false,
                "hyperV": false,
                "targetWorkerCount": 0,
                "targetWorkerSizeId": 0
            }
        },
        {
            "type": "microsoft.alertsmanagement/smartdetectoralertrules",
            "apiVersion": "2019-06-01",
            "name": "[parameters('smartdetectoralertrules_failure_anomalies___etn_qnamaker_ai_name')]",
            "location": "global",
            "dependsOn": [
                "[resourceId('microsoft.insights/components', parameters('components_etn_qnamaker_ai_name'))]"
            ],
            "properties": {
                "description": "Failure Anomalies notifies you of an unusual rise in the rate of failed HTTP requests or dependency calls.",
                "state": "Enabled",
                "severity": "Sev3",
                "frequency": "PT1M",
                "detector": {
                    "id": "FailureAnomaliesDetector",
                    "name": "Failure Anomalies",
                    "description": "Detects if your application experiences an abnormal rise in the rate of HTTP requests or dependency calls that are reported as failed. The anomaly detection uses machine learning algorithms and occurs in near real time, therefore there's no need to define a frequency for this signal.<br/></br/>To help you triage and diagnose the problem, an analysis of the characteristics of the failures and related telemetry is provided with the detection. This feature works for any app, hosted in the cloud or on your own servers, that generates request or dependency telemetry - for example, if you have a worker role that calls <a class=\"ext-smartDetecor-link\" href=\\\"https://learn.microsoft.com/en-us/azure/application-insights/app-insights-api-custom-events-metrics#trackrequest\\\" target=\\\"_blank\\\">TrackRequest()</a> or <a class=\"ext-smartDetecor-link\" href=\\\"https://learn.microsoft.com/en-us/azure/application-insights/app-insights-api-custom-events-metrics#trackdependency\\\" target=\\\"_blank\\\">TrackDependency()</a>.",
                    "supportedResourceTypes": [
                        "ApplicationInsights"
                    ],
                    "imagePaths": [
                        "https://globalsmartdetectors.blob.core.windows.net/detectors/FailureAnomaliesDetector/v0.18/FailureAnomaly.png"
                    ]
                },
                "scope": [
                    "[resourceId('microsoft.insights/components', parameters('components_etn_qnamaker_ai_name'))]"
                ],
                "actionGroups": {
                    "groupIds": [
                        "[parameters('actiongroups_application_20insights_20smart_20detection_externalid')]"
                    ]
                }
            }
        },
        {
            "type": "Microsoft.Web/sites",
            "apiVersion": "2018-11-01",
            "name": "[parameters('sites_etn_qnamaker_name')]",
            "location": "West US",
            "dependsOn": [
                "[resourceId('Microsoft.Web/serverfarms', parameters('serverfarms_etn_qnamaker_name'))]"
            ],
            "tags": {
                "hidden-related:/subscriptions/REDACTED/resourcegroups/ENTP-Chatbot-rg/providers/Microsoft.Web/serverfarms/etn-qnamaker": "empty"
            },
            "kind": "app",
            "properties": {
                "enabled": true,
                "hostNameSslStates": [
                    {
                        "name": "[concat(parameters('sites_etn_qnamaker_name'), '.azurewebsites.net')]",
                        "sslState": "Disabled",
                        "hostType": "Standard"
                    },
                    {
                        "name": "[concat(parameters('sites_etn_qnamaker_name'), '.scm.azurewebsites.net')]",
                        "sslState": "Disabled",
                        "hostType": "Repository"
                    }
                ],
                "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', parameters('serverfarms_etn_qnamaker_name'))]",
                "reserved": false,
                "isXenon": false,
                "hyperV": false,
                "scmSiteAlsoStopped": false,
                "clientAffinityEnabled": true,
                "clientCertEnabled": false,
                "hostNamesDisabled": false,
                "containerSize": 0,
                "dailyMemoryTimeQuota": 0,
                "httpsOnly": false,
                "redundancyMode": "None"
            }
        },
        {
            "type": "Microsoft.Web/sites/config",
            "apiVersion": "2018-11-01",
            "name": "[concat(parameters('sites_etn_qnamaker_name'), '/web')]",
            "location": "West US",
            "dependsOn": [
                "[resourceId('Microsoft.Web/sites', parameters('sites_etn_qnamaker_name'))]"
            ],
            "tags": {
                "hidden-related:/subscriptions/REDACTED/resourcegroups/ENTP-Chatbot-rg/providers/Microsoft.Web/serverfarms/etn-qnamaker": "empty"
            },
            "properties": {
                "numberOfWorkers": 1,
                "defaultDocuments": [
                    "Default.htm",
                    "Default.html",
                    "Default.asp",
                    "index.htm",
                    "index.html",
                    "iisstart.htm",
                    "default.aspx",
                    "index.php",
                    "hostingstart.html"
                ],
                "netFrameworkVersion": "v4.0",
                "phpVersion": "5.6",
                "requestTracingEnabled": false,
                "remoteDebuggingEnabled": false,
                "httpLoggingEnabled": false,
                "logsDirectorySizeLimit": 35,
                "detailedErrorLoggingEnabled": false,
                "publishingUsername": "[concat('$',parameters('sites_etn_qnamaker_name'))]",
                "scmType": "None",
                "use32BitWorkerProcess": true,
                "webSocketsEnabled": false,
                "alwaysOn": false,
                "managedPipelineMode": "Integrated",
                "virtualApplications": [
                    {
                        "virtualPath": "/",
                        "physicalPath": "site\\wwwroot",
                        "preloadEnabled": false
                    }
                ],
                "loadBalancing": "LeastRequests",
                "experiments": {
                    "rampUpRules": []
                },
                "autoHealEnabled": false,
                "cors": {
                    "allowedOrigins": [
                        "*"
                    ],
                    "supportCredentials": false
                },
                "localMySqlEnabled": false,
                "ipSecurityRestrictions": [
                    {
                        "ipAddress": "Any",
                        "action": "Allow",
                        "priority": 1,
                        "name": "Allow all",
                        "description": "Allow all access"
                    }
                ],
                "scmIpSecurityRestrictions": [
                    {
                        "ipAddress": "Any",
                        "action": "Allow",
                        "priority": 1,
                        "name": "Allow all",
                        "description": "Allow all access"
                    }
                ],
                "scmIpSecurityRestrictionsUseMain": false,
                "http20Enabled": false,
                "minTlsVersion": "1.2",
                "ftpsState": "AllAllowed",
                "reservedInstanceCount": 0
            }
        },
        {
            "type": "Microsoft.Web/sites/hostNameBindings",
            "apiVersion": "2018-11-01",
            "name": "[concat(parameters('sites_etn_qnamaker_name'), '/', parameters('sites_etn_qnamaker_name'), '.azurewebsites.net')]",
            "location": "West US",
            "dependsOn": [
                "[resourceId('Microsoft.Web/sites', parameters('sites_etn_qnamaker_name'))]"
            ],
            "properties": {
                "siteName": "[parameters('sites_etn_qnamaker_name')]",
                "hostNameType": "Verified"
            }
        }
    ]
}

Here are just a few of the sites I checked

• https://github.com/MicrosoftDocs/azure-docs/issues/44719
• https://github.com/MicrosoftDocs/azure-docs/issues/40089
• Unable to create knowledgebase for azure cognitive service (Error: "No Endpoint keys found.")

EDIT: KB creation fails both through qnamaker.ai and via API. On qnamaker.ai, I get this message when trying to create a KB:

And here is the PowerShell script I was using to try and create it programmatically:

$body = Get-Content '$(System.DefaultWorkingDirectory)/_AveryCreek_OEM_CSC_Bot/models/qnamaker/Avery_Creek_Commercial_QnA.json' | Out-String

$header = @{
  "Content-Type"="application/json"
  "Ocp-Apim-Subscription-Key"="$(QNA_KEY)"
}

Invoke-RestMethod -Uri "https://westus.api.cognitive.microsoft.com/qnamaker/v4.0/knowledgebases/create" -Method 'Post' -Body $body -Headers $header

Answer 1

Searching for issues with endpoint keys and qnamaker turns up a fair few results.

I've just closed a case with Azure support for the same issue, and here are some of the steps we checked on the way to fixing this, hopefully one of these will be useful for anyone having this issue in the future as the error message doesn't give you much to go on:

• First, check the troubleshooting FAQ https://learn.microsoft.com/en-us/azure/cognitive-services/qnamaker/troubleshooting. There's nothing on the endpoint keys issue, but when you hit something else it's a good starting point.
• All services - check your naming. For example, for me my search service was named differently than the rest of my config was expecting, and also my cognitive services runtime endpoint in the api-properties was incorrect. Still deployed though - you won't always get an error on the service itself if you provide incorrect names to later created services, you'll just fail at the point of creating your KBs.
• All services - check your SKUs. While there's no problem that I could find being on free/basic, you can only have 1 qna cognitive service on a free subscription, so you'll need to tear down and recreate or update as you go.
• QnA cognitive service - config settings (keys and values) are case-sensitive.
• Qna web app and web app plan - check your quotas haven't been hit, particularly memory and CPU.
• QnA Web App - You should be able to go the https://{endpoint}/qnamaker/corehealthstatus and see a positive json response like this (or if there's an initException, you've at least got another error to go on):

{"processId":4920,"runtimeVersion":"5.46.0","initException":"","startupTime":"10/28/2020 2:44:39 PM"}

• Qna Web App - You should also be able to go the https://{endpoint}/qnamaker/proxyhealthstatus and see a positive json response like this.:

{
  "coreVersion": "5.46.0",
  "coreProcessId": 4920,
  "coreUrl": "http://localhost:50061"
}

• Qna Web App - Don't try to create a KB, whether through the qnamaker portal or dynamically, if your app doesn't show similar successes on those two check endpoints - build a wait if need be. You'll almost certainly see the endpoint errors via the API if you hit it immediately.

For the check endpoints above, the endpoint is visible in the overview section of your web app in the portal, and usually is the name of your app e.g. https://example-app-qnamaker-webapp.azurewebsites.net/qnamaker/corehealthstatus if the app was called example-app-qnamaker-webapp. In my own creation scripts, I checked against coreProcessId > 0 and startupTime is a valid date to indicate service readiness before creating a KB.

EDIT: I'd also add that if it takes a long time to deploy, part of your config is probably wrong. Every time I've had things work correctly, it's been a rapid deployment (and that goes for the services, knowledge bases, and calls to both az cli and the qnamaker REST api).

Answer 2

I suspect you may have been downvoted because this looks an awful lot more like a bug report than a Stack Overflow question. From the first issue you linked:

We will go ahead and close this issue as this is a service level issue and the best way to report it if it occurs again is through the QnA portal from "General Enquiry through uservoice" option from the top right corner.

I'll try to answer you anyway. You say you've tried creating all the resources in the same region, but remember that resource groups have locations too. You should make sure the resource group is also in the same region according to the answer to the Stack Overflow question you linked to: Unable to create knowledgebase for azure cognitive service (Error: "No Endpoint keys found.")

It seems that there is sometimes the problem that the endpoint keys can only be found, if the Resource Group holding all resources for the QnA Maker Service (like App Service, Application Insights, Search Service and the Application Service Plan) is hosted in the same region as the QnA Maker Service itself.

I also see that you've tried not putting anything else on the app service plan, and you've tried deleting and redeploying. But you might also try just waiting a while, or retrying more persistently. From another GitHub issue:

These failures are intermittent, If I persistently retry a failure, the knowledgebase will often eventually get created.

And from this issue:

According to the QnA Maker team, this error is shown when the QnA Maker service has not finished provisioning. There appear to be service issues QnA Maker right now that are causing the provisioning process to take even longer than the time we wait in the script.

If you would like to raise an issue through UserVoice, I highly recommend posting it on the forum so that other people can see the problem and upvote it.