4

I have the following code loading Azure Key Vault secrets into the configuration of a worker process on startup. The process is configured in Azure and associated to a certificate which is also installed on the machine where the process runs.

This code works most of the time, which is the infuriating problem. A few times a day, on the call to Build(), connecting to the key vault fails with a message of Unauthorized, and takes down the entire application. Then, it inexplicably starts working again on restart. I believe this may have something to do with Azure Active Directory token expiration, but I don't know what to do about it, as the code below is almost verbatim from the documentation. This process is long-running and unattended, so random failures are not acceptable.

I have tried adding logic to wait a few seconds on failure and retry, but it didn't seem to help. Using the certificate method of authentication is a hard requirement from the security team.

using (var certStore = new X509Store(certLocation, StoreLocation.CurrentUser))
{
    try
    {
        certStore.Open(OpenFlags.ReadOnly | OpenFlags.OpenExistingOnly);
        var certSearch = certStore.Certificates.Find(X509FindType.FindByThumbprint, thumbprint, false);

        // We expect a single result
        if (certSearch.Count == 1)
        {
            var result = certSearch.OfType<X509Certificate2>().Single();

            //Add the key vault as a config provider
            builder.AddAzureKeyVault($"https://{config[keyVaultName]}.vault.azure.net/",
                                     config[applicationId],
                                     result);

            //This will fail if Keyvault is not configured properly
            builder.Build();                                    
        }
    }
    finally
    {
         certStore.Close();
    }
}
SouthShoreAK
  • 4,176
  • 2
  • 26
  • 48
  • 1
    not really related but maybe you could you managed identity rather than using a cert ? – Thomas Jan 14 '20 at 09:37
  • What documentation are you referring to in this issue? This does not look like any Key Vault code that I have seen. – Matt Small Feb 07 '20 at 13:59
  • @MattSmall See the section on using x509 certificates: https://learn.microsoft.com/en-us/aspnet/core/security/key-vault-configuration?view=aspnetcore-3.1 – SouthShoreAK Feb 12 '20 at 14:15
  • A couple of things: 1) As stated above, an MSI is preferable in an Azure environment. I believe that using the certificate vs. the MSI in an Azure environment is actually less secure as the process for acquiring a token goes over the wire whereas a token is delivered directly to the process in an MSI scenario. 2) In order to get to the bottom of this specific issue, you need to set up logging as described in the article: When the app fails to load configuration using the provider, an error message is written to the ASP.NET Core Logging. There's a link to it in the docs. – Matt Small Feb 12 '20 at 14:24
  • @MattSmall the process having this issue does not run in Azure, so MSI is not an option. It is a local service that connects to a key vault in Azure. This is why the certificate is used. The logging simply returns a message that says "401:Unauthorized" - it looks like there is a problem in an underlying HttpClient somewhere – SouthShoreAK Feb 13 '20 at 16:41
  • This looks relevant. https://azidentity.azurewebsites.net/post/2019/07/31/key-vault-client-why-am-i-seeing-http-401 – Troy Witthoeft May 27 '20 at 22:38

0 Answers0