I found a way!
If I want to use the environment variables set under App Settings:
Set objWSH = CreateObject("WScript.Shell")
Set objUserVariables = objWSH.Environment("Process")
Response.Write(objUserVariables("APPSETTING_testAppSet"))
the prefix APPSETTING_ will be different if the variable is stored under Connection String
One other way I was able to use is to store the DB connection string in the Azure Key Vault. And then use OAuth access token to access Azure Key Vault. In this method, you have to send a POST request to Azure with ClientID and ClientSecret in the request body, then you will get an access token from the HTTP response. After that, send a GET request to the Key Vault endpoint with the access token in the request header. Then you will get the value of the key vault secret from the HTTP response.
Another way to do it is to use the MSI_ENDPOINT and MSI_SECRET and send the HTTP request to get the access token. And with that access token, you can access a key vault secret as well (you have to make sure that the Key Vault Access Policy is setup correctly).