Integrating an Azure web app and Azure OpenAI with a virtual network

Question

I'm trying to secure my Azure OpenAI web app as much as possible, but I can't figure out how to keep traffic behind the application gateway inside my virtual network.

Here's what I have so far:

• Application gateway with a public IP (right now it's listening on port 80, there's no SSL) inside its own subnet.
• Web app (created via app services) behind the application gateway with the default *.azurewebsites.net domain. Using access restrictions, I only allowed access to this app from the gateway's public IP. This web app calls my Azure OpenAI deployment via the Python SDK.
• Azure OpenAI deployment allowing access from all networks. There's a private endpoint for Azure OpenAI, but it has no effect since under Networking > Firewalls and virtual networks I choose All networks.
• Right now, I'm not using any authentication mechanism for the users like Active Directory.

Diagram:

I would now like to limit network traffic like this:

• Put the web app in the same virtual network used by Azure OpenAI's private endpoint
• Allow traffic to Azure OpenAI only from the web app's subnet
• Limit Azure OpenAI traffic by using the private endpoint
• Maybe use a private endpoint for the web app

I started by enabling VNet integration for the web app and putting it in its own subnet. The subnet is in same virtual network as Azure OpenAI and has a service endpoint for Microsoft.CognitiveServices.

The app can no longer call Azure OpenAI (I get a 500 error). I tried whitelisting the app's subnet in the Azure OpenAI networking settings ("Allow access from Selected Networks and Private Endpoints") but that didn't help.

How should I change my setup so as little traffic as possible leaves my virtual network?

Answer 1

Integrating an Azure web app and Azure OpenAI with a virtual network.

To integrate Azure Web App and Azure OpenAI with a virtual network, follow the below steps

1. Disable all networks and enable Private Endpoint with your virtual network and subnet in Open AI

2. Once the Open AI Studio is created, it will be accessible only from the same virtual network. If you try to access it from another network, you will get the below error

3.Create a Web app with a Private endpoint using the same virtual network and subnet..

4. Once I create the web app, I will deploy the Open AI to the same web app.

5.If you try to access the Open AI from outside the network, you will get the below error.

6.The same is working if you try to access the Open AI from the same virtual network subnet address

To test the connection, I created a virtual machine within the same VNet and tested the Chat Service. It is working as expected.

Reference: DNS configuration & Stack Link