3

Due to a recent layoff I got bombarded to azure admin out of the blue. I am pretty new to this and haven't yet got the chance to follow an admin course.

Facing the following issue: We host a couple of websites on an Azure Windows Server VM running IIS. These are accessible through an application gateway with a public IP. I was asked to add two new listeners for a new part of the website. I created the appropriate targeting in the backend pool, created http and https settings and added the listeners and Rules. However, when browsing to the site, it throws a 502 error and when i check the backend health, it gives below error.

enter image description here

Cannot connect to backend server. Check whether any NSG/UDR/Firewall is blocking access to server. Check if application is running on correct port.

I opened up the appropriate inbound ports on the NSG of the AZ Web interface on the VM and also on the local firewall of the server hosting IIS. AFAIK there are no additional NSG rules on the application gateway.

What am i missing here? :s

Ked Mardemootoo
  • 1,480
  • 1
  • 5
  • 19
Dries Van den Bergh
  • 61
  • 1
  • 8

4 Answers4

4

I have extensive experience working with Application Gateways and I can tell you that a 502 Bad Gateway means something is definitely wrong at the backend or misconfigured AGW settings - that's what the error says, so nothing surprising. From my experience here are different scenarios I've faced for this error:

  1. Backend server can't be reached due to an NSG Rule controlling access from the AGW subnet to the backend subnet.
  2. Backend server can be reached but the port is not opened at the server's firewall.
  3. Backend server can be reached, port is opened but application is not listening on those ports or application is not even running.
  4. AGW listeners were misconfigured.

Here's what you can try:

First validate whether the Application and VM are fine by trying to access the application from another VM in the same subnet.

Next, try to get a VM in a different subnet and try to access the application, to mimic the AGW trying to connect to the backend. This will help you validate whether your NSGs are properly configured.

Lastly, revisit all the AGW settings and look for any misconfiguration in the listeners or other settings. (Added this based on your comments).

Taking this approach to troubleshooting will quickly help you identify which layer is causing the issue. Also, it would be a good practice to start documenting all AGW errors you get along your journey and also the remediation steps etc. This will help you tremendously in the future - this is not the last time you'll face issues with your AGW!

Ked Mardemootoo
  • 1,480
  • 1
  • 5
  • 19
  • 1
    Hi, Appreciate the tips. I have a NSG allowing connection inbound an outbound in the IIS server's virtual subnet. There is no NSG in the subnet of the application gateway. The corresponding ports are listening on the vm, tested this with telnet from another vm. I am really out of ideas, we have a bunch of other subsites configured exactly the same and they work just fine.. – Dries Van den Bergh Oct 06 '21 at 11:52
  • 2
    Ok did you also verify that the listeners are using the correct backend pool? If you've not tried it yet, I'd suggest removing all the changes you've done and set it up again from scratch. There could be something else you've missed. Also what about adding additional (but different) listener for one of those existing subsites that already has a working listener? Just to confirm you're configuring it the right way – Ked Mardemootoo Oct 06 '21 at 12:19
  • 2
    Figured it out, thanks a bunch for those last simple tips :D. Deleting and recreating the listeners revealed i just made a typo in the destination port of the HTTP setting.... Cant believe I spend half a day troubleshooting this.. – Dries Van den Bergh Oct 06 '21 at 13:52
  • 1
    Awesome, I'll edit my answer (and my own documentation) to add that in too! – Ked Mardemootoo Oct 06 '21 at 14:31
0

If you've checked your firewall issues and it's not solving the problem it could be user permissions on your VM.

I then ran the following command in ps and it sorted it for me.

** netsh http add urlacl url="http://*:{port}/" user="Everyone" **

A good test to see if this will work is if you can access your app using invoke-webrequest and using a localhost URI, but not using the server's NIC Private IP.

You'll also need to set your host address to use the wildcard in your config file.

CodingCretin
  • 401
  • 4
  • 9
0

In my case, I had to change the Backend Pool target from FDQN to IP to get a more specific error message, which was: "Backend certificate is invalid. Current date is not within the "Valid from" and "Valid to" date range on the certificate."

And that was because of a missing SSL Certificate binding in the server: 0.0.0.0:443, in my case.

List them with: netsh http show sslcert

RookieRoo
  • 161
  • 1
  • 6
0

In my case the backend pool was a RHEL 8 which had the firewall blocking all incoming connections by default. I used sudo firewall-cmd --add-port=8080/tcp --permanent in the machine without changing anything in the application gateway configs

MostafaBakr
  • 107
  • 11