ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup error when destroying resources with terraform

Question

I am creating Azure infra using terraform. I am able to create AppGateway in gateway subnet. The AppGateway required NSG rule to all access on ports 65200 - 65535, I have added the NSG. I am able to communicate with app behind AppGateway. But my jenkins pipeline fails when I try to destroy the complete setup, it says -

Error: Deleting Security Rule: (Name "AllowGatewayManagerInbound" / Network Security Group 
Name "gateway" / Resource Group "primary"): network.SecurityRulesClient#Delete: Failure 
sending request: StatusCode=400 -- Original Error: 
Code="ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup" Message="Network 
security group /subscriptions/****/resourceGroups/primary/providers/Microsoft.Network/networkSecurityGroups
/gateway blocks incoming internet traffic on ports 65200 - 65535 to subnet 
/subscriptions/****/resourceGroups/primary/providers/Microsoft.Network/virtualNetworks/primary/subnets/gateway, 
associated with Application Gateway subscriptions/****/resourceGroups/primary/providers/Microsoft.Network/applicationGateways/primary-centralus. 
This is not permitted for Application Gateways that have V2 Sku." Details=[]

Terraform code to create subnet, NSG and create AppGateway.

   resource "azurerm_network_security_group" "gateway" {
     name                = "gateway"
     location            = var.location
     resource_group_name = azurerm_resource_group.app.name
     tags                = var.tags
   }

   resource "azurerm_network_security_rule" "gateway_allow_gateway_manager_https_inbound" {
     name                        = "AllowGatewayManagerInbound"
     description                 = "Allow Azure application GatewayManager on management ports"
     resource_group_name         = azurerm_network_security_group.gateway.resource_group_name
     network_security_group_name = azurerm_network_security_group.gateway.name
     priority                    = 2510
     direction                   = "Inbound"
     access                      = "Allow"
     protocol                    = "Tcp"
     source_port_range           = "*"
     source_address_prefix       = "GatewayManager"
     destination_port_range      = "65200-65535"
     destination_address_prefix  = "*"
   }


   module "app_gateway" {
     source               = "../../modules/app_gateway"
     name                 = "${azurerm_resource_group.app.name}-${var.location}"
     location             = azurerm_resource_group.app.location
     resource_group_name  = azurerm_resource_group.app.name
     vnet_subnet_id       = azurerm_subnet.gateway.id
     app_public_dns_zone  = local.app_public_dns_zone
     a_record_domain_name = local.a_record_subdomain
     key_vault            = local.key_vault
     ssl_certificates     = local.ssl_certificates
     env                  = local.suffix
     tags                 = var.tags
     depends_on = [
       azurerm_network_security_group.gateway
     ]
   }

I have added depends_on relationship between AppGateway and NSG as AppGateway depends on NSG. I need help to destry these resources using terraform.

can you please share the complete NSG configuration with rules?? As, these needs to be examined considering the error that you are encountering. — Kartik Bhiwapurkar, Sep 13 '22 at 10:37

score 1 · Answer 1 · answered Sep 15 '22 at 14:28

• The ‘Destroy’ task through the terraform code that you are using is failing because inbound connectivity from the Jenkins pipeline is not possible through the NSG to the Azure resources, i.e., Application gateway in this case since the NSG is blocking the Jenkins pipeline access to the Azure resources on ports 65200 – 65535.

Thus, since you have deployed the ‘Application gateway’ in the ‘Gateway’ subnet and you have already allowed inbound network connectivity through the NSG to the application deployed behind the application gateway.

• Therefore, ensure that this allow rule’s priority is set higher than the deny rules for the same category. Also, allow TCP ports 65200 - 65535 for the application gateway v2 SKU with the destination subnet as ‘Any’ and source as ‘GatewayManager’ service tag for the communication between the Jenkin pipeline and the Azure Resource manager to happen.

Do check and ensure that the below rules in the NSG are set correctly: -

a) Outbound Internet connectivity can't be blocked. Default outbound rules in the NSG allow Internet connectivity.

b) Don't remove the default outbound rules.

c) Don't create other outbound rules that deny any outbound connectivity.

d) Traffic from the ‘AzureLoadBalancer’ tag with the destination subnet as Any must be allowed.

Finally, do check the priority for all the above stated rules and configurations for if the priority of the inbound rules is set higher than the deny rules, then they won’t be effective. Please find the below snapshot for your reference: -

Thank you so much [@Kartik](https://stackoverflow.com/users/16526895/kartik-bhiwapurkar). This answer is helpful, but we found a different solution through terraform. I will put that answer soon. — Sunil, Sep 16 '22 at 17:04

ApplicationGatewaySubnetInboundTrafficBlockedByNetworkSecurityGroup error when destroying resources with terraform

Terraform code to create subnet, NSG and create AppGateway.

1 Answers1