7

Recently i figured out that my AKS cluster holds a subnet which is too small. Therefor im trying to add a second subnet and nodepool which is possible with the Azure CNI nowadays and then create a single proper subnet instead and migrate it back.

During terraform plan all goes well with a valid response however while applying it throws an error.

Error: Error Creating/Updating Subnet "me-test-k8s-subnet2" (Virtual Network "me-test-k8s-vnet" / Resource Group "me-test-k8s-rg"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="NetcfgInvalidSubnet" Message="Subnet 'me-test-k8s-subnet2' is not valid in virtual network 'me-test-k8s-vnet'." Details=[]

  on main.tf line 28, in resource "azurerm_subnet" "subnet2":
  28: resource "azurerm_subnet" "subnet2" {

My original cluster is created with this configuration of Terraform:

  name     = "${var.cluster_name}-rg"
  location = "${var.location}"
}

resource "azurerm_virtual_network" "network" {
  name                = "${var.cluster_name}-vnet"
  location            = "${azurerm_resource_group.rg.location}"
  resource_group_name = "${azurerm_resource_group.rg.name}"
  address_space       = ["10.1.0.0/16"]
}

resource "azurerm_subnet" "subnet" {
  name                 = "${var.cluster_name}-subnet"
  resource_group_name  = "${azurerm_resource_group.rg.name}"
  address_prefixes     = ["10.1.0.0/24"]
  virtual_network_name = "${azurerm_virtual_network.network.name}"
}

To make things more easy i decided to first add the subnet to the network without the nodepool. This will bring me to this terraform plan:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_subnet.subnet2 will be created
  + resource "azurerm_subnet" "subnet2" {
      + address_prefix                                 = (known after apply)
      + address_prefixes                               = [
          + "10.2.0.0/22",
        ]
      + enforce_private_link_endpoint_network_policies = false
      + enforce_private_link_service_network_policies  = false
      + id                                             = (known after apply)
      + name                                           = "me-test-k8s-subnet2"
      + resource_group_name                            = "me-test-k8s-rg"
      + virtual_network_name                           = "me-test-k8s-vnet"
    }

Hope that someone can explain me why this error occurs.

Best, Pim

Dirkos
  • 488
  • 1
  • 10
  • 33
  • 1
    I may be misreading your setup, but are you adding a 10.2.0.0/22 subnet to a 10.1.0.0/16 network? ie- 10.2.0.0/22 is not part of that network – Howard_Roark Sep 15 '20 at 15:07
  • I face this issue as well.. ended up removing the cluster and create new one with a larger sunbet range – Amit Baranes Sep 15 '20 at 15:11
  • @Howard_Roark correct. I have a /24 subnet which i can not change. However if correct i can add a second subnet to the network, migrate everything and done. But i should be able to add a /22 subnet to a /16 vnet right? – Dirkos Sep 15 '20 at 15:16
  • 1
    When you have a 10.1.0.0/16 network that encompasses all IP's that look like 10.1.x.y (ie- they will all need to start with 10.1). Here you are adding a 10.2 that is not part of that range – Howard_Roark Sep 15 '20 at 15:18

3 Answers3

6

When creating a subnet in a virtual network, it is mandatory to check if it is not jumping out of the network range.

You are just out of the range with your network mask: 10.1.0.0/16

First host: 10.1.0.1    
Last  host: 10.1.255.254

And you are trying to create subnet 10.2.0.0/22.

For not overlapping with subnets that are already created, 10.1.4.0/22, can be accepted, for instance.

Oleh Tarasenko
  • 601
  • 5
  • 17
  • The problem is that the /24 is too small. I should be able to add a /22 subnet to a /16 network right? – Dirkos Sep 15 '20 at 15:14
  • Yes, you can go with 10.1.0.0/22, but you will overlap with 10.1.0.0/24 – Oleh Tarasenko Sep 15 '20 at 15:17
  • thanks, i will give it a try. The overlap is not a problem directly since i am going to migrate to the new subnet. I guess in this case everything overlaps that fits the vnet right? – Dirkos Sep 15 '20 at 15:19
  • You can go with 10.1.4.0/22 for instance, it will be ok – Oleh Tarasenko Sep 15 '20 at 15:19
  • 1
    Thanks @Oleh, maybe a nice addition to add this in the explanation answer as well. I just accepted it and the resource is now being created – Dirkos Sep 15 '20 at 15:28
2

As mentioned in my comment and in someone's answer, Azure is throwing this error because you are trying to add a 10.2.0.0/22 subnet to a 10.1.0.0/16 network. ie- 10.2.0.0/22 is not part of that network.

I also want to point out that when you run a plan that is not submitting the actual API calls to Azure to make the changes, which is why things looked fine to you when you ran your plan, but Azure complained when you tried to apply it. I think the explanation is good in this tutorial. The excerpts that are applicable are:

Once you are happy with your declared configuration, you can ask Terraform to generate an execution plan for it. The plan command in the CLI is used to generate an execution plan from a configuration. The execution plan tells you what changes Terraform would need to make to bring your current infrastructure to the declared state in your configuration.

If you accept the plan you can instruct Terraform to apply changes. Terraform will make the API calls required to implement the changes. If anything goes wrong terraform will not attempt to automatically rollback the infrastructure to the state it was in before running apply. This is because apply adheres to the plan

Howard_Roark
  • 4,088
  • 1
  • 14
  • 24
0

You might also run to a similar error if you try to deploy another vnet into a subscription where there already is a vnet with the same address space.

Kristian Schneider
  • 177
  • 1
  • 7