7

I have an Azure Function and an Azure Service Plan that was both created using the following Terraform code:

resource "azurerm_app_service_plan" "asp" {
    name = "asp-${var.environment}"
    resource_group_name      = var.rg_name
    location                 = var.location
    kind = "FunctionApp"
    reserved            = true
    sku {
        tier = "ElasticPremium"
        size = "EP1"
    }
}

resource "azurerm_function_app" "function" {
    name = "function-${var.environment}"
    resource_group_name= var.rg_name
    location= var.location
    app_service_plan_id= azurerm_app_service_plan.asp.id
    storage_connection_string=azurerm_storage_account.storage.primary_connection_string
    os_type = "linux"


    site_config {
      linux_fx_version = "DOCKER|${data.azurerm_container_registry.acr.login_server}/${var.image_name}:latest"
    }

    identity {
      type = "SystemAssigned"
    }


    app_settings              = {
    #Lots of variables, but irrelevant for this issue I assume?
    }

    depends_on = [azurerm_app_service_plan.asp]
    version = "~2"

}

resource "azurerm_storage_account" "storage" {
  name                     = "storage${var.environment}"
  resource_group_name      = var.rg_name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

The function works fine.

The issue is that any change I now try to do in Terraform ends up in the following error during apply:

2020-08-25T06:31:23.256Z [DEBUG] plugin.terraform-provider-azurerm_v2.24.0_x5: {"Code":"Conflict","Message":"Server farm 'asp-staging' cannot be deleted because it has web app(s) function-staging assigned to it.","Target":null,"Details":[{"Message":"Server farm 'asp-staging' cannot be deleted because it has web app(s) function-staging assigned to it."},{"Code":"Conflict"},{"ErrorEntity":{"ExtendedCode":"11003","MessageTemplate":"Server farm '{0}' cannot be deleted because it has web app(s) {1} assigned to it.","Parameters":["asp-staging","function-staging"],"Code":"Conflict","Message":"Server farm 'asp-staging' cannot be deleted because it has web app(s) function-staging assigned to it."}}],"Innererror":null}
...
Error: Error deleting App Service Plan "asp-staging" (Resource Group "my-resource-group"): web.AppServicePlansClient#Delete: Failure sending request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=<nil> <nil>

I have another service plan with an app service, and have had no problems applying while they are running. I have even tried removing all references to the function and its service plan and still get the same error.

I am able to delete the Function and its service plan from the portal and then Terraform applies fine once when it create the function and service plan. As long as those are present when Terraform applies it fails.

This workaround of manually deleting the function and service plan is not feasible in the long run, so I hope someone can help me point out the issue. Is there some error in the way I have created the function or service plan?

provider "azurerm" {
  version = "~> 2.24.0"
...

Edit: As suggested this might be a provider bug, so I have created this issue: https://github.com/terraform-providers/terraform-provider-azurerm/issues/8241

Edit2: On the bug forum they claim it is a configuration error and that I am missing a dependency. I have updated the code with a depends_on, I still have the same error.

Stian
  • 1,221
  • 1
  • 19
  • 26
  • 2
    this sounds more like a bug with the terraform provider, hence you should open an issue in the github repo. as a workaround, you could try to add an explicit depends_on to the function to see if this helps – silent Aug 25 '20 at 07:26
  • 1
    Sometimes I feel like it's a timing issue. Adding a `null_resource` declaring the dependnecy between app service and app plan with a short delay/sleep helps. – pijemcolu Aug 25 '20 at 07:38
  • Thanks pilemcolu, tested adding a 30 sec sleep between them with depends_on, and still same error – Stian Aug 25 '20 at 09:03
  • Is that all code you use? Please share the complete code. – Charles Xu Aug 26 '20 at 01:21
  • Hi Charles, thanks for your interest. This is part of a much larger Terraform project building much else, I tried to create a minimal example and since the code works fine when I run with the service plan and function deleted manually in advanced I assumed they were the issue. Is there anythin in particular that you believe could be relevant? I added the storage account as it is directly referenced. – Stian Aug 26 '20 at 06:15
  • 1
    According to my knowloage, the service plan and the function app will be created sequencely without depends_on. So it's not necessary and not the issue. And you can delete them with the command `terraform destroy` without any problem. So I think the problem is your function app, maybe there is anything else wrong in it so that it cannot be delete and cause the error you got. – Charles Xu Aug 26 '20 at 06:23
  • Any update for the question? And do you find the reason? – Charles Xu Aug 27 '20 at 01:02
  • Hi, No I am still stuck with my bad workaround, on the provider git a person said I was missing a dependency and closed the issue. Unless I misunderstand "dependency" I have explicitly stated this with depends_on, I even tested the null_resource with 30s wait between them. – Stian Aug 27 '20 at 06:14
  • 1
    Do you only assign the function app to the service plan? And if you can delete the function app in another way without any error? Or when you delete the function app you get the error? – Charles Xu Aug 27 '20 at 07:15
  • This is the only app assigned to this service plan. Both are created by this Terraform. They are created successfully with no error if I delete them manually in advance in Azure Portal. But once they are created any run, even with no change to the Terraform code it fails with the error. – Stian Aug 27 '20 at 07:48
  • I mean do you get any error when you delete the function app? – Charles Xu Aug 27 '20 at 08:08
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/220543/discussion-between-stian-and-charles-xu). – Stian Aug 27 '20 at 08:20

2 Answers2

5

I found the issue. On every apply the service plan was reapplied every time:

# azurerm_app_service_plan.asp must be replaced
-/+ resource "azurerm_app_service_plan" "asp" {
      ~ id                           = "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.Web/serverfarms/asp" -> (known after apply)
      - is_xenon                     = false -> null
      ~ kind                         = "elastic" -> "FunctionApp" # forces replacement
        location                     = "norwayeast"
      ~ maximum_elastic_worker_count = 1 -> (known after apply)
      ~ maximum_number_of_workers    = 20 -> (known after apply)
        name                         = "asp"
      - per_site_scaling             = false -> null
        reserved                     = true
        resource_group_name          = "xxx"
      - tags                         = {
          - "Owner"       = "XXX"
          - "Service"     = "XXX"
          - "environment" = "staging"
        } -> null

Even though I created it as kind="FunctionApp" it seems it was changed to "elastic"

I now changed it to kind="elastic" and Terraform has stopped destroying my service plan on every apply :)

Thanks a lot to Charles Xu for lots of help!

Stian
  • 1,221
  • 1
  • 19
  • 26
  • 1
    I am planning to move to FunctionApp kind the solution you provided isn't interesting in my case, did you find another alternative ? – Mehdi Benmoha Jun 01 '21 at 11:43
  • I did not look more into it as this solved our issue. In general I try to avoid Azure App Service where I can as I have had lots of issues with them and can mostly use DataBricks instead :) – Stian Jun 01 '21 at 13:50
  • Yes that's the same thing for me, I am trying to migrate some function apps from app service plan to Serverless mode, but when you set the kind to elastic in terraform that means you are using an AppService plan.. – Mehdi Benmoha Jun 01 '21 at 14:10
  • I ran into this issue while attempting to pick up an existing project & learn terraform at the same time. Once I figured out that I could view the generated Terraform Plan through the Azure pipeline build step I was able to see the changes that necessitated the app service plan to be destroyed and then re-created. In my case it was the name that was changed. – RIanGillis Mar 07 '22 at 16:45
0

The reason (in my case) for the 409 was that app service plans with active app services cannot be deleted. When terraform tries to delete the app service before the apps are migrated to the new plan then the 409 happens. The solution was to keep the existing app service plan during the migration and rerun terraform after the applications have been migrated to a new plan to remove the old plan.

Nicholas_Jones
  • 253
  • 2
  • 8