3

I have a simple AWS CloudFormation template for creating a hosted zone for example.com along with an SSL/TLS certificate for that domain. It looks something like this:

…

Resources:
  HostedZone:
    Type: AWS::Route53::HostedZone
    Properties: 
      Name: example.com

  Certificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: example.com
      ValidationMethod: DNS
      DomainValidationOptions:
        - DomainName: example.com
          HostedZoneId: !Ref HostedZone

I had understood that if I don't specify a DeletionPolicy, by default CloudFormation deletes resources in a stack when a stack is deleted. There are a few exceptions such as a non-empty S3 bucket, but that makes sense because in that case someone went in and added things to the bucket independent of the CloudFormation stack I presume.

However when I delete the stack for the above template, I get an error:

The specified hosted zone contains non-required resource record sets and so cannot be deleted. (Service: Route53, Status Code: 400, Request ID: …)

Of course the error message is understandable: an extra record was added to the hosted zone in order to verify the certificate, and the hosted zone can't be deleted unless you first delete that extra record. The same thing would happen if I created all this manually.

Still that behavior isn't really consistent with the documentation of DeletionPolicy (which says resources should be deleted automatically), and there is nothing here that wasn't created by the stack. It would seem that, despite all the assurances that "CloudFormation is declarative" and "CloudFormation figures out the order in which to do things", it comes with a lot of caveats at least when it comes to deletion.

In this case CloudFormation should figure out that it added the record after creating the hosted zone, so it should remove that record before deleting the hosted zone. But it can't figure that out.

If CloudFormation can figure out how to delete the hosted zone (which it can), and if CloudFormation can figure out how to remove the load balancer (which it can), and if CloudFormation can figure out how to remove the record from the hosted zone pointing to the load balancer before deleting the hosted zone (which it can), why can't CloudFormation figure out how to delete the certificate verification record from the hosted zone before deleting the hosted zone?

Is there some workaround to add to my stack definition to allow the hosted zone to be deleted when the stack is deleted? If I have to go in and manually remove a record every time, it defeats the purpose of CloudFormation as far as deletion is concerned.

Garret Wilson
  • 18,219
  • 30
  • 144
  • 272

1 Answers1

0

You have to create a custom resource in a form of a lambda function. The function would execute as part of a delete procedure of your stack and clean up your hosted zone.

Your custom resource would have to use AWS SDK, e.g. boto3, to delete records from your zone. Specifically you can use change_resource_record_sets which can delete the records.

Marcin
  • 215,873
  • 14
  • 235
  • 294
  • That's a great tip with a link, and it will be useful to me to for doing more research. But could you explain further how this is done and give a rough example, so we can turn this into a full answer? Thanks. – Garret Wilson Feb 28 '23 at 13:11
  • @GarretWilson I added a bit more info to the answer, but I do not have an example. You can find it in AWS docs [here](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/walkthrough-custom-resources-lambda-lookup-amiids.html). – Marcin Mar 01 '23 at 01:41
  • I still don't understand why CloudFormation can figure out how to remove the record pointing to the load balancer but can't figure out how to remove the record used for certificate verification. – Garret Wilson Mar 08 '23 at 22:54
  • @GarretWilson This is the question for AWS. I did not design nor maintain CloudFormation implementation. – Marcin Mar 08 '23 at 22:55
  • 1
    Marcin fair enough. I guess it was a rhetorical question. Arg, these little pointless glitches are irritating. They should hire me to fix it. – Garret Wilson Mar 08 '23 at 22:58
  • @GarretWilson That's why many people choose to use terraform instead. – Marcin Mar 08 '23 at 23:07
  • Yeah, I'm seeing that. I thought I'd learn both so I would know how to compare them beyond these little one-page comparisons online which I'm sure don't reveal the real-life glitches one encounters. But for some reason I had also thought I had to pay to use Terraform on AWS. Maybe I was confused and was thinking about Kubernetes vs ECS. Anyway if Terraform is free, too, I'll probably eventually switch, but at least I'll know which is best from experience. – Garret Wilson Mar 08 '23 at 23:13
  • @GarretWilson TF is cloud agnostic. So once you learn it once, you can work with any cloud. CloudFormation is AWS specific only. – Marcin Mar 08 '23 at 23:14
  • Oh, I remember one reason I went ahead and started with CloudFormation. It is my understanding that CloudFormation will allow any developer to upload a new CloudFormation template and perform needed changes to bring the cloud state in line with the new template, but I had read that Terraform requires the current state to be kept on one machine or something. But we're getting off topic here, and I don't want to abuse Stack Overflow. Thanks for suggestions. I'll keep Terraform in mind. – Garret Wilson Mar 08 '23 at 23:15