Difference Between name_regex and filter by name in terraform

Question

I am new to terraform and going through the AMI data source. as per this documentation for data source. I get the below code for getting the AMI ID

data "aws_ami" "test_ami" {
  executable_users = ["self"]
  most_recent      = true
  name_regex       = "^myami-\\d{3}"
  owners           = ["self"]

  filter {
    name   = "name"
    values = ["myami-*"]
  }

  filter {
    name   = "root-device-type"
    values = ["ebs"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

I am confused between the attribute name_regex and filter by name. name_regex will also filter the list of AMIs and filter by name also. is there any difference between them?

I guess it's explained pretty good in the documentation you linked: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami#name_regex. — Marko E, Apr 12 '23 at 06:57

score 1 · Answer 1 · answered Apr 12 '23 at 23:33

Most of the data sources in the hashicorp/aws provider are directly wrapping read-type operations from the underlying AWS APIs.

In the case of aws_ami, the underlying API operation is ec2:DescribeImages.

As mentioned in the documentation for aws_ami, the name_regex argument is special in that it is implemented by the provider itself, rather than by the remote API:

name_regex - (Optional) Regex string to apply to the AMI list returned by AWS. This allows more advanced filtering not supported from the AWS API. This filtering is done locally on what AWS returns, and could have a performance impact if the result is large. Combine this with other options to narrow down the list AWS returns.

The important difference is the one described above: if you use name_regex then the provider will fetch all of the images that match all of the other arguments you set, and then from that result it will remove any that have names which don't match your regex pattern.

This allows you to specify a subset of images to return using more specific filters than the API supports. In your case, you specified that the pattern must begin with myami- and then any three ASCII digits.

The "name" filter built in to the API only supports a simpler wildcard language which supports * to represent "any number of any character" or ? to represent any single character, as described in List and filter using the CLI and API. The rule myami-* therefore only requires that the name start with myami-, and cannot express the additional constraint that it must be followed by three ASCII digits.

Specifying both of these together is a good compromise to deal with the problem described in the documentation above: the filtering could have a performance impact if the resultset is large. If you specify them both together then:

The provider will send an ec2:DescribeImages call to the API to request only images whose names start with myami-. The remote server will perform that filter itself, and so the API response will only include images with that prefix.
Once the provider receives that response it will then check each of the results to see if the name also has the required three ASCII digits after the prefix, and throw away any of the results that don't.

If you were to use only name_regex then the AWS provider would first need to retrieve all matching images from the API regardless of their names, which could be a significantly larger resultset that would cause performance problems.

Difference Between name_regex and filter by name in terraform

1 Answers1