How to Cache Terraform init

Question

We have terraform templates that provision infrastructure and also kick off shell scripts that in turn invoke ansible playbooks to deploy an application.

Both terraform templates are packaged and get invoked from a UI application in an automated way - the intention is provide self-service to developers where they get the entire stack - infra and application deployed with single click.

This automation gets invoked every time a developer raises a request and in the code, we are running terraform init every time to ensure we don't run into any non-availability of plugins down the line. With every run of terraform init, it's loading the plugin binaries over and over into memory and we are running out of memory. Is there a possible solution to prevent terraform init from loading plugin binaries into memory over and over and reuse existing binaries if they are already in memory?

score 1 · Answer 1 · answered Jun 12 '23 at 16:53

I assume that when you say "memory" in your question you mean files in the filesystem, rather than data in RAM. terraform init does not load the provider plugins into RAM, but it does extract them into a cache directory under the .terraform directory.

If your intention is to allow your system to keep working when registry.terraform.io is down or when a plugin has been deleted then you can configure Terraform to install plugins from a local directory instead of fetching them over the network every time.

First choose a directory where you'll place your copies of the provider plugins. For example here I'm going to use /opt/terraform/plugins, but you can use any directory path that Terraform will be able to access.

In the home directory of the system user that Terraform will run under (that is: the directory that will be in the HOME environment variable when Terraform runs) create a file named .terraformrc and write the following inside it:

provider_installation {
  filesystem_mirror {
    path = "/opt/terraform/plugins"
  }
}

This file is the Terraform CLI configuration and the provider_installation block describes the provider installation methods. The default provider installation configuration is to install providers directly from their origin registries, but the custom configuration I showed above tells Terraform to consider only what's placed in the specified directory.

If you run terraform init with that configuration in place then you should see it fail to install, because there aren't yet any plugins in that directory.

You'll need to populate the directory with a directory structure matching one of the filesystem mirror layouts. Since your goal is to save disk space, you should use the "unpacked" layout since that gives Terraform the opportunity to use the plugins directly from the mirror directory, rather than having to extract them first.

To do this you'll need to find the distribution packages for the providers you intend to use and download them onto your system. The locations of these source packages vary depending on whether they are official HashiCorp providers or third-party providers written by partners or the community.

For official HashiCorp providers you can find them on releases.hashicorp.com. For example, the hashicorp/aws provider packages are under terraform-provider-aws on that site. If you wanted to make hashicorp/aws v5.2.0 available in your mirror then you could follow the following steps:

Download the .zip package for the platform you are using. For example, if you are running a Linux distribution on an x86_64/amd64 processor then you should download the linux_amd64 package.
Under your mirror directory create the necessary data structure. For the example path I showed above and a linux_amd64 package that would be /opt/terraform/plugins/registry.terraform.io/hashicorp/aws/5.2.0/linux_amd64.
Extract the plugin zip file into the directory created in the previous step. After you've done that the linux_amd64 should contain a single executable file directly inside that directory (not in a subdirectory).

For third-party providers you can find their GitHub repository through Terraform Registry. Third-party providers use GitHub Releases to publish their .zip files, so you can download the appropriate package from there and then install it into a similar directory structure as I showed for the hashicorp/aws provider above.

(Note that hashicorp/aws is short for registry.terraform.io/hashicorp/aws, because a provider address without a hostname prefix is always assumed to belong to registry.terraform.io. The directory for that hostname is required in the filesystem mirror directory layout.)

After you've filled your local mirror with all of the providers you intend to use, you should be able to run terraform init and see it install all of the providers from your mirror without accessing Terraform Registry. If you inspect the size of the .terraform directory then you should see it considerably smaller than with direct installation, because Terraform will try to create symlinks to the directories inside the filesystem mirror rather than creating full copies of the plugin package directories in there.

How to Cache Terraform init

1 Answers1