Terratest - Use mock AWS services

Question

I am trying to do AWS resource provisioning through terraform and planning to have a CICD pipeline with terratest unit test cases for the terraform code. My problem is I have CloudFront in my infrastructure and it takes about 20 mins for the creation and around the same time for removal. I don't want the CI build to take around 45 mins just for running unit test cases.

I came across localstack to mock the AWS environment but didn't find ways to point terratest to localstack resources. This is what I tried

Created a localstack docker container
Configured the terraform to point to localstack - added endpoint url in the provider section.
Applied the terraform config and verified bucket created in the localstack.
Wrote a simple terratest test case to assert if the bucket exists.

Terraform code is as follows,

terraform {
  backend "s3" {
    bucket  = "<bucket-name>"
    key     = "state/terraform.tfstate"
    region  = "us-east-1"
    profile = "default"
  }
}

provider "aws" {
  region = "us-east-1"
  s3_force_path_style = true
  skip_metadata_api_check = true
  endpoints {
    s3             = "http://localhost:4572"
  }
}

resource "aws_s3_bucket" "test_bucket" {

  bucket = "test-bucket"
  acl    = "public-read-write"
  cors_rule {
    allowed_headers = ["*"]
    allowed_methods = ["GET", "HEAD", "PUT"]
    allowed_origins = ["*"]
    expose_headers  = ["ETag"]
  }
  region = "us-east-1"
}

output "name" {
  value = "${aws_s3_bucket.test_bucket.bucket}"
}

output "region" {
  value = "${aws_s3_bucket.test_bucket.region}"
}

When the terratest test case as given below was executed a bucket was created in the localstack. But I couldn't find any api or config that would point the terratest AWS module to localstack endpoints. AssertS3BucketExists by default check the AWS environment for the bucket and the assertion fails.

Terratest code is as follows.

package aws

import (
"fmt"
"testing"

"github.com/gruntwork-io/terratest/modules/aws"
"github.com/gruntwork-io/terratest/modules/terraform"
)

func TestWebServer(t *testing.T) {
    terraformOptions := &terraform.Options{
    // The path to where your Terraform code is located
    TerraformDir: ".",
}


terraform.InitAndApply(t, terraformOptions)

name := terraform.Output(t, terraformOptions, "name")
region := terraform.Output(t, terraformOptions, "region")
aws.AssertS3BucketExists(t, region, name)

Any help here would be much appreciated.

What have you tried so far? You should edit your question to include your code and any errors output. — ydaetskcoR, Feb 24 '20 at 14:46

score 2 · Answer 1 · answered Jul 24 '20 at 15:45

To accomplish mocking AWS with terratest without modifying the library, you could use something like moto's standalone server mode. Since there's no (obvious) way to change endpoints in terratest, it may be necessary to modify local DNS resolution to point endpoints at the local moto server.

With terratest, there's no way to inject a mock AWS into the library itself since the interface used for connecting to AWS is not exposed.

score 1 · Answer 2 · answered Sep 24 '20 at 07:12

There's a working PR that will get Terratest to verify resources in Localstack.

You can update your go.mod file with the replace directive to test it.

module github.com/GITHUB_USERNAME/REPO_NAME

go 1.15

require (
    github.com/gruntwork-io/terratest v0.30.0
    github.com/stretchr/testify v1.6.1
)

replace github.com/gruntwork-io/terratest v0.30.0 => github.com/ffernandezcast/terratest v0.28.6-0.20200915124510-25813206bebc

Then update your Terratest test with the following vars to configure the custom endpoints for the aws package.

    var LocalEndpoints = map[string]string{
        "apigateway":     "http://localhost:4566",
        "cloudformation": "http://localhost:4566",
        "cloudwatch":     "http://localhost:4566",
        "dynamodb":       "http://localhost:4566",
        "es":             "http://localhost:4566",
        "firehose":       "http://localhost:4566",
        "iam":            "http://localhost:4566",
        "kinesis":        "http://localhost:4566",
        "lambda":         "http://localhost:4566",
        "route53":        "http://localhost:4566",
        "redshift":       "http://localhost:4566",
        "s3":             "http://localhost:4566",
        "secretsmanager": "http://localhost:4566",
        "ses":            "http://localhost:4566",
        "sns":            "http://localhost:4566",
        "sqs":            "http://localhost:4566",
        "ssm":            "http://localhost:4566",
        "stepfunctions":  "http://localhost:4566",
        "sts":            "http://localhost:4566",
    }
    aws.SetAwsEndpointsOverrides(LocalEndpoints)

Then, run go test and Terratest will now verify resources in LocalStack. I detailed more of this here https://jq1.io/posts/go_mod_terratest_localstack/.

Hopefully, that PR will be merged into Terratest master branch soon.

~jq1

Terratest - Use mock AWS services

2 Answers2