Terraform: ElastiCache Redis cluster with specified availability zones?

Question

I use this Terraform example to create an ElastiCache Redis cluster (clustered mode enabled): https://www.terraform.io/docs/providers/aws/r/elasticache_replication_group.html#redis-cluster-mode-enabled

resource "aws_elasticache_replication_group" "example" {
  replication_group_id = "example-group"
  engine_version = "5.0.5"
  node_type = "cache.r5.large"
  port = 6379
  automatic_failover_enabled = true

  cluster_mode {
    replicas_per_node_group = 1
    num_node_groups = 6
  }
}

But how do I specify availability nodes for clusters and replicas? It's possible via AWS console. I was hoping to add availability_zones = ["us-east-1a", "us-east-1c"] to specify that all master nodes must be in us-east-1a and all replicas in us-east-1c, but got Error creating Elasticache Replication Group: InvalidParameterCombination: PreferredCacheClusterAZs can only be specified for one node group.

I use Terraform v0.12.17 and aws provider v2.34.0.

score 4 · Answer 1 · answered Dec 05 '19 at 15:22

It seems to me that this is impossible with the current Terraform aws provider (https://github.com/terraform-providers/terraform-provider-aws/issues/5104)

However, I found a useful workaround (it does not allow setting arbitrary AZ for each particular node as AWS console does, but it covers the most common use case): if you specify VPC subnets for your replication group via subnet_group_name key, cache instances will be created in the AZs of those subnets (order of subnets in a subnet group matters).

Example Terraform config:

resource "aws_elasticache_subnet_group" "redis_subnet_group" {
  name       = "example-subnet-group"
  subnet_ids = ["subnet-123", "subnet-456"]
}        

resource "aws_elasticache_replication_group" "redis_replication_group" {
  replication_group_id          = "example-replication-group"
  engine_version                = "5.0.5"
  node_type                     = "cache.r5.large"
  port                          = 6379
  automatic_failover_enabled    = true
  subnet_group_name             = aws_elasticache_subnet_group.redis_subnet_group.name

  cluster_mode {
    replicas_per_node_group = 1
    num_node_groups         = 6
  }
}

Result: I got a 6-shard cluster with all primary nodes in AZ of a subnet-123 and all replicas in AZ of a subnet-456. I have not tested it with more than one replica per node group.

score 1 · Answer 2 · answered Jul 20 '20 at 22:46

Adding description to the problem as mentioned here.

The reason this is occurring is because the availability_zones argument is not compatible with Redis Cluster Mode Enabled replication groups where there is more than 1 shard.

In the Elasticache SDK, this is the full documentation for the parameter that availability_zones sets:

// A list of EC2 Availability Zones in which the replication group's clusters
// are created. The order of the Availability Zones in the list is the order
// in which clusters are allocated. The primary cluster is created in the first
// AZ in the list.
//
// This parameter is not used if there is more than one node group (shard).
// You should use NodeGroupConfiguration instead.
//
// If you are creating your replication group in an Amazon VPC (recommended),
// you can only locate clusters in Availability Zones associated with the subnets
// in the selected subnet group.
//
// The number of Availability Zones listed must equal the value of NumCacheClusters.
//
// Default: system chosen Availability Zones.
PreferredCacheClusterAZs []*string `locationNameList:"AvailabilityZone" type:"list"`

So to explicitly configure availability_zones for the same availability zone multiple times for Redis Cluster Mode Disabled or single shard replication groups, the attribute does need to be migrated similar to how preferred_availability_zones was for the aws_elasticache_cluster resource.

For Redis Cluster Mode Enabled replication groups (e.g. when using cluster_mode in Terraform), we currently do not have the ability to set availability zones via the NodeGroupConfiguration parameter, which would likely require changing the cluster_mode argument.

Terraform: ElastiCache Redis cluster with specified availability zones?

2 Answers2