7

We're planning to use Redis (cluster mode disabled) in AWS ElastiCache.

Multi-AZ with automatic failover feature helps keeping minimal downtime in case the primary node goes down.

However, nothing is said about scheduled maintenance. I understand that different availability zones are in different places in region, but will failover work in case of maintenance? Will all nodes go down simultaneously or one by one with automatic failover? Will be data left intact?

Docs do not cover these questions. They just say I should pick a maintenance interval, they don't say what will happen with my data.

efpies
  • 3,625
  • 6
  • 34
  • 45
  • It's a good question indeed. I recommend simply avoiding maintenance altogether by manually replacing your nodes before maintenance: Go to AWS Console -> ElastiCache -> Redis -> Click cluster name -> Add Node (x2) -> Wait for nodes to be created and status to change to "in-sync" -> Delete old **replica** which is scheduled for maintenance -> Click Actions -> Failover primary -> Delete old **primary** which is scheduled for maintenance -> Done! No more sketchy maintenance that you have no control over =) – Elad Nava Jul 23 '18 at 08:53
  • 3
    One note: ElastiCache Redis primary node failover is quite slow and your cluster will be unreachable for a varying period of time (around 45 seconds). Make sure your apps can handle and recover from this, either rejecting or queueing up requests in the meantime. – Elad Nava Jul 23 '18 at 09:16

1 Answers1

1

The Elasticache FAQs shed a bit more light on the scheduled maintenance.

From the Elasticache FAQs:

Q: What is a maintenance window? Will my nodes be available during software maintenance?

You can think of the Amazon ElastiCache maintenance window as an opportunity to control when software patching occurs, in the event either are requested or required. If a "maintenance" event is scheduled for a given week, it will be initiated and completed at some point during the 60 minute maintenance window you identify.

Your nodes could incur some downtime during your maintenance window if software patching is scheduled. Please refer to Engine Version Management for more details. Patching can be user requested - for example cache software upgrade, or determined as required (if we identify any security vulnerabilities in the system or caching software). Software patching occurs infrequently (typically once every few months) and should seldom require more than a fraction of your maintenance window...

The engine version documentation does mention that for engine version upgrades of multi-AZ enabled clusters, the primary node will continue to be available to service requests during the upgrade process, except for a few minutes when a failover is initiated.

Also from FAQs:

Q: What is Multi-AZ for ElastiCache for Redis Cluster?

Each shard of an ElastiCache for Redis cluster consists of a primary and up to five read replicas. Redis asynchronously replicates the data from the primary to the read replicas. During certain types of planned maintenance, or in the unlikely event of ElastiCache node failure or Availability Zone failure, Amazon ElastiCache will automatically detect the failure of a primary, select a read-replica, and promote it to become the new primary.

user818510
  • 3,414
  • 26
  • 17
  • 2
    Yes, but from this (“Your nodes could incur some downtime during your maintenance window if software patching is scheduled”) it's not clear if all nodes will go down at the same time or one by one, since I provide a maintenance window for a whole cluster. – efpies Jul 27 '17 at 15:08