0

I'm new to ElasticSearch but I'm already using ElasticSearch on my production site.

I installed ElasticSearch in one EC2 instance. After using ElasticSearch for several month, now currently the CPU of the instance already hit 100% at peak time, and the queries became so slow. But at non-peak time, the CPU usage is only 10%.

I know I can use autoscaling to spin new instance on a condition where an instance is on a certain high CPU usage. I have used this in my webserver with the help of AWS autoscaling and Elastic Load Balancer. But can I do this using ElasticSearch? Where do I start?

Petra Barus
  • 121
  • 1
  • 5

1 Answers1

3

This has been answered pretty well on StackOverflow

The full answer is worth reading but here are the key points:

  1. Moving and re-allocating shards is resource intensive. So having a server get added or removed on the fly can put a load on the system.
  2. You should already have 2 nodes for ElasticSearch already. It performs better that way and keeps the data safer.
  3. You can't adjust the number of shards upwards and downwards when removing or adding servers. What this means is that when you move down from 2 to 1 servers, suddenly you're going to have a lot of unallocated shards.

That said, I actually have my ES servers behind an Auto-Scale. But I have it set to always keep the same number of servers; it's only there to ensure that there are two servers on hand at all times, not to scale up or down.

My recommendation? Set up two t2.small or t2.micro servers as your servers rather than just one server. Have both running all the time. Then, when you know traffic is going to be high for a given period of time (say a week or so), stop one server, bump up its instance type (you can do this without changing any other aspect of the server) and then start up the server again. If you do this in sequence and make sure that the cluster returns to a green state, you can then upsize the second server and your users should not have any interruption of service.

TLDR: Auto-Scaling based on load is not recommend for ElasticSearch servers. Move to a pair of tiny instances (behind an load balancer) and if you can anticipate load, upsize each server individually as needed.

Community
  • 1
Jordan Reiter
  • 1,290
  • 4
  • 20
  • 40