8

I have the following production setup for my Node JS application: Application production architecture

I am now going to integrate Elasticsearch in this setup. My question is regarding the best practices for deploying Elasticsearch in a production environment. All my instances are virtual machines, and I understand that Elasticsearch uses a lot of memory.

Should I therefore set up Elasticsearch on its own server (server 3), set it up on both server 1 and server 2 as a cluster (much like the Mongo DB replica set) or install it as a separate instance on each server.

What would be the benefits of the chosen method?

Many thanks!

Axelfran
  • 964
  • 1
  • 7
  • 17
  • I'm have a similar requirements and your diagram is very useful! Could you share what are you using for each component? as load balancer (nginx, haproxy)? and how did you ended-up solving the elasticsearch deployment? – Adrian Jul 25 '15 at 16:32
  • 1
    Hi Adrian. I'm using HAproxy as a load balancer and Node JS with Express to run the application. Furthermore I use PM2 to start the application etc. ES is running as a node on each server and is set up as a cluster, just like the Mongo DB replica set. This has been working very well for me so far. I hope I answered your questions, but feel free to comment if you need furher explanation:) – Axelfran Jul 27 '15 at 06:54
  • Great! One last thing, by any chance do you have a gist or example config to setup replica sets in MongoDB and Elasticsearch? – Adrian Jul 27 '15 at 21:37
  • I'm sorry, but unfortunately I don't. – Axelfran Jul 28 '15 at 12:44

1 Answers1

1

Option 2.

Briefly.. I would definitely set this up on both servers - giving you two nodes. Given the options you have stated, this will provide the maximum distribution, load balancing, performance and fault tolerance.

Ensure that you manually configure your memory allocation carefully, assigning 50% of the total allocated to heap on each node, and leave the rest to Lucene for indexing.

Gillespie
  • 2,228
  • 2
  • 18
  • 25
  • Are you now talking about the total memory allocated to the elasticsearch process, or the total memory of the virtual machine? I know where I can adjust the memory allocated to the Elasticsearch heap, but where can I adjust how much is reserved for Lucene? – Axelfran Aug 27 '14 at 11:12
  • Just the elasticsearch process. Don't worry about reserving for lucene, that is handled automatically from the remainder of available memory. – Gillespie Aug 27 '14 at 11:59