4

While using Solr (we are currently using 3.5), how do we setup the Masters for a Failover?

Lets say in my Setup I have Two Masters and Two Slaves. The Application commits all the writes to One Active Master, and both the slaves get the updates from this Active Master. There is another repeater which serves the same purpose of the Master.

Now my question is if the Master for some reason comes down, how can I make the Repeater as a Master without any Manual intervention. How can the slaves start getting the updates from the Repeater instead of the broken Master. Is there a recommended way to do this? Are there any other recommended Master/Slave setup's to ensure High availability of the Solr systems?

javanna
  • 59,145
  • 14
  • 144
  • 125
Ravi
  • 843
  • 2
  • 15
  • 31

2 Answers2

4

At this time, your best option is probably to investigate the SolrCloud functionality present in the current Solr 4.0 alpha, which at the time of this writing is due for its final release within a few months. The goal of SolrCloud is to handle data distribution and master election, using the ZooKeeper distributed database to maintain consensus within the cluster about which nodes are serving in while roles.

There are other more traditional ways to set up failover for Solr 3's replicated master-slave architecture, but I personally wouldn't want to make that investment with Solr 4.0 so near to release.

Edit: See Linux-HA, for one such traditional approach. Personally, I would create a purpose-built daemon that reconfigures your cores and load balancer, using ZooKeeper for presence detection and distributed locks.

If outsourcing is an option, you might consider a hosted service such as my own humble Websolr. We provide this kind of distribution and hot failover by default, so our customers don't have to worry as much about the mechanics of how it's implemented.

Nick Zadrozny
  • 7,906
  • 33
  • 38
  • Thanks for the Response Nick. While I would like to explore the option of Solr Cloud is a possibility, I would like to know if there is a way to do this with Solr 3's, because we had already built our framework on this. – Ravi Aug 10 '12 at 14:41
  • 2
    Part of my point is that any solution you set up pursue for automatic master election in Solr 3 is going to be more work than moving to Solr 4. Plus, it may actively delay your move to Solr 4 in the future as a sunk cost. – Nick Zadrozny Aug 13 '12 at 16:01
  • Edited with a few thoughts on automatic master election for Solr < 4. – Nick Zadrozny Aug 13 '12 at 16:13
  • Thanks Nick. Definitely Solr4 is the way we would like to go, but since it is still in the Alpha version, we were thinking of the other solutions. The option of creating a daemon reconfiguring the cores using the Zookeeper, sounds to be a good approach to take. – Ravi Aug 14 '12 at 13:59
3

I agree with Nick. The way replication works in Solr 3.x is not always handy, especially for master fail-over. If you are going to consider Solr 4 you might want to have a look at elasticsearch too, which solves this kind of problems in a really brilliant way!

It uses push replication instead of the pull mechanism used by Solr. That means the document is literally reindexed on all nodes. It might sound strange but that allows to reduce the network load (due to segment merge for example). Furthermore, a node is elected as master and if it crashes one other node will automatically replace it becoming the new master.

javanna
  • 59,145
  • 14
  • 144
  • 125
  • Thanks Javanna for the response! – Ravi Aug 10 '12 at 14:43
  • 2
    A hearty +1 for ElasticSearch, which is designed for distribution from the ground up, with no legacy code to hold it back. In both cases (ES, SolrCloud), multi-master push replication is a huge win. – Nick Zadrozny Aug 13 '12 at 15:57