5

SolrCloud, thanks to ZooKeeper integration, has some nice utilities for managing and reloading core/collection configuration.

However, this only fully covers the case of trivial updates - but there are also nontrivial updates. Nontrivial in this case means resulting in some changes that make an updated node and/or its cores incompatible with some previous state.

In particular, these subcases come to mind:

  1. A code update necessitating a restart of an underlying Solr instance.
  2. A schema change that requires a full rebuild of a core.

My question is: how can SolrCloud and the associated Zookeeper services be used to make such updates easier, more reliable, and/or ensuring higher availability?

Note: I was hoping for some APIs/functionality that "understands" such updates. So far the most notable thing I've found is collection aliasing in CoreAdmin, which would allow for a smoother transition between the "old" and "new" versions - a little disappointing given the aforementioned hopes.

Community
  • 1
  • 1
mikołak
  • 9,605
  • 1
  • 48
  • 70

1 Answers1

-1

I am not sure what you mean by 

A code update necessitating a restart of an underlying Solr instance.

You mean that the Solr code changed? (in eg. a newer version) Or that the application accessing the Solr instance changed? (in eg. your codebase)

In the first scenario, just bringing up a new instance, and adding it to the ZooKeeper, even if the version differs, should be the end of it.

In the second case, it really doesn't matter what happens to the application accessing the data, right?

Then you mention what I believe is the most "common" scenario

A schema change that requires a full rebuild of a core.

If you are changing the schema, and this implies you are changing some of your indexes, your fields and/or meta-data, you can't really expect Solr to be agnostic of this change and keep running, and returning results, as their hashes no longer correspond to the same structures.

I think the best approach here would be to try to identify the depth of the changes, and either reload into a new index the updated structure, and then do the required code changes to your application, so it will query these new structures, or if a downtime window is allowed, just delete and rebuild the whole thing (this sort of attacks your "ensuring higher availability" requirement though)

I think it would be same as a hot update to a DB table in SQL and having two versions of the application using both, the old and new structures, it can be done with sort of an extreme care, and you will be better off by splitting them apart if you can...

Not sure if this helps, cheers,

Mike.

migueldiab
  • 82
  • 1
  • 4
  • *Re code update:* 1. I meant a change in custom service-specific code within the Solr instance, but a version update is a valid subcase. 2. "Should"? Do you know this for a **fact**, or is that your conjecture? *Re rebuild case:* I appreciate your willingness help, but you do realize **I included a "manual" solution in my question already**, in the final paragraph? This question is not about general solutions to these problems, but **whether SolrCloud/Zookeeper has additional facilities that help in solving them, and if so, how.** I'm sorry, but as it stands, I have to downvote your answer. – mikołak Dec 19 '13 at 20:21
  • Sorry to hear I wasn't helpful... Still don't think it called for a downvote, as it may stand useful unless someone finds a specific way to do what you ask :( Your question, as it stands, asks for Solr/ZK to "understand" a non-trivial update to the core index, this I'm afraid it can't do. About a new version of Solr running on the same ZK, yes I know that for a **fact**. Your "manual" solution is as-good-as-it-gets unless you are willing to code something to prevent this (adding a codeVersion field and querying for that or something like that) – migueldiab Jan 06 '14 at 13:42
  • No, it didn't *call* for a downvote, because those are not a punishment or a repression; and as an asker, I believe I do have the right to *some* opinion as to whether the answer is useful or not (especially since there was an ongoing bounty at the time) ;). Finally, if your answer is derived from facts, please edit it to explain/reference them. – mikołak Jan 16 '14 at 15:16