3

My scala app needs to interact with solr cloud via zookeeper.I need to upload the solr configuration files(conf folder) to the zookeeper. It is done using the following shell command :

 cloud-scripts/zkcli.sh -cmd upconfig -zkhost zkHostAddress:2181 -d solr/collection/conf/ -n collection

I need this config upload to be done from my scala app itself and not use this shell command . How can I achieve this ?

Harsh Gupta
  • 339
  • 4
  • 20
  • SolrCloud clients already interact themselves transparently with zookeeper. Why don't you use a java client embedded in your scala app, like Solrj ? – Grooveek Dec 06 '13 at 13:24
  • I would really like to use solrj but i don't see any function call in the api which uploads the configurations. – Harsh Gupta Dec 06 '13 at 13:25
  • So, certainly misunderstood what you wanted to do. You want to update which kind of configurations via Zookeeper ? Cores adresses ? What does the zkcli.sh do ? Maybe you should have a look to zookeeper clients to embed in your app like [Netflix curator](https://github.com/Netflix/curator)? – Grooveek Dec 06 '13 at 13:49
  • So let me clarify it better. As far as I know the zkcli.sh takes all the parameters and uploads the solr conf folder contents(that also includes schema.xml) by **sending a request to zookeeper host(which is provided as parameter)** and the remote zookeeper inturn takes care of creating the same core configuration on solrcloud.Since my scala app is creating the conf folder and its content locally how do I upload the configuration . – Harsh Gupta Dec 06 '13 at 13:56
  • I had to reverse engineer the apache solr libraries to find out how it should be done: #1 Requires "org.apache.solr" % "solr-core" % "4.3.1" in sbt #2 The zkcli.sh passes the command-line arguments to ZkCLI.java. #3 I extracted the code snippet & used directly in scala app `val zkServerAdd : String = zkHost var zkClient :SolrZkClient = null; try { zkClient = new SolrZkClient(zkServerAdd,30000,30000,new OnReconnect() { override def command() {} }) } ZkController.uploadConfigDir(zkClient, new File(confdir), core)` – Harsh Gupta Dec 11 '13 at 15:08
  • @Harsh Gupta. Would you be so kind to make an answer from your own comment? It seems as if you managed to do this trick, but unfortunatelly I can't figure out from your comment what exactly you did. – Jacobian Aug 29 '15 at 13:23

1 Answers1

0

Netflix wrote a good Zookeeper library called Curator (now an Apache project) that works quite well from Scala e.g the following code establishes a connection to ZooKeeper

private def getConnection(connection : String): Option[CuratorFramework] = {
   val retryPolicy = new ExponentialBackoffRetry(100,3)
    val newConn = CuratorFrameworkFactory.builder().
      connectString(connection).
      retryPolicy(retryPolicy).connectionTimeoutMs(1000).
      build
    try {
    newConn.start()
    newConn.checkExists.forPath(SOME_PATH)

    Some(newConn)
    } catch {
      case e:Exception => {
        error(e)
        None
      }
    }
}
Arnon Rotem-Gal-Oz
  • 25,469
  • 3
  • 45
  • 68