Achieve soft-commit programmatically

Question

AutoSoftCommit and autoCommit are configured at solrConfig.xml.

Can I configure these settings at my application level using Java?

What I want to know is, Can I configure autoSoftCommit and autoCommit using SolrJ library?

Reason I am asking this question is, application is a springBoot app with API and solr both bundled in the same build. And same build is used as master(indexer) and slaves(replicated through index replication using http call -

http://slave_machine/coreName/replication?masterUrl=http://master_machine/coreName&command=fetchindex)

If I modify solrconfig.xml, it will be changed for master as well as slave.

MatsLindh · Accepted Answer · 2018-05-30T07:47:11.367

You can replicate different files to your clients - it doesn't have to be the same as the configuration file for the master node.

<requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="master">
      [....]
      <str name="confFiles">solrconfig-slave.xml:solrconfig.xml,schema.xml,stopwords.txt</str>
    </lst>
</requestHandler>

This will replicate the file solrconfig-slave.xml on the master as solrconfig.xml on the slave, allowing you to have complete control over the setting for both the master and the slaves.

If you're running in cloud mode (which it seems you aren't, since you're talking about explicit replication), the autoCommit values can be set for the whole collection on the cluster using the Config API and the updateHandler.autoCommit.* values (this is also valid for a single server, but you have to call the config API for each server in that case, since settings aren't distributed across servers automagically).

The other alternative is to use commitWithin instead, which can be sent per request - allowing you to tell Solr that it can wait up to N milliseconds before committing the content to the index. This is usually the preferred way, since it allows you to index from multiple threads and servers without doing explicit commits - so that your commits doesn't conflict with each other.

I will be having the same build for master and slave. If I add replication reuestHandler, this will go for both master as well as slave. Plus I want to know whether autoSoftCommits are possible using solrJ or not — Vibhav Singh Rohilla, May 30 '18 at 07:01
autoSoftCommits isn't per-request-specific. It's a setting for how the server should behave across requests (so you can set it using the Config API if necessary as explained). You can use the same config file for both request and slave, and then just change the value you need changing in the xml file that's being replicated to the slave. To have a per-request setting, `commitWithin` is the proper setting. — MatsLindh, May 30 '18 at 07:45

Abhijit Bashetti · Answer 2 · 2019-07-01T12:47:05.500

You can add the autoCommit and autoSoftCommit through Java code while creating the collection. Please refer the below code for the same. Initialize the value like "collectionName", "solrZKConfigName", "numShards" etc.

String solrZkHostPort = "10.14.40.11:2181,10.14.40.11:2182,10.14.40.11:2183";
List<String> zk_Hosts = Arrays.asList(solrZkHostPort.split(","));
CloudSolrClient cloudSolrClient = new CloudSolrClient.Builder(zk_Hosts, Optional.empty()).build();
Map<String, String> collectionProperties = new HashMap<>();
collectionProperties.put("solr.autoCommit.maxTime", 10000);
collectionProperties.put("solr.autoSoftCommit.maxTime", 15000);
final CollectionAdminRequest.Create adminRequest = CollectionAdminRequest.Create
                .createCollection(collectionName, solrZKConfigName, numShards, numReplicas) .setMaxShardsPerNode(maxShardsPerNode).setProperties(collectionProperties);
CollectionAdminResponse adminResponse = adminRequest.process(cloudSolrClient);

Another alternative option is through Config API

Map<String, String> props= new HashMap<>();
props.put("solr.autoCommit.maxTime", 10000);
props.put("solr.autoSoftCommit.maxTime", 15000);

StringBuilder command = new StringBuilder("{\"set-property\": {");
   for (Map.Entry<String, String> entry: props.entrySet())
   {
      command.append('"').append(entry.getKey()).append('"').append(':');
      command.append(entry.getValue()).append(',');
   }
   command.setLength(command.length()-1); // remove last comma
   command.append("}}");

   GenericSolrRequest rq = new GenericSolrRequest(SolrRequest.METHOD.POST, "/config", null);
   ContentStream content = new ContentStreamBase.StringStream(command.toString());
   rq.setContentStreams(Collections.singleton(content));
   rq.process(solrClient);

Achieve soft-commit programmatically

2 Answers2