SOLR 4.4 indexing incrementally

Question

When I kick off a full-import via:

curl -X POST http://_master_host_:_port_/solr/dataimport-xml?command=full-import

It is causing our index to update incrementally. So say our inventory count is 900,000, it will suddenly be wiped out and be 400 -> 1000 -> ... -> 900000 (updating incrementally instead of waiting for import to finish then swap out).

I have no idea why it is clearing out our index as previous versions would wait until the import completed before swapping. All our solrconfig settings are the same so I am not sure what is causing this. Any ideas or something I am missing?

Answer 1

Found the culprit:

 229   <!-- The default high-performance update handler -->
 230   <updateHandler class="solr.DirectUpdateHandler2">
 231     <!-- AutoCommit
 232 
 233          Perform a hard commit automatically under certain conditions.
 234          Instead of enabling autoCommit, consider using "commitWithin"
 235          when adding documents.
 236 
 237          http://wiki.apache.org/solr/UpdateXmlMessages
 238 
 239          maxDocs - Maximum number of documents to add since the last
 240                    commit before automatically triggering a new commit.
 241 
 242          maxTime - Maximum amount of time in ms that is allowed to pass
 243                    since a document was added before automaticly
 244                    triggering a new commit.
 245          openSearcher - if false, the commit causes recent index changes
 246          to be flushed to stable storage, but does not cause a new
 247          searcher to be opened to make those changes visible.
 248       -->
 249      <autoCommit>
 250        <maxTime>15000</maxTime>
 251        <openSearcher>false</openSearcher>
 252      </autoCommit>
 253 
 254     <!-- softAutoCommit is like autoCommit except it causes a
 255          'soft' commit which only ensures that changes are visible
 256          but does not ensure that data is synced to disk.  This is
 257          faster and more near-realtime friendly than a hard commit.
 258       -->
 259 
 260     <autoSoftCommit>
 261       <maxTime>1000</maxTime>
 262     </autoSoftCommit>

commented out the autocommit and autosoftcommit sections and the index doesn't update incrementally any more. Appears this is a new argument added in solr 4. Hopefully this helps someone else in the future.

EDIT: This is located in the SolrConfig/conf/solrconfig.xml file.