Set last_modified field when not defined in document in Solr

Question

I'm using Solr 4.6 example's SimplePostTool to import documents from the filesystem to Solr. All it's ok, but the field last_modified is filled only when the original document has metadata for it. If the field is not present Solr extractor leaves the field blank.

I tried to modify SimplePostTool to set this field using the file system modification date, but then I get this error when I try to import files that already have last_modified field from the metadata:

430584 [qtp1214238505-16] ERROR org.apache.solr.core.SolrCore  – 
  org.apache.solr.common.SolrException: ERROR: 
  [doc=4861976] multiple values encountered for non multiValued field 
  last_modified: [2013-12-22T14:03:10.000Z, 2013-07-02T11:29:20.000Z]

I'm thinking about using a custom field for file system date, but in my case, metadata date if preferable when is available. Is there any way to merge them at import time?

Thanks!

score 0 · Answer 1 · answered Dec 23 '13 at 15:29

0

You can set a default value in your schema. Something like this should work:

<field name="my_date" type="date" indexed="true" stored="true" multiValued="false" default="NOW" />

Field Type Definition:

<fieldType name="date"     class="solr.TrieDateField" sortMissingLast="true" omitNorms="true"/>

answered Dec 23 '13 at 15:29

mjalajel

2,171
21
27

That looks fine for indexing new created/received documents, but in our case we need to create new full index from a document database of 8 years old. Setting the current date by default for oldest documents is not a good solution in our scenario. – Javier Alvarez Dec 23 '13 at 17:36

score 0 · Answer 2 · answered Dec 23 '13 at 17:00

0

while creating a document the solr takes all input as text and then validates according to the given data type , Hence any form of valid date format accepted ,would work fine with the solr . For current time Any default value

regards

Rajat

answered Dec 23 '13 at 17:00

Rajat Rastogi

17
5

score 0 · Accepted Answer · answered Dec 24 '13 at 11:00

I finally solved the issue creating a custom Update Request Processor, as explained here: http://wiki.apache.org/solr/UpdateRequestProcessor

My processor is as follows:

package com.mycompany.solr;

import java.io.IOException;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.response.SolrQueryResponse;
import org.apache.solr.update.AddUpdateCommand;
import org.apache.solr.update.processor.UpdateRequestProcessor;
import org.apache.solr.update.processor.UpdateRequestProcessorFactory;

public class LastModifiedMergeProcessorFactory 
   extends UpdateRequestProcessorFactory {

  @Override
  public UpdateRequestProcessor getInstance(SolrQueryRequest req, 
       SolrQueryResponse rsp, UpdateRequestProcessor next) {
    return new LastModifiedMergeProcessor(next);
  }
} 

class LastModifiedMergeProcessor extends UpdateRequestProcessor {

  public LastModifiedMergeProcessor(UpdateRequestProcessor next) {
    super(next);
  }

  @Override
  public void processAdd(AddUpdateCommand cmd) throws IOException {
    SolrInputDocument doc = cmd.getSolrInputDocument();

    Object metaDate = doc.getFieldValue( "last_modified" );
    Object fileDate = doc.getFieldValue( "file_date" );
    if( metaDate == null && fileDate != null) {
        doc.addField( "last_modified", fileDate );
    }

      // pass it up the chain
      super.processAdd(cmd);
    }   
  }

Where file_date is a field I set with the file modification date at import time.

Set last_modified field when not defined in document in Solr

3 Answers3