Solr 'Invalid Date String' Exception

Question

I'm new to Solr. I successfully indexed some data, but after passing a date attribute to an appropriate solr field I'm receiving an exception:

I'm passing the String "15.06.2011 13:51:41", which is a common date format for countries like Germany. However, after some googling I found out, that Solr expects a date to be in the UTC format.

My specific question is: Can I transform my format to UTC at some point at Solr? Would this be a case for the DateFormatTransformer or is it a MUST to pass it in the correct format initially?

EDIT: I am not using a DataImportHandler. I'm using the DirectUpdateHandler2 by passing the data directly to Solr. Since it's called 'direct update', are my chances pretty bad in transforming anything there?

For the records, the exception I got was:

ERROR - 2013-09-13 15:52:07.705; org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: Invalid Date String:'15.06.2011 13:51:41'
    at org.apache.solr.schema.DateField.parseMath(DateField.java:182)
    at org.apache.solr.schema.TrieField.createField(TrieField.java:616)
    at org.apache.solr.schema.TrieField.createFields(TrieField.java:655)
    at org.apache.solr.schema.TrieDateField.createFields(TrieDateField.java:157)
    at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:47)
    at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:118)
    at org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:73)
    at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:210)
    at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:556)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:692)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435)
    at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.doAdd(ExtractingDocumentLoader.java:121)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.addDoc(ExtractingDocumentLoader.java:126)
    at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:228)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
    at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
    at org.eclipse.jetty.server.Server.handle(Server.java:368)
    at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
    at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
    at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
    at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
    at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
    at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
    at java.lang.Thread.run(Unknown Source)

score 26 · Accepted Answer · answered Sep 16 '13 at 15:56

26

According to the Solr DateField documentation, this format is required:

date field shall be of the form 1995-12-31T23:59:59Z The trailing "Z" designates UTC time and is mandatory (See below for an explanation of UTC). Optional fractional seconds are allowed, as long as they do not end in a trailing 0 (but any precision beyond milliseconds will be ignored). All other parts are mandatory.

The only option is to transform the date prior to sending it into Solr. If you are using DataImportHandler, it would be possible within that context/process.

answered Sep 16 '13 at 15:56

Paige Cook

22,415
3
57
68

1

Thanks for your response. The framework I'm using (ManifoldCF) uses the DirectUpdateHandler2 and the DataImportHandler is not used at all. Is there any possibility to achieve a date transformation prior to the DirectUpdateHandler2 call? Speaking of Solr of course. – Benjamin Brandmeier Sep 18 '13 at 14:35
Given that you are using the DirectUpdateHandler2, from what I can determine, there is not any way within Solr to add a data transformation to this process short of creating a customized version of the DirectUpdateHandler. I am not familiar with ManfioldCF, so I cannot offer if there would be any options within there. – Paige Cook Sep 18 '13 at 17:09
1

As far as I can tell, there aren't any offers in ManifoldCF regarding this (since this is Connector specific). However, I will accept your answer, since I didn't clarify enough in my question. Thanks again for your commitment. – Benjamin Brandmeier Sep 19 '13 at 06:56

Solr 'Invalid Date String' Exception

1 Answers1

Linked