1

I'm trying to use Serilogger together with Elasticsearch and I'm clueless on how to get these to work together.

[2015-06-02 12:45:49,149][DEBUG][action.bulk              ] [Jimaine Szardos] [logstash-2015.06.02][4] failed to execute bulk item (index) index {[logstash-2015.06.02][logevent][AU2z4ILA3Y9Gz86fGG3z], source[{"@timestamp":"2015-06-02T12:45:43.4368603+02:00","level":"Information","messageTemplate":"Creating new Taxi {0}","message":"Creating new Taxi \"84/10749/18/27/80\"","fields":{"0":"84/10749/18/27/80","SourceContext":"Akka.Logger.Serilog.SerilogLogger"}}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [fields.0]
        at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:409)
        at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
        at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
        at org.elasticsearch.index.mapper.object.ObjectMapper.serializeObject(ObjectMapper.java:554)
        at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:487)
        at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
        at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
        at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:453)
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432)
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149)
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515)
        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NumberFormatException: For input string: "84/10749/18/27/80"
        at java.lang.NumberFormatException.forInputString(Unknown Source)
        at java.lang.Long.parseLong(Unknown Source)
        at java.lang.Long.parseLong(Unknown Source)
        at org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
        at org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:300)
        at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:236)
        at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:399)
        ... 14 more

This seems to be the content of the logentry

[{
  "@timestamp":"2015-06-02T12:45:43.4368603+02:00", 
  "level":"Information",
  "messageTemplate":"Creating new Taxi {0}", 
  "message":"Creating new Taxi \"84/10749/18/27/80\"",
  "fields": {
       "0":"84/10749/18/27/80",
       "SourceContext":"Akka.Logger.Serilog.SerilogLogger"
  }
}]

Is the reason for the issue here that Serilog posts malformed logentries or is there something else going on here?

Roger Johansson
  • 22,764
  • 18
  • 97
  • 193
  • It seems that `fields.0` has been mapped as a number in your mapping type and `"84/10749/18/27/80"` cannot be converted to a number. Can you show the output of this command `curl -XGET localhost:9200/logstash-2015.06.02/_mapping/logevent`? Do you have a custom mapping or did you let Serilog handle it for you? – Val Jun 02 '15 at 11:13
  • `λ curl -XGET localhost:9200/logstash-2015.06.02/_mapping/logevent {"logstash-2015.06.02":{"mappings":{"logevent":{"properties":{"@timestamp":{"type":"date","format":"dateOptionalTime"},"fields":{"properties":{"0":{"type":"long"},"Message":{"type":"string"},"SourceContext":{"type":"string"}}},"level":{"type":"string"},"message":{"type":"string"},"messageTemplate":{"type":"string"}}}}}}` – Roger Johansson Jun 02 '15 at 11:18
  • serilog has created the mapping – Roger Johansson Jun 02 '15 at 11:18
  • 1
    There we go, `"fields":{"properties":{"0":{"type":"long"‌​}`. Are you using [serilog-sinks-elasticsearch](https://github.com/serilog/serilog-sinks-elasticsearch)? – Val Jun 02 '15 at 11:23
  • Yes. I do think I found the problem. when using Serilog, the logformat is ofc different than normal string.format in .NET updated it now and atleast my custom log entries seems to pass now.. thanks! – Roger Johansson Jun 02 '15 at 11:29
  • Cool, glad you found it. – Val Jun 02 '15 at 11:33
  • 1
    I've raised this issue on the Serilog project to see what the status of numeric property names is - these should be handled by the sink. https://github.com/serilog/serilog/issues/462 – Nicholas Blumhardt Jun 02 '15 at 20:55
  • Can you validate that a template is added for your indexes? If you use ES sink version 2 you can register a template automatically that will map any fields.{digit} to a string which is the safest way to store this. So which version of the sink are you using, do you use the template and if so, is it indeed there? Keeping in mind that it will only be applied for new indexes. – Michiel Jun 03 '15 at 18:56

0 Answers0