1

I can't figure out how to index my json from stdin into ElasticSearch via Logstash. Here's my configuration:

input {
    stdin {
        codec => "json"
    }
}

filter {
    json {
      source => "results"
    }
}

output {
  elasticsearch {
    action => index
    document_id => "%{objectId}"
    document_type => "result"
    embedded => true
    index => "my_index"
  }

  stdout { 
    codec => "json"
  }
}

I ran Logstash with the following input:

{"results": "{\"objectId\": \"hello\", \"name\" : \"world\"}"}

And if you were wondering why I need the "results" field, it's because the real input is a JSON object with the "results" field containing the JSON to index. When I try to run with the above line, I get the following error:

failed action with response of 400, dropping action: ["index", {:_id=>"hello", :_index=>"my_index", :_type=>"result", :_routing=>nil}, #<LogStash::Event:0x57cf4ab @metadata={"retry_count"=>0}, @accessors=#<LogStash::Util::Accessors:0x31bbe100 @store={"results"=>"{\"objectId\": \"hello\", \"name\" : \"world\"}", "@version"=>"1", "@timestamp"=>"2015-05-30T05:59:23.972Z", "host"=>"laptop", "objectId"=>"hello", "name"=>"world"}, @lut={"host"=>[{"results"=>"{\"objectId\": \"hello\", \"name\" : \"world\"}", "@version"=>"1", "@timestamp"=>"2015-05-30T05:59:23.972Z", "host"=>"laptop", "objectId"=>"hello", "name"=>"world"}, "host"], "results"=>[{"results"=>"{\"objectId\": \"hello\", \"name\" : \"world\"}", "@version"=>"1", "@timestamp"=>"2015-05-30T05:59:23.972Z", "host"=>"laptop", "objectId"=>"hello", "name"=>"world"}, "results"], "objectId"=>[{"results"=>"{\"objectId\": \"hello\", \"name\" : \"world\"}", "@version"=>"1", "@timestamp"=>"2015-05-30T05:59:23.972Z", "host"=>"laptop", "objectId"=>"hello", "name"=>"world"}, "objectId"]}>, @data={"results"=>"{\"objectId\": \"hello\", \"name\" : \"world\"}", "@version"=>"1", "@timestamp"=>"2015-05-30T05:59:23.972Z", "host"=>"laptop", "objectId"=>"hello", "name"=>"world"}, @metadata_accessors=#<LogStash::Util::Accessors:0x29458b08 @store={"retry_count"=>0}, @lut={}>, @cancelled=false>] {:level=>:warn}

What am I doing wrong?

restassured
  • 1,008
  • 2
  • 9
  • 12
  • 1
    Try only using one of the json codec or filter. – Alain Collins May 31 '15 at 00:01
  • Thanks, @AlainCollins. I believe the error is caused by the output filter codec => "json". Once I changed that to codec => "rubydebug", for instance, it was able to display the results of the json filter correctly. Though one step closer, unfortunately, this is not a real example of the JSON I get as an input. The real example contains an array of JSONs. And that gives me a completely different error. I edited my post to show the error. Any suggestions? – restassured May 31 '15 at 14:11

0 Answers0