1

I have a json file which is having data like this

{
    "foo" : "bar",
    "test" : {
        "steps" : [{
                "response_time" : "100"
            }, {
                "response_time" : "101",
                "more_nested" : [{
                        "hello" : "world"
                    }, {
                        "hello2" : "world2"
                    }
                ]
            }
        ]
    }
}

I am using logstash filter, which is giving me wrong result

input {  
      file {
          sincedb_path =>  ".../sincedb_path.txt"
          path => ".../test.json"
          type => "test"
          start_position => "beginning"
      }
}



filter{
    json{
        source => "message"
    }

}



output {  
stdout { codec => rubydebug}

 }

How can i accomplish the events like steps.response_time :100 steps.more_nested.hello : world.

I tried with ruby but not working. I am using logstash 5.x version

ritesh
  • 35
  • 6
  • In your `test.json` file you need to make sure that the whole JSON is on a single line, i.e. not pretty-printed – Val Sep 14 '17 at 13:06
  • i have tried that and with that i am not able to create index, it fails. I get this error in console : 2017-09-14T18:48:05,532][DEBUG][logstash.inputs.file ] each: file grew bin/test.json: old size 0, new size 219 – ritesh Sep 14 '17 at 13:19
  • also my bigger problem is how to get the nested array into separate events. – ritesh Sep 14 '17 at 13:20
  • Try to remove the `sincedb_path.txt` file first – Val Sep 14 '17 at 13:33
  • I am getting same issue : [2017-09-14T19:12:01,596][DEBUG][logstash.inputs.file ] each: file grew: .../logstash-5.5.2/bin/test.json: old size 0, new size 3349, after delete the line for sincedb_path itself from conf file – ritesh Sep 14 '17 at 13:42
  • Keep the setting in your logstash config, but simply delete the `sincedb_path.txt` file from the filesystem, so that Logstash restarts from the beginning of the JSON file. Also maybe rename your JSON file – Val Sep 14 '17 at 13:45
  • i delete the sincedb_path.txt file. and renamed the jsonfile. rename the setting in conf file as well.When i ran sincedb path txt file got generated. But i am getting same error no changes – ritesh Sep 14 '17 at 13:53
  • Note that it is not an error but a debug message. – Val Sep 14 '17 at 13:56
  • yes, but i am not able to create index when i make a single line json. With pretty print at least it creates index, but i am not able parse that – ritesh Sep 14 '17 at 13:57
  • Have you inserted a newline character at the end of the line so that Logstash picks it up? Also you don't have any `elasticsearch` output in your logstash config, so nothing can land into ES – Val Sep 14 '17 at 13:59
  • yes, the new line text was a problem.Now at least it is creating 2 columns. But back to my original question, i am not able to parse the nested json i still get test.steps : { "response_time": "100" }, { "more_nested": [ { "hello": "world" }, { "hello2": "world2" } ], "response_time": "101" }..... i need to get test.steps.response_time :100 – ritesh Sep 14 '17 at 14:07

0 Answers0