How to easily promote a JSON member to the main event level?

Question

I'm using an http_poller to hit an API endpoint for some info I want to index with elasticsearch. The result is in JSON and is a list of records, looking like this:

{
  "result": [
     {...},
     {...},
     ...
  ]
}

Each result object in the array is what I really want to turn into an event that gets indexed in ElasticSearch, so I tried using the split filter to turn the object into a series of events instead. It worked reasonably well, but now I have a series of events that look like this:

{ 
  result: { ... }
}

My current filter looks like this:

filter {
  if [type] == "history" {
    split {
      field => "result"
    }
  }
}

Each of those result objects has about 20 fields, most of which I want, so while I know I can transform them by doing something along the lines of

filter {
      if [type] == "history" {
        split {
          field => "result"
        }
        mutate {
           add_field => { "field1" => "%{[result][field1]}"
           #... x15-20 more fields
           remove_field => "result"
        }
      }
    }

But with so many fields I was hoping there's a one-liner to just copy all the fields of the 'result' value up to be the event.

So, I marked as duplicate, but I noticed that while the general solution of "use a ruby filter" is still correct, the code in the other answer didn't work as expected in my version of Logstash, 5.4, but the one here did. — Paul, May 25 '17 at 19:48

score 2 · Accepted Answer · answered May 25 '17 at 15:04

This can be done with a ruby filter like this:

       ruby {
                code => '
                        if (event.get("result"))
                                event.get("result").each { |k,v|
                                        event.set(k,v);
                                }
                                event.remove("result");
                        end
                '
        }

I don't know of any way to do this with any of the built in/publicly available filters.

How to easily promote a JSON member to the main event level?

1 Answers1