Add data to dynamic nested hash in logstash

Question

I want to put a value into part of a nested hash, but name that part depending on upstream filters. This is to refactor and reduce overall code size as currently each of the 20+ incoming event types have their own section like this with 18 lines in the logstash file (but currently the %{detail_part} bit is hard-coded).

# Working code
filter {
    if [result] == 0 {
        # Success
        mutate {
            add_field => {
                "[Thing][ThingDetail][OtherThing][MoreDetail]" => "true"
            }
        }
    }
    else {
        # Failed
        mutate {
            add_field => {
                "[Thing][ThingDetail][OtherThing][MoreDetail]" => "false"
            }
         }
    }
}

Above is hard-coded to "OtherThing". Below has a variable, but doesn't work.

# Non-Working code
filter {
    if [result] == 0 {
        # Success
        mutate {
            add_field => {
                "[Thing][ThingDetail][%{detail_part}][MoreDetail]" => "true"
            }
        }
    }
    else {
        # Failed
        mutate {
            add_field => {
                "[Thing][ThingDetail][%{detail_part}][MoreDetail]" => "false"
            }
         }
    }
}

In the above (non-Working code), detail_part is set in an upstream filter to a string value like "OtherThing". This currently compiles and runs, but no XML is output from it, so I don't think anything is set into the hash as a result of these statements.

I know it can be done with embedded Ruby code, but I would like a way that is as simple as possible. The output of this process is going to XML so I am constrained to use this kind of nested hash.

Is this possible with Logstash?

score 2 · Answer 1 · answered Aug 10 '15 at 23:13

It turns out that yes, Logstash supports this, just the syntax was wrong. So here is the fix:

filter {
    # This field could be set conditionally in an if statement ...
    mutate { add_field => { "[detail_part]" => "Result" } }

    if [result] == 0 {
        # Success
        mutate {
            add_field => {
                "[Thing][ThingDetail][%{[detail_part]}][MoreDetail]" => "true"
            }
        }
    }
    else {
        # Failed
        mutate {
            add_field => {
                "[Thing][ThingDetail][%{[detail_part]}][MoreDetail]" => "false"
            }
        }
    }
}

I just couldn't find any non-trivial examples that did things like this.

Add data to dynamic nested hash in logstash

1 Answers1