logstash config loop through fields

Question

ruby/logstash noob here using ELK stack.

I got a bunch of fields

[Message][Detail][Readout][Value1]

[Message][Detail][Readout][Value2]

[Message][Detail][Readout][Value3]

which I want to loop through using ruby in the logstash config.

Then I want to perform a simple operation on each, for example change them from hex to decimal e.g.

event.set('[currField]', event.get('[currField]').to_s.hex);

but I cant find the correct syntax using google.. any help appreciated.

I know the names of the fields, so worst case I'll have to hard code them, but I'd like to avoid that if possible.

EDIT: i have not tested my config yet, so i dont know if "Readout" will be a hash map; im using grok filter to add the values in the config

"(?<[Message][Detail][Readout][Value1]>(?<=0x.{8})([A-F0-9]{2}))",
"(?<[Message][Detail][Readout][Value2]>(?<=0x.{8})([A-F0-9]{2}))"

etc

Pseudo:

event.get('[Message][Detail][Readout]') each { |k, v|
  event[k] = newValue;
}

So [Message][Detail][Readout] is a hash and you want to iterate over the members? — Badger, Dec 27 '20 at 19:16
nope, they are seperate fields, but all with [Readout] prefix — Master Azazel, Dec 27 '20 at 19:21
Are you saying the field names contain square brackets, or is [Message] an object that contains a [Detail] hash? — Badger, Dec 27 '20 at 19:31
no thats just how you access them in the logstash config i think.. Think theyre with dots when i look at them in kibana. `Message.Detail.Readout.Value1` — Master Azazel, Dec 27 '20 at 19:45
i think its a hash, looked it up in raw JSON, but im adding my fields using grok, so im not sure if they will be added as a hash too — Master Azazel, Dec 27 '20 at 19:57

Badger · Accepted Answer · 2020-12-28T01:39:34.537

2

You would use .each to iterate over the [Message][Detail][Readout] hash. Your pseudo-code would set the values at the top-level. To overwrite them use

ruby {
    code => '
        readout = event.get("[Message][Detail][Readout]")
        if readout
            readout.each { |k, v|
                event.set("[Message][Detail][Readout][#{k}]", v.to_s.hex)
            }
        end
    '
}

edited Dec 28 '20 at 01:39

answered Dec 27 '20 at 23:06

Badger

3,943
2
6
17

your suggestion worked like a charm; thanks! – Master Azazel Jan 20 '21 at 14:19

logstash config loop through fields

1 Answers1