Send multiple dynamic metrics to Ganglia in logstash

Question

I want to cound the number of messages sent from different instances of program (nodes). These nodes appear dynamically. The messages format will be like:

node01 - I feel like I'm loosing the grip.
node02 - I'm sad.
node02 - I'm loosing the reason to live.

I indend to parse logs and count statistics in Logstash (with metrics filter) and then send them to Ganglia for display. However, each call to ganglia output in Logstash only allows to send one statistic (not an array like in Graphite output). I only know how to achieve my Logstash configuration with hardcoding of node names:

input { stdin { } }

filter
{
    grok { match => [ "message", "%{WORD:instance} - %{GREEDYDATA:data}" ] } 

    metrics
    { 
        meter => "events.%{instance}"
        add_tag => "metric"
        flush_interval => 1
    }
}

output 
{

    if "metric" in [tags]
    {
        ganglia
        {
            group => "node01"
            host => "239.2.11.71"
            metric => "events"
            metric_type => "uint32"
            value => "%{events.node01.rate_1m}"
        }

        ganglia
        {
            group => "node02"
            host => "239.2.11.71"
            metric => "events"
            metric_type => "uint32"
            value => "%{events.node02.rate_1m}"
        }

        stdout { codec => rubydebug { } }
    }
}

Maybe you could advise me how to create a configuration that is agnostic of node names? I thought of looping through events array in ruby filter (which allows you to write arbitrary code), and creating new events, but I don't know how to create new event from code.

I also thought of splitting events using multiline, but I don't know how to translate the events array to something splittable. I think I could use gsub in mutate filter, but I'm not sure how to use it on array and how to obtain node names from it.

Update

I checked the included solution and it does not work either, I think Logstash only uses one output of one type. I would have to clone my outputs and tag them separately to make it work (not sure if this is possible). Anyway, it's not the desired solution, I only wanted to present better what I'm trying to achieve.

Send multiple dynamic metrics to Ganglia in logstash

0 Answers0