how do you create filters on time series data in logstash

Question

there is a file that has entries like this:

2016-01-22 10:01:44.043, cash.read, 93.67088

2016-01-22 10:01:44.043, cahhe.size, 79

<timestamp>   <metric> <value>

There are 1000's of metrics. Can somebody guide me through creating filter to create entries in logstash to push it to elasticsearch?

Have you met the grok debugger or read any information on learning to use grok? http://svops.com/blog/introduction-to-logstash-grok-patterns/ — Alain Collins, Jan 29 '16 at 00:00

score 0 · Answer 1 · answered Feb 01 '16 at 10:33

You can use grok and kv filters to achive that. The filter will be something like:

filter {
   grok {
        match => {
        message => "%{TIMESTAMP_ISO8601:timestamp},%{SPACE}%{GREEDYDATA:metrics}"
        }
        }

  date {
            match => [ "timestamp", "ISO8601" ]
            remove_field => [ "timestamp" ]
    }


  kv {
        source => "metrics"
        value_split => ","
        trim => " "
        remove_field => [ "metrics", "message" ]
    }

}

how do you create filters on time series data in logstash

1 Answers1