1

I am reading data from files by defining path as *.log etc,

Files names are like app1a_test2_heep.log , cdc2a_test3_heep.log etc

How to configure logstash so that the part of string that is string before underscore (app1a, cdc2a..) should be grepped and added to host field and removing the default host.

Eg:

fileName: app1a_test2_heep.log

host => app1a

Thanks in advance, Ravi

Ravi Kishore
  • 139
  • 1
  • 2
  • 7

2 Answers2

6

Ruby filter can do what you want.

input {
       file {
               path => "/home/benlim/app1a_test2_heep.log"
       }
}

filter {
        ruby {
                code => "
                        filename = event['path'].split('/').last
                        event['host'] = filename.split('_').first
                "
        }
}

output {
        stdout { codec => rubydebug }
}

First, get the filename from the path. Then, get the hostname.

Ban-Chuan Lim
  • 7,840
  • 4
  • 35
  • 52
  • Thanks you very much..Where do I need to use this filter? Is this in file input or filter section? – Ravi Kishore Apr 23 '15 at 10:03
  • Im getting an error if I add this in filter section............ Error: Expected one of #, => at line 31, column 8 (byte 759) after input { – Ravi Kishore Apr 23 '15 at 10:47
  • You must add this in your filter section. Or can you provide your full config, because the error shows your have miss some symbol, like: [, { or #. I have updated my full config. you can try it. – Ban-Chuan Lim Apr 24 '15 at 03:29
  • Thank you very muchhh Ben. It worked for me. It was completely my mistake in the previous stage, I missed } part. Thanks again – Ravi Kishore Apr 24 '15 at 05:16
-2

You can gsub (substitute) in mutate filter. It takes 3 arguments - field, what to sub and on what. You can use regexp here.

filter {
  mutate {
    gsub => [
      # replace all after slashes with nothing
      "host", "[_.*]", ""
    ]
  }
}
Eddy
  • 41
  • 1
  • 9
  • This regexp doesn't do what you want. It effectively removes all underscores, periods, and asterisks from the contents of the host field without even taking the fileName field into account. – Magnus Bäck Apr 23 '15 at 09:47