Logstash using gsub

Question

I would like to use the gsub filter or a ruby code filter to do the following in logstash.

I have a field which is dynamically named eg. P12IP3, P12IP2, P13IP1 etc. I would like to remove all white space characters in these fields. However, the following does not seem to work

gsub => ["/(.)IP(.)/"," ",""]

I've tried some variations using ruby code filter as well, but could not get it to work. Can someone suggest a solution?

Sample Conf of what I have tried

               grok {

                         patterns_dir => "/etc/logstash/patterns"

                         match => [ "message", "iLO %{BASE16NUM:P16F1} %{HLA_TS_1:ts1} / %{BASE16NUM:P16F2}
 %{BASE16NUM:P16F3} :
 %{BASE16NUM:P16F4} %{BASE16NUM:P16F5} Browser login : OA
 Administrator1 \- \ %{IP_HLA:P16IP1} \( DNS name not found \) \." ]

                         add_tag => [ "pattern", "16" ]
                         tag_on_failure => []
                 }

                grok {

                         patterns_dir => "/etc/logstash/patterns"

                         match => [ "message", "iLO %{BASE16NUM:P17F1} %{HLA_TS_1:ts1} / %{BASE16NUM:P17F2} %{BASE16NUM:P17F3} :
 %{BASE16NUM:P17F4} %{BASE16NUM:P17F5} Browser login : OA
 Administrator3 \- \ %{IP_HLA:P17IP1} \( DNS name not found \) \." ]

                         add_tag => [ "pattern", "17" ]
                         tag_on_failure => []
                 }

                  mutate{
                          gsub => [
                          "/(.*)IP(.*)/"," ",""
                          ]
                          }

Here above you can see that there are two IP fields P16IP1 and P17IP1, what I want is that both of them should be replaced by the gsub mutation filter such that all white space is removed in the values of the field.

I am also providing the input, the following is an input for the first pattern (16).

iLO 2 2012 / 31 / 14 13 : 24 : 01 / 2011 12 : 52 1 Browser login : OA Administrator1 - 15 . 33 . 64 . 119 ( DNS name not found ) .

Here the output for the IP field is currently "P16IP1":"15 . 33 . 64 . 119", what I would like is for the output to be "P16IP1":"15.33.64.119"

You need to show real sample input and what you expect out of the `gsub`. — the Tin Man, Jul 17 '15 at 17:48
Hi Please see if the details make sense now with the example.. I am able to make it work if the fieldnames are not variable, i.e. if the field name is simply IP instead of P12IP2 etc. — tsar2512, Jul 17 '15 at 17:58
It's really important to provide sample input. In this case it'd be example strings showing what you're trying to match. Without that we're just guessing what you mean. See http://stackoverflow.com/help/mcve. — the Tin Man, Jul 17 '15 at 19:07
please have a look I have updated the input and expected output — tsar2512, Jul 17 '15 at 19:48

the Tin Man · Answer 1 · 2015-07-17T19:05:40.243

1

Removing all whitespace from a string is easy:

"a \t\n\r\fb".gsub(/\s+/, '') # => "ab"

/\s+/ is the regular expression way of saying "all whitespace characters". This is its definition:

/\s/ - A whitespace character: /[ \t\r\n\f]/

If you're trying to match lines containing variants on

P12IP2
P01IP1
P99IP9

then you can use a pattern like:

/P\d{2}IP\d/

http://rubular.com/r/MCnY87DkZv

From there you can capture the leading/trailing characters:

/^(.+)P\d{2}IP\d(.+)/

http://rubular.com/r/HmekyYzXcU

If it's possible that the first two digits in the string can be shorter or longer than nn you can adjust the {2} size to whatever. See the Regexp documentation for how it works.

edited Jul 17 '15 at 19:05

answered Jul 17 '15 at 17:44

the Tin Man

158,662
42
215
303

I do not want to remove all whitespace characters, I want to remove only the whitespace characters in the field whose key matches [*IP*] – tsar2512 Jul 17 '15 at 17:48
Perhaps you need to define your question better. We need input examples and what your output will be after manipulation. Also show a minimal example of your code that demonstrates the problem you're having. As is, you haven't given us enough to really tell what you want. See http://stackoverflow.com/help/mcve – the Tin Man Jul 17 '15 at 17:51
How do I do this in Logstash? – tsar2512 Jul 17 '15 at 19:48
your answer seems correct, but can you let me know how it would work in logstash? The question was in regards to logstash – tsar2512 Jul 20 '15 at 15:54

Logstash using gsub

1 Answers1