4

I have an input that is like the following

2015-12-02T12:39:57.546461+08:00 222.55.201.19 c): start_time="2015-12-02 ...some other log info... src=192.168.1.153 dst=8.8.8.8 src_port=51145 dst_port=13 ...some other log info... start_time="2015-12-02 src=192.168.1.123 dst=8.8.8.8 src_port=52245 dst_port=53 ...some other log info...

The above has many src and dst and all these entire are in one line.

When I use the "message" => "src=%{IP:SRC} dst=%{IP:DST}". I manage to get only the first src and dst. I am unable to get the rest of the src and dst in the line.

What command should I use to achieve what I need? I heard that I might need to do some split but I cannot find information on this.

baudsp
  • 4,076
  • 1
  • 17
  • 35
Shawn Sim
  • 545
  • 1
  • 5
  • 17
  • I feel I encountered the same error as you : http://stackoverflow.com/questions/29164972/how-to-split-logstash-event-containing-multiple-times-the-same-pattern – vdolez Jan 13 '16 at 13:20
  • 1
    To solve my problem, I had to design multiple logstash conf and link them. You can learn more about the split filter in the documentation : https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html – vdolez Jan 13 '16 at 13:24
  • I have solved this problem via a work around. I have made changes to how the source is generated. The source now has record per line. – Shawn Sim Jan 20 '16 at 08:08
  • 1
    I have tried the split method. It does fix the problem but the amount of codes significantly increase. – Shawn Sim Jan 20 '16 at 08:08

2 Answers2

1

I have solved this problem via a work around. I have made changes to how the source is generated. The source now has record per line.

I have also tried the "split" method. It does fix the problem but the amount of codes significantly increase

Shawn Sim
  • 545
  • 1
  • 5
  • 17
  • Could you tell the Logstash version you were using ? Nice to here you resolved your problem. – vdolez Jan 20 '16 at 08:13
1

Another much better way is the kv filter

Shawn Sim
  • 545
  • 1
  • 5
  • 17