2

I am trying to remove all the lines starting from < in my input file in logstash. I tried below filter but no success.

I am new to ELS stack, please pardon if it has already answered.

Sample Input lines

Thread #1: t@-1680123584, lwp=22843, ref=0x1b0f8550, session=9F0071A66D89544155D149CCE2453E9A:mx2135649930e123d964:(WebServiceFacade.java:84), ms=0x78714bc0
<Start Stack Trace>
<1 - ADK Verbose Trace Entry>

  <2 - Launching Program Object>
  
  <3 - ADK Verbose Trace Entry>

  <4 - Launching Program Object>

<End Stack Trace>
Thread #2: t@-1680123584, lwp=22843, ref=0x1b0f8550, session=9F0071A66D89544155D149CCE2453E9A:mx2135649930e123d964:(WebServiceFacade.java:84), ms=0x78714bc0
<Start Stack Trace>
  <1 - ADK Verbose Trace Entry>

  <2 - Launching Program Object>
  
  <3 - ADK Verbose Trace Entry>

  <4 - Launching Program Object>

<End Stack Trace>

My current filter is like

    if[message =~ "^<.*"] {
        drop { }
    }

But somehow it is not working, I am still getting these lines in my output

 "message" => "Thread #1: t@-1680123584, lwp=22843, ref=0x1b0f8550, session=9F0071A66D89544155D149CCE2453E9A:mx2135649930e123d964:(WebServiceFacade.java:84), ms=0x78714bc0\r\n<Start Stack Trace>\r\n<1 - ADK Verbose Trace Entry>\r\n\r\n  <2 - Launching Program Object>\r\n  \r\n  <3 - ADK Verbose Trace Entry>\r\n\r\n  <4 - Launching Program Object>\r\n\r\n<End Stack Trace>\r",
Ankit Sachan
  • 51
  • 2
  • hi perhaps add whitespace (`\s*`) to the beginning, since the message starts with `\r\n`. – IronMan Feb 17 '21 at 19:03
  • How are you sending your messages to logstash? Are you using filebeat with multiline pattern or have a multiline pattern in your logstash input? The message you shared is one single event for logstash. – leandrojmp Feb 17 '21 at 19:16

2 Answers2

0

You should have square brackets around the field name. Parentheses around the expression are optional. Try

if [message] =~ "^<" { drop { } }
Badger
  • 3,943
  • 2
  • 6
  • 17
0

I managed to fix it, I am using multiline codec in input, because of that it is not considering the lines with \r or \n. My actual expression is working fine if I remove multiline codec.

if[message =~ "^<.*"] {
        drop { }
    }
Ankit Sachan
  • 51
  • 2
  • I have some further questions on similar topic, Can I ask here in same thread or I should start a new thread. Please suggest. – Ankit Sachan Feb 18 '21 at 13:56