8

I have a logfile in which the data is separated by a pipe symbol. "|". An example is below. Does anyone know how to write a GROK pattern to extract it for logstash?

2014-01-07 11:58:48.7694|LOGLEVEL|LOGSOURCE|LOGMESSAGE

baudsp
  • 4,076
  • 1
  • 17
  • 35
CodeRunner
  • 391
  • 2
  • 4
  • 9

4 Answers4

13

You can use gsub API to change the pipe "|" to space and the use GROK to extract it.

For example:

filter {
    grok {
            match => ["message","%{DATESTAMP:time}\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}"]
    }
}

The above configuration is worked on me with your log. Hope this can help you.

Ban-Chuan Lim
  • 7,840
  • 4
  • 35
  • 52
  • 13
    grok { match => ["message","%{DATESTAMP:time}\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}"] } This also will work, it worked for me – AD14 Jan 07 '15 at 11:39
  • @ArulT, your comment has more upvotes than the answer, can you please change it to a proper answer? – Custodio Oct 09 '20 at 10:17
6

use this filter:

it works for me. use this site to verify grok patern, https://grokdebug.herokuapp.com/

(?<date>(([0-9]+)-*)+ ([0-9]+:*)+.*)\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}
Jeeva N
  • 431
  • 1
  • 5
  • 17
1

This worked for me

grok { match => ["message","%{DATESTAMP:time}\|%{WORD:LOGLEVEL}\|%{WORD:LOGSOURCE}\|%{WORD:LOGMESSAGE}"] }
AD14
  • 1,218
  • 19
  • 32
0

The LOGMESSAGE part can contain a long content. For this reason, I recommend the following usage.

%{GREEDYDATA:LOGMESSAGE}

Ahmet MUM
  • 1
  • 1
  • Use http://grokdebug.herokuapp.com/ – Ahmet MUM Sep 29 '21 at 05:53
  • 1
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Sep 29 '21 at 06:07