0

I have some logs which look like this : 

2019-10-24 15:14:46,183 [http-nio-8080-exec-2] [bf7ccfa6-e24f-4854-9b1f-753a7886e351, 10.0.22.13, /project/api/traductions/ng/fr, GET, , Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36, 10.128.22.82, 10.255.0.3, 10.0.10.56, 10.0.22.27] [] INFO  ContextUserRepositorySpring - This is my message, with any characters in it. (like those $ - )
2019-10-24 15:21:51,699 [http-nio-8080-exec-8] [, , , , , , ] [] INFO  SecurityAuditEventListener - AuditEvent onApplicationEvent: org.springframework.boot.actuate.audit.listener.AuditApplicationEvent[source=AuditEvent [timestamp=2019-10-24T13:21:51.699813Z, principal=EOFK, type=AUTHENTIFICATION_SUCCESS, data={isauthenticated=true, authTpye=Authentification JWT, requestPath=/api/efsef/subscribe}]]
2019-10-24 15:23:22,578 [http-nio-8080-exec-2] [32d1189f-eac5-47ad-b52e-33323e07c4d7, 10.0.22.13, /webai/api/environnement/, GET, , Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0, 10.128.20.145, 10.255.0.3, 10.0.10.56, 10.0.22.27] [] INFO  SecurityAuditEventListener - AuditEvent onApplicationEvent: ServletRequestHandledEvent: url=[/project/api/environnement/]; client=[10.0.22.13]; method=[GET]; servlet=[dispatcherServlet]; session=[null]; user=[null]; time=[2ms]; status=[OK]

I have a grok parse like that:

"^%{TIMESTAMP_ISO8601:date}\s+\[%{EMAILLOCALPART:thread}\]\s\[(?:|%{UUID:ident}),\s(?:|%{IP:remoteHost}),\s(?:|%{URIPATH:uri}),\s(?:|%{WORD:verb}),\s(?:|%{NOTSPACE:query}),\s(?<userAgent>[^,]*),\s(?:|%{IP:ip1},\s%{IP:ip2},\s%{IP:ip3},\s%{IP:ip4})\]\s\[\]\s%{LOGLEVEL:loglevel}\s+%{NOTSPACE:logger}\s\-\s%{GREEDYDATA:logmessage}"

In this case, the second and third lines are well parsed. But not the first one, because of this comma present. (KHTML, like Gecko). And the regex fails at this level : (?<userAgent>[^,]*),\s

Now I'm wondering if there is a way to get all the logs well parsed. With the current logs. Just by changing the grok matching regex...

jayooin
  • 99
  • 2
  • you can use multiple grok patterns. If first pattern fails it goes to second pattern and match. refer [this](https://stackoverflow.com/questions/28450501/multiple-patterns-in-one-log) – Ishara Madhawa Oct 25 '19 at 08:36
  • I know that, in fact i already use several ones. But I want to get the first line parsed. I can't find a way to do that. – jayooin Oct 29 '19 at 11:35

0 Answers0