0

I have reading logs from a log file which is recording multiline type. While reading QRadar assembling two record and take it as a one log.

I have describe start and end pattern of the log line while adding the log source to QRadar as:

  • Start Pattern RegEx: ^(\d{7})\,

  • End Pattern RegEx: (\d{2}:\d{2}:\d{2})$

I should have read the logs like :

1158896,someuser,Inner User,Minor,10.6.130.11,2019-09-29 03:01:15,Security Management,Log in to the server,Network Management,Succeeded,User name: someuser,2019-09-29 03:01:15
1158897,someuser,Inner User,Minor,10.6.130.11,2019-09-29 03:03:16,Security Management,Log out the server,Network Management,Succeeded,"User name: someuserOnline duration: 0 day(s) 0 hour(s) 2 minute(s) 1 second(s)",2019-09-29 03:03:16

But I receive some of them assembled, like:

1158896,someuser,Inner User,Minor,10.6.130.11,2019-09-29 03:01:15,Security Management,Log in to the server,Network Management,Succeeded,User name: someuser,2019-09-29 03:01:151158897,someuser,Inner User,Minor,10.6.130.11,2019-09-29 03:03:16,Security Management,Log out the server,Network Management,Succeeded,"User name: someuserOnline duration: 0 day(s) 0 hour(s) 2 minute(s) 1 second(s)",2019-09-29 03:03:16

Here are the regex101.com records of my start and end pattern of RegEx.

https://regex101.com/r/2IfMR7/3

https://regex101.com/r/2IfMR7/4

As you see, it works normally in regex101.com Why QRadar is reading them as one?

Das_Geek
  • 2,775
  • 7
  • 20
  • 26
Gogol
  • 3
  • 2

1 Answers1

0

You (or qradar) might be using a greedy quantifier coupled with a multiline capture character.

If you're doing something like this: ^(\d{7})\,(?:\n|.)*(\d{2}:\d{2}:\d{2})$ where the central group is (?:\n|.)* or any similar phrase matching across multiple lines, the greedy operator * means it'll try to match from the very first 7 digits to the very last timestamp on the entire log page, ignoring your start and end anchors. Try using *? instead; the ? makes it non-greedy, so it'll stop at the first timestamp.

Compare: greedy vs non-greedy.

Nick Reed
  • 4,989
  • 4
  • 17
  • 37
  • Actually, in QRadar, if you were add log source by reading from log file (which is written not line-by-line but multiline, you have to select from the settings multiline. Wen it is selected **Multiline**, it asks you **start pattern of regex**, and **end pattern of regex**. So there is no choice to enter exact the same regex you offer. That's why I write the way I show you in regex101.com. It works when I write them in multiline (https://regex101.com/r/2IfMR7/5) however, QRadar may count 2 as 1 sometime – Gogol Oct 01 '19 at 13:25
  • @Gogol after reading through the QRadar wiki, you might have luck if you remove the end regex. I don't have the software myself, so I'm not sure exactly how it works, but several wiki answers for similar problems recommend leaving the end regex blank. I'd guess that if you do, QRadar only matches up to the next occurrence of a start regex. – Nick Reed Oct 01 '19 at 16:40
  • Thank you for your response. I will edit the settings now and write the result. – Gogol Oct 02 '19 at 10:15
  • Thank you again, Nick. It works well now. Would you share the link you found on QRadar wiki? – Gogol Oct 06 '19 at 09:51
  • @Gogol [Here's](https://www.ibm.com/support/knowledgecenter/en/SS42VS_7.2.7/com.ibm.qradar.doc/c_qradar_cust_evt_fl_prop.html) the [links](https://www.ibm.com/support/knowledgecenter/en/SS42VS_7.2.8/com.ibm.qradar.doc/c_qradar_adm_dsm_ed_prop_config.html) I used. Glad to hear everything is working well! Please consider accepting the answer if it addresses your question, so future readers know what works. – Nick Reed Oct 06 '19 at 17:46