Is posible get the UUID from a log using syslog-ng?

Question

I have my logs centralized into a syslog and I want to obtain the UUID from the coming log to save it into a database. Is this possible?

The log message looks like this:

Mar 28 14:14:26 172.17.42.1 1 2016-03-28T08:14:26.912-03:00 0e40cd94516b service-godzilla-central - Audit - 0c9886fc-ab7f-42a1-8081-5ae6409a0e66 No correlationId found in Header. One was generated.

Mar 28 14:14:26 172.17.42.1 1 2016-03-28T08:14:26.927-03:00 0e40cd94516b service-godzilla-central - Audit - 0c9886fc-ab7f-42a1-8081-5ae6409a0e66 Entrada - Controller - initParameter [{"terminalId":3354,"channel":5,"version":"AU-014"}]

Mar 28 14:14:26 172.17.42.1 1 2016-03-28T08:14:26.927-03:00 0e40cd94516b service-godzilla-central - Audit - 0c9886fc-ab7f-42a1-8081-5ae6409a0e66 Entrada - Service - parameters [{"terminalId":3354,"channel":5,"version":"AU-014","correlationId":"0c9886fc-ab7f-42a1-8081-5ae6409a0e66"}]

Hi, is the UUID already available in the log message, and you want to extract it, or do you want to generate a unique ID for each message with syslog-ng? In the first case, you can probably use a parser, but it depends on the message format. In the second case, newer syslog-ng versions have a template function that can generate UUIDs, see http://docbuilder.balabit/job/syslog-ng-OSE-master-github/lastSuccessfulBuild/artifact/en/out/en/syslog-ng-ose-guide-admin/html/reference-template-functions.html#template-function-uuid — Robert Fekete, Mar 03 '16 at 07:44
The UUID already available in the log message, and I want to extract it — Rys, Mar 03 '16 at 12:45
Then it depends on the message format. Do you receive the message directly from an application, or via a syslog protocol from another host? Can you post a sample message? Probably you will need to use a parser to extract the uuid from the message: https://www.balabit.com/documents/syslog-ng-ose-3.7-guides/en/syslog-ng-ose-guide-admin/html/chapter-parsers.html — Robert Fekete, Mar 04 '16 at 07:23

score 0 · Answer 1 · answered Mar 30 '16 at 10:54

To get the UUID (0c9886fc-ab7f-42a1-8081-5ae6409a0e66), you can use a csv-parser that splits the message into columns at every space. See The syslog-ng Administrator Guide - Parsing messages with comma-separated values for details.

Basically, you have to create a csv-parser that uses a whitespace as a delimiter, and add names for the columns (the last column that has a name will contain the remainder of the message). Something like:

parser p_uuid {
csv-parser(columns("COLUMN1", "COLUMN2", "COLUMN3", "COLUMN4", "COLUMN5", "COLUMN6", "COLUMN7", "COLUMN8" )
     flags(greedy)
     delimiters(" ")
     );

};

Then use this parser in a log path. Try to use it with a file destination where you use the column names in a template to see when you get the parsing right. You can use the parsed columns as macros in the template, for example:

log { source(s_local);
parser(p_uuid); destination(d_file);};
};

destination d_file {
file ("/var/log/parsed-logs" template("${ISODATE} ${HOST} Col1=${COLUMN1} Col2=${COLUMN2} Col3=${COLUMN3} Col4=${COLUMN4} Col5=${COLUMN5} Col6=${COLUMN6} Col6=${COLUMN6} Col7=${COLUMN7} Col8=${COLUMN8}\n") );

};

If you need, you can also use multiple parsers to parse different parts of the messages (for example, if you need the JSON parts of the message, you could use first a csv-parser with the [] delimiters, that would produce two columns, and then run a json-parser on the second column).

Is posible get the UUID from a log using syslog-ng?

1 Answers1