How can I set up syslog-ng to send only invalid JSON logs to a destination

Question

I've set up syslog-ng 3.25.1 to read log messages from a file, parse them as JSON, and forward them to an ElasticSearch cluster.

This works well when the messages are formatted correctly as JSON, but in cases when json-parser fails to find a valid JSON object in my log message, I'd like to wrap the non-JSON log message in a JSON object (via a format-json template) and send it to a different index in ElasticSearch.

I understand that json-parser can act as a filter to pass only valid JSON messages. Is there a way to create a filter that passes only invalid JSON messages?

score 1 · Answer 1 · answered Jun 08 '20 at 21:13

I'm not sure if this is the best way to get it working, but I wound up using if/else flow control in my logging path, similar to the following.

source s_json_source {
 ...
};

destination d_valid_json {
  ... // includes template definition
};

destination d_invalid_json {
  ... // includes template definition that wraps the ${MSG} in a timestamped json object
};

parser p_json {
  json-parser();
};

log {
  source(s_json_source)
  if {
    parser(p_json);
    destination(d_valid_json);
  } else {
    destination(d_invalid_json);
  };
};

How can I set up syslog-ng to send only invalid JSON logs to a destination

1 Answers1