parse_audit_log not working with td-agent

Question

I have installed the td-agent with version td-agent-4.1.0-1.el7.x86_64 on CentOS7. I'm not able to load the gem.

# td-agent-gem install fluent-plugin-filter-parse-audit-log
Fetching fluent-plugin-filter-parse-audit-log-0.1.2.gem
Fetching audit_log_parser-0.1.3.gem
Successfully installed audit_log_parser-0.1.3
Successfully installed fluent-plugin-filter-parse-audit-log-0.1.2
Parsing documentation for audit_log_parser-0.1.3
Installing ri documentation for audit_log_parser-0.1.3
Parsing documentation for fluent-plugin-filter-parse-audit-log-0.1.2
Installing ri documentation for fluent-plugin-filter-parse-audit-log-0.1.2
Done installing documentation for audit_log_parser, fluent-plugin-filter-parse-audit-log after 0 seconds
2 gems installed

Input config:

<source>
  @type tail
  path /var/log/audit/audit.log
  pos_file /var/log/td-agent/audit.pos
  tag syslog.audit
  <parse>
    @type parse_audit_log
  </parse>
</source>

Error to load parse_audit_log

   # /opt/td-agent/bin/fluentd --dry-run --config /etc/td-agent/td-agent.conf
    2021-03-24 15:01:19 +0000 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '4.3.3'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-filter-parse-audit-log' version '0.1.2'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-flowcounter-simple' version '0.1.0'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-kafka' version '0.16.0'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-prometheus' version '1.8.5'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.0.2'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.3.0'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-s3' version '1.5.1'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-sd-dns' version '0.1.0'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.2'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-td' version '1.1.0'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluent-plugin-webhdfs' version '1.4.0'
    2021-03-24 15:01:19 +0000 [info]: gem 'fluentd' version '1.12.1'
    2021-03-24 15:01:19 +0000 [info]: starting fluentd-1.12.1 as dry run mode ruby="2.7.2"
    2021-03-24 15:01:19 +0000 [warn]: [output_td] Use different plugin for secondary. Check the plugin works with primary like secondary_file primary="Fluent::Plugin::TreasureDataLogOutput" secondary="Fluent::Plugin::FileOutput"
    2021-03-24 15:01:20 +0000 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'parse_audit_log'. Run 'gem search -rd fluent-plugin' to find plugins"

Could you please let me know what are the way to load plugins? I have followed the https://docs.fluentd.org/deployment/plugin-management#for-td-agent but still some issues while restarting the service with systemctl.

It's a filter plugin. You're using it as a parser plugin. See a sample config here: https://github.com/winebarrel/fluent-plugin-filter-parse-audit-log#output. Use it in a `` section. — Azeem, Mar 24 '21 at 16:25
Thanks Azeem! What should be the input `` section? I have used `json` but I'm getting following in the td-agent.log file. `#0 pattern not matched: "type=USER_START msg=audit(1616646601.591:134838):xxxxxxxxxx....` Sorry I'm pretty new at fluentd and learning the — Nix, Mar 25 '21 at 05:26
Sure. What kind of logs are in audit.log? Can you please share a sample? The `` section signifies the format of the incoming logs so you need to configure that accordingly. — Azeem, Mar 25 '21 at 07:00
Sample audit logs: 'type=USER_END msg=audit(1616664601.911:135103): pid=323135 uid=0 auid=0 ses=17032 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'` Also, can be found: `https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/sec-understanding_audit_log_files` — Nix, Mar 25 '21 at 09:38
Right. So, it's not JSON. You can use [none](https://docs.fluentd.org/parser/none) parser plugin to route these logs as-is and then send logs to the [stdout](https://docs.fluentd.org/output/stdout) output plugin to complete the flow. — Azeem, Mar 25 '21 at 10:00

parse_audit_log not working with td-agent

0 Answers0