0

I have the EFK stack and want to add nginx log parses to fluentD in Openshift. I read https://docs.fluentd.org/v1.0/articles/parser_nginx but I don't understand how to add a plugin in a daemon set. How is plugin added in fluentd to parse logs.

Danny
  • 147
  • 1
  • 13

1 Answers1

1

Thanks for attaching configmap and deployment yaml. Can you try below config.

From Parser Plugin Overview. They provided a simple example to get nginx logs.

<source>
  @type tail
  path /path/to/input/file
  <parse>
    @type nginx
    keep_time_key true
  </parse>
</source>

Can you add above config in configmap and try. The nginx parser is in-built, the link you provided is explanation of parser internals. If you don't like the way it is selecting/parsing, you can write your own.

Anyways, by using @type we can define type of parser we want.

PS: Right now I dont have any setup to test this. Also I have little knowledge in fluentd

EDIT:

I believe you already knew how fluentd is getting logs... In your pastebin link, at volumeMounts section, it mounting host's /var/lib/docker/containers which has all container meta data and logs, etc

Veerendra K
  • 2,145
  • 7
  • 32
  • 61
  • I added that , how am I supposed to view teh nginx logs parsing in kibana now? – Danny Mar 22 '19 at 09:56
  • Again I have less knowledge on kibana and it's filter rules. You can search all logs from a kubernetes node and see nginx logs. Make sure you generate traffic on nginx pod. Verify connectivity b/w fluentd daemonset and ELK. I see `host` is commented in configmap that you posted. – Veerendra K Mar 22 '19 at 10:08