0

I am setting up an EFK stack which works as expected but came with a few drawbacks that I would like to work around.

  1. If my fluentd instance is not running, other containers using that use the fluetnd log driver will fail to start. This is a pretty big concern since the main application is now dependent on my log aggregator. 
ERROR: for sharedcontainers_mongo_s3_backup_1  Cannot start service mongo_s3_backup: b'failed to initialize logging driver: dial tcp [::1]:24224: connect: connection refused'
  1. When using the fluentd logging driver, I am no longer able to view the logs in the command-line when running docker-compose logs. While this is okay more times than not, there could be an issue with the log aggregator (maybe a situation which broke all the containers) and you are now unable to easily view your logs. Are you able to use multiple logging drivers or another workaround?
redis_1               | WARNING: no logs are available with the 'fluentd' log driver

docker-compose.yml:

logging:
  driver: "fluentd"
  options:
    fluentd-address: localhost:24224
leeman24
  • 2,729
  • 3
  • 29
  • 42

1 Answers1

2

In docker community version you have only few logging drivers that support 'dual logging': local, json-file, journald.

So, there are several options:

  1. Use journald log driver for dockers, this way you can see the logs using docker logs command and also using journalctl. In order to forward the logs to fluentd, we configure syslog on the host to forward all the logs to fluentd port.

  2. Use docker Enterprise version. This version provides 'dual' logging for other drivers including fluentd log driver.

Check this link for more details docs.docker.com/config/containers/logging/configure

vmax33
  • 643
  • 5
  • 13
  • I don't have any requirements to use journalctl and don't love this approach although it is better than having the issues I mentioned in my post. I'll keep this in mind. – leeman24 Jun 17 '19 at 19:24
  • 1
    In docker community version you have only few logging drivers that support 'dual logging': local, json-file, journald. If you want 'dual' logging for other drivers including fluentd, user docker Enterprise version. – vmax33 Jun 18 '19 at 07:44