5

The current setup I am working with is a Docker compose stack with multiple containers. These containers send their logging information to a logging container (inside the compose stack) running the Fluentd daemon. The configuration for Fluentd consists of one in_forward source that collects the logs and writes them to separate files, depending on the container. My Fluentd configuration file looks similar to this:

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match container1>
   @type copy
   <store>
     @type file
     path /fluentd/log/container1.*.log
     format single_value
     message_key "log"
   </store>
</match>

...

My docker-compose.yml file looks something like this:

version: '3'

services:

  container1:
    build: ./container1
    container_name: "container1" 
    depends_on:
     - "logger" 
    logging:
      driver: "fluentd"
      options:
        tag: container1  
    networks:
      static-net:
        ipv4_address: 172.28.0.4  


  ...


  logger:
    build: ./logger
    container_name: "logger"
    ports:
     - "24224:24224"
     - "24224:24224/udp"
    volumes:
     - ./logger/logs:/fluentd/log
    networks:
      static-net:
        ipv4_address: 172.28.0.5          

networks:
  static-net:
    ipam:
      driver: default
      config:
       - subnet: 172.28.0.0/16

Everything works as expected, but I would ideally like to set Fluentd to keep a certain number of log files. I can change the size of the log files by configuring the chunk_limit_size parameter in a buffer section. However, even though I want this option, I still do not want Fluentd writing an endless amount of files. The buffer_queue_limit and overflow_action in the buffer configuration do not seem to affect anything. This application will be running continuously once deployed, so log rotation is a necessity. Several questions I have:

  1. Does Fluentd support log rotation for writing logs to files? If so, what parameters do I set in the Fluentd configuration file?
  2. If not, can I configure Docker in such a way that I can utilize its log rotation of the json logging driver for Fluentd?
  3. And if that is not possible, is there a way to add log rotation into Fluentd via a plugin or perhaps in the Fluentd docker container itself (or a sidecar container)?
Rh1303
  • 51
  • 1
  • 1
  • 2

1 Answers1

4

When you are using fluentd logging driver for docker then there is no container log files, there are only fluentd logs, and to rotate them you can use this link. If you want docker to keep logs and to rotate them, then you have to change your stackfile from: 

    logging:
      driver: "fluentd"
      options:
        tag: container1

to 

  logging:
   driver: "json-file"
   options:
      max-size: "5m" #max size of log file
      max-file: "2" #how many files should docker keep

in fluentd you have to use the in_tail plugin instead of forward (fluentd should have access to log files in /var/lib/docker/containers/*/*-json.log )

<source>
  type tail
  read_from_head true
  pos_file fluentd-docker.pos
  path /var/lib/docker/containers/*/*-json.log
  tag docker.*
  format json
</source>
hichamx
  • 794
  • 3
  • 7
  • 18
  • I was thinking that was specifically for the Fluentd logs and not necessarily for container logs that it gets as input and outputs to files. I could definitely be wrong about that though. Thanks – Rh1303 Apr 03 '18 at 17:11
  • 1
    --log-rotate-age and --log-rotate-size did not do anything. It seems that fluentd ignores that. I'm using the fluent/fluentd:latest docker image and using my own fluent.conf file to configure Fluentd. There isn't anything fancy in that – Rh1303 Apr 03 '18 at 19:21
  • 1
    That is one way of doing it...I guess. Essentially I would be copying the json log files and saving them in my own destination as raw log files? It seems a little redundant to have 2 sets of log files in different places, but I guess that's what I have to do. My goal was for fluentd to handle all of the logs but be able to rotate the individual log files it outputs in the "match" sections...if that makes sense. – Rh1303 Apr 04 '18 at 13:34