How to implement Fail2Ban inside a Docker container running Nginx

Question

I am trying to implement fail2ban inside my docker environment that uses my Nginx logs. <-- I may in the future just target the entire traefik logs.

version: '3'
services:
  fail2ban:
    image: 'crazymax/fail2ban:latest'
    restart: 'always'
    network_mode: 'host'
    cap_add:
      - 'NET_ADMIN'
      - 'NET_RAW'
    volumes:
      - 'nginx-log:/var/log:ro'
      - 'fail2ban-data:/data'
    env_file:
      - './fail2ban.env'
  laravel-mysql:
    [SNIP]
  laravel-php:
    [SNIP]
  laravel-nginx:
    image: 'nginx:alpine'
    restart: 'always'
    depends_on:
      - 'laravel-php'
    expose:
      - '80'
    volumes:
      - 'laravel-src:/var/www/html'
      - './nginx.conf:/etc/nginx/conf.d/default.conf'
      - 'nginx-log:/var/log/nginx'
    networks:
      - 'traefik'
      - 'laravel'
    labels:
      - 'traefik.enable=true'
      - 'traefik.docker.network=traefik'
      - 'traefik.http.routers.nginx.entrypoints=http'
      - 'traefik.http.routers.nginx.rule=Host(`${DOMAIN}`) || Host(`www.${DOMAIN}`)'
      - 'traefik.http.routers.nginx.middlewares=redirect@file'
      - 'traefik.http.routers.nginx-https.rule=Host(`${DOMAIN}`) || Host(`www.${DOMAIN}`)'
      - 'traefik.http.routers.nginx-https.tls=true'
      - 'traefik.http.routers.nginx-https.tls.certresolver=${DNS_PROVIDER}'
      - 'traefik.http.routers.nginx-https.tls.domains[0].main=${DOMAIN}'
      - 'traefik.http.routers.nginx-https.tls.domains[1].main=www.${DOMAIN}'
      - 'traefik.http.routers.nginx.service=nginx'
      - 'traefik.http.services.nginx.loadbalancer.server.port=80'
      - 'traefik.http.services.nginx.loadBalancer.passHostHeader=true'
      - 'traefik.http.middlewares.https_redirect.redirectscheme.scheme=https'
      - 'traefik.http.middlewares.https-redirect.redirectscheme.scheme=https'
      - 'traefik.http.middlewares.https-redirect.headers.customrequestheaders.X-Forwarded-Proto=https'
      - 'traefik.http.routers.nginx.middlewares=https-redirect'
      - 'traefik.http.middlewares.https_redirect.redirectscheme.permanent=true'
      - 'traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)'
      - 'traefik.http.routers.http_catchall.entrypoints=http'
      - 'traefik.http.routers.http_catchall.middlewares=https_redirect'
networks:
  laravel:
    driver: 'bridge'
  traefik:
    name: '${TRAEFIK_NETWORK}'
    external: 'true'
volumes:
  laravel-database:
    driver: 'local'
  laravel-src:
    driver: 'local'
  nginx-log:
    driver: 'local'
  fail2ban-data:
    driver: 'local'

Running docker logs laravel_fail2ban_1 --tail 100 after docker-compose up -d shows me:

Setting timezone to Europe/London...
Setting SSMTP configuration...
WARNING: SSMTP_HOST must be defined if you want fail2ban to send emails
Initializing files and folders...
Setting Fail2ban configuration...
Checking for custom actions in /data/action.d...
Checking for custom filters in /data/filter.d...
2021-08-01 11:40:13,199 fail2ban.configreader   [1]: INFO    Loading configs for fail2ban under /etc/fail2ban
2021-08-01 11:40:13,202 fail2ban.configparserinc[1]: INFO      Loading files: ['/etc/fail2ban/fail2ban.conf']
2021-08-01 11:40:13,203 fail2ban.configparserinc[1]: INFO      Loading files: ['/etc/fail2ban/fail2ban.conf']
2021-08-01 11:40:13,204 fail2ban                [1]: INFO    Using socket file /var/run/fail2ban/fail2ban.sock
2021-08-01 11:40:13,204 fail2ban                [1]: INFO    Using pid file /var/run/fail2ban/fail2ban.pid, [INFO] logging to STDOUT
2021-08-01 11:40:13,218 fail2ban.configreader   [1]: INFO    Loading configs for jail under /etc/fail2ban
2021-08-01 11:40:13,219 fail2ban.configparserinc[1]: INFO      Loading files: ['/etc/fail2ban/jail.conf']
2021-08-01 11:40:13,255 fail2ban.configparserinc[1]: INFO      Loading files: ['/etc/fail2ban/paths-debian.conf']
2021-08-01 11:40:13,257 fail2ban.configparserinc[1]: INFO      Loading files: ['/etc/fail2ban/paths-common.conf']
2021-08-01 11:40:13,260 fail2ban.configparserinc[1]: INFO      Loading files: ['/etc/fail2ban/paths-overrides.local']
2021-08-01 11:40:13,263 fail2ban.configparserinc[1]: INFO      Loading files: ['/etc/fail2ban/paths-common.conf', '/etc/fail2ban/paths-debian.conf', '/etc/fail2ban/jail.conf']
2021-08-01 11:40:13,369 fail2ban.server         [1]: INFO    --------------------------------------------------
2021-08-01 11:40:13,372 fail2ban.server         [1]: INFO    Starting Fail2ban v0.11.2
2021-08-01 11:40:13,373 fail2ban.observer       [1]: INFO    Observer start...
2021-08-01 11:40:13,382 fail2ban.database       [1]: INFO    Connected to fail2ban persistent database '/data/db/fail2ban.sqlite3'
2021-08-01 11:40:13,385 fail2ban.database       [1]: WARNING New database created. Version '4'
Server ready

If I now attempt to stress my application, no logs are populated in fail2ban but if I --follow my nginx container logs, I see the requests firing away.

If I docker exec -it -u root laravel_fail2ban_1 /bin/bash -c 'ls -la /var/log' I can see my logs in the correct location:

total 8
drwxr-xr-x    2 root     root          4096 Aug  1 11:35 .
drwxr-xr-x    1 root     root          4096 Dec 16  2020 ..
lrwxrwxrwx    1 root     root            11 Jul  6 20:40 access.log -> /dev/stdout
lrwxrwxrwx    1 root     root            11 Jul  6 20:40 error.log -> /dev/stderr

I see the issue may be when I try cat /var/log/access.log. It is symlinked to the /dev/stdout which means the terminal tries attaching to it. I cannot unlink it whilst running:

docker exec -it -u root laravel_fail2ban_1 /bin/bash -c 'unlink /var/log/access.log'
unlink: can't remove file '/var/log/access.log': Read-only file system

Any help appreciated to get this working. I need to keep the symlink on the volume so I can use docker logs on my nginx container.

score 2 · Accepted Answer · answered Aug 01 '21 at 12:41

2

If /var/log/access.log is a symlink to stdout, it's not going to be available in the other container: /dev/stdout points to the stdout of the current process, so when fail2ban attempts to read from it, it gets its own stdout, rather than the stdout of the nginx process.

If you want fail2ban to be able to read the logs from nginx, you will need to write them to an actual file. If you also want them showing up on the container stdout, you can run something like a tail -f in the background of the nginx container.

answered Aug 01 '21 at 12:41

larsks

277,717
41
399
399

I take it this action needs to be done outside the container then? Thanks for the explanation, I guessed it would be an issue with the Stdout because when I tried to cat it, it showed nothing and attached to the process – Jaquarh Aug 01 '21 at 13:23
Well, no, it doesn't need to be done outside the container. I was describing how you would solve it *inside* a container: write nginx logs to a file inside the nginx container, mount the log directory inside the fail2ban container, etc. – larsks Aug 01 '21 at 13:27
I see, I could do this in the nginx config right? The as you say, tail f it for the output. Or do I need to do this another way? – Jaquarh Aug 01 '21 at 14:45
https://stackoverflow.com/a/60429415/5897602 Something like this perhaps? – Jaquarh Aug 01 '21 at 14:55
This worked. Removing the symlink now allows me to cat the access log inside the fail2ban container in the `/var/log` directory. No mention of the STDOUT logs of the fail2ban container of the log being used --> how can I verify this is now working do you know? – Jaquarh Aug 01 '21 at 16:12
I guess do something that would trigger your `fail2ban` rules and see if it works? I'm not fall that familiar with the `fail2ban` side of things. – larsks Aug 01 '21 at 22:47
I managed to fix this doing what you said. ```unlink "`docker volume inspect --format '{{ .Mountpoint }}' laravel_nginx-log`/access.log" && docker-compose up -d --force-recreate``` and then I just added a cmd for `tail -f` in the nginx container inside the `docker-compose.yml`. – Jaquarh Aug 01 '21 at 22:52

How to implement Fail2Ban inside a Docker container running Nginx

1 Answers1