2

I have added stunnel to a Redis container and PHP-FPM container to securely transfer application data between services on a docker swarm cluster. I haven't been able to find any other similar questions, so I'm wondering if I'm taking the wrong approach here.

I have this working in my local environment, it's when I deploy it to the swarm that it fails.

Problem

When I try to ping from the client container by executing redis-cli -p 8001 ping

Then I get the following error: Error: Connection reset by peer

When I take a look at the logs for stunnel I can see that it accepted the connection on the client and then fails when attempting to send it to the redis server container as seen below

2018.05.19 16:42:39 LOG5[ui]: Configuration successful
2018.05.19 16:45:19 LOG7[0]: Service [redis-client] started
2018.05.19 16:45:19 LOG5[0]: Service [redis-client] accepted connection from 127.0.0.1:41710
2018.05.19 16:45:19 LOG6[0]: s_connect: connecting 10.0.0.5:6379
2018.05.19 16:45:19 LOG7[0]: s_connect: s_poll_wait 10.0.0.5:6379: waiting 10 seconds
2018.05.19 16:45:19 LOG3[0]: s_connect: connect 10.0.0.5:6379: Connection refused (111)
2018.05.19 16:45:19 LOG5[0]: Connection reset: 0 byte(s) sent to SSL, 0 byte(s) sent to socket
2018.05.19 16:45:19 LOG7[0]: Local descriptor (FD=3) closed
2018.05.19 16:45:19 LOG7[0]: Service [redis-client] finished (0 left)

Configuration Details

Here's the stunnel configuration on the Redis server

pid = /run/stunnel-redis.pid
output = /tmp/stunnel.log

[redis-server]
cert = /etc/stunnel/redis-server.crt
key = /etc/stunnel/redis-server.key
accept = redis_master:6379
connect = 127.0.0.1:6378

And here's the stunnel configuration for the client

pid = /run/stunnel-redis.pid
output = /tmp/stunnel.log

[redis-client]
client = yes
accept = 127.0.0.1:8001
connect = redis_master:6379
CAfile = /etc/stunnel/redis-server.crt
verify = 4
debug = 7

This is what my docker-stack.yml file looks like for these two services

php_fpm:
    build:
        context: .
        dockerfile: fpm.Dockerfile
    image: registry.github.com/hidden
    ports:
        - "8001"

redis_master:
    build:
        context: .
        dockerfile: redis.Dockerfile
    image: registry.github.com/hidden
    ports:
        - "6378"
        - "6379"
    sysctls:
        - net.core.somaxconn=511
    volumes:
        - redis-data:/data

Output of netstat -plunt in the fpm client container

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:8001          0.0.0.0:*               LISTEN      208/stunnel4
tcp        0      0 127.0.0.11:45281        0.0.0.0:*               LISTEN      -
tcp6       0      0 :::9000                 :::*                    LISTEN      52/php-fpm.conf)
udp        0      0 127.0.0.11:43781        0.0.0.0:*                           -

Output of netstat -plunt in the redis server container

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.11:39294        0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:6378            0.0.0.0:*               LISTEN      8/redis-server *:63
tcp        0      0 10.0.0.14:6379          0.0.0.0:*               LISTEN      37/stunnel4
tcp6       0      0 :::6378                 :::*                    LISTEN      8/redis-server *:63
udp        0      0 127.0.0.11:44855        0.0.0.0:*                           -

I've confirmed there is no firewall active on the host machine. These services are currently on the same host, but they will soon be on separate hosts, hence the need for stunnel.

These services are deployed with the docker stack command so an overlay network is automatically created and attached to both of these services.

Anyone have any thoughts on why the request from the client to the server is being refused?

mitchcodes
  • 41
  • 3
  • Why din't you just encrypt the network? https://docs.docker.com/v17.09/engine/userguide/networking/overlay-security-model/ – Constantin Galbenu May 19 '18 at 17:29
  • Because here it mentions "This encryption imposes a non-negligible performance penalty" https://docs.docker.com/network/overlay/#encrypt-traffic-on-an-overlay-network – mitchcodes May 19 '18 at 17:50

1 Answers1

2

FINALLY got this working! I hope this helps someone else. The problem was the stunnel configuration on the redis-server, the correct configureation is as follows:

[redis-server]
cert = /etc/stunnel/redis-server.crt
key = /etc/stunnel/redis-server.key
accept = 6379
connect = 6378

The problem appears to be that I had used the hostname redis_master in the accept option, switching it to only the port fixed the problem.

mitchcodes
  • 41
  • 3