The setup: An nginx container vhost manager which uses systemd to automatically spawn php-fpm containers when their respective vhosts are activated.
The configuration: A shared volume for sockets in /var/run. systemd-socket-proxyd used to listen on nginx's writing socket then start the fpm container, wait until the fpm container creates a listening socket, then proxy nginx's writing socket to fpm's listening socket.
nginx:/etc/nginx/conf.d/default:
try_files $uri =404;
root /usr/share/nginx/html;
fastcgi_pass unix:/var/run/docker-apps/vhost.fpm-waker.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
include /etc/nginx/fastcgi_params;
}
This tells nginx to pass php requests to /var/run/docker-apps/vhost.fpm-waker.sock. First we must ensure nginx and fpm have the same permissions:
nginx:/etc/nginx/nginx.conf:
user www-data;
Next we make sure fpm will open a unix socket where systemd expects to find it:
fpm:/etc/php5/fpm/pool.d/www.conf:
listen = /var/run/docker-apps/vhost.fpm.sock
listen.owner = www-data
listen.group = www-data
Now the configuration is ready, we'll start (but not run) our fpm container:
docker create \
--name vhost \
--ipc="container:nginx" \
-v /var/run/docker-apps:/var/run/docker-apps \
fpm
The import part here is the volume mapping which ensures an area of the FS writeable by both of the sockets we're going to use.
host:/vagrant/docker/docker-compose.yml:
nginx:
image: nginx
container_name: switchboard.noflag.org.uk
ports:
- 80:80
- 443:443
volumes:
- /var/run/docker-apps:/var/run/docker-apps
Finally, our systemd service unit descriptors on the host:
[Unit]
Description=vhost fpm container
[Service]
ExecStart=/usr/bin/docker start -a vhost
ExecStartPost=/bin/sleep 2
ExecStop=/usr/bin/docker stop vhost
This service descriptor allows systemd to take control of starting and stopping the container. Let's test it:
# systemctl start vhost.service
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cacb32d1468c fpm "/usr/sbin/php5-fpm -" 26 hours ago Up 26 minutes 9005/tcp vhost
Yes, this service descriptor works fine.
Now we tell systemd when and how to create a listening socket proxy for nginx to write to:
# more vhost.fpm-waker.socket
[Socket]
ListenStream=/var/run/docker-apps/vhost.fpm-waker.sock
SocketUser=www-data
SocketGroup=www-data
[Install]
WantedBy=sockets.target
This descriptor describes the socket systemd will open for nginx to write to. When the socket is activated, so will vhost.fpm-waker.service be.
# more vhost.fpm-waker.service
[Unit]
Requires=vhost.fpm.service
After=vhost.fpm.service
[Install]
Also=vhost.fpm-waker.socket
[Service]
ExecStart=/lib/systemd/systemd-socket-proxyd /var/run/docker-apps/vhost.fpm.sock
vhost.fpm-waker.service automatically starts vhost.service which, as we have seen, successfully starts the fpm container. Requests to the socket are proxied by /lib/systemd/systemd-socket-proxyd until fpm has made listening socket /var/run/docker-apps/vhost.fpm.sock available.
What happens if we skip systemd-socket-proxyd altogether?
# more vhost.fpm-waker.service
ExecStart=docker start -a vhost
# systemctl start vhost.fpm-waker.service
# docker logs vhost
[24-Nov-2016 13:07:06] ERROR: An another FPM instance seems to already listen on /var/run/docker-apps/vhost.fpm-waker.sock
[24-Nov-2016 13:07:06] ERROR: FPM initialization failed
So we can confirm that systemd-socket-proxyd is necessary for this process as fpm will not be happy being passed a pre-existing listening socket.
What happens when we put all the pieces together?
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2f92b7ce5e7b nginx "nginx -g 'daemon off" 28 hours ago Up About an hour 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginx
# systemctl enable vhost.fpm-waker.socket
Created symlink from /etc/systemd/system/sockets.target.wants/vhost.fpm-waker.socket to /etc/systemd/system/vhost.fpm-waker.socket.
# ls -la /var/run/docker-apps
. ..
# systemctl start vhost.fpm-waker.socket
# ls -la /var/run/docker-apps
srw-rw---- 1 www-data www-data 0 Nov 24 21:31 vhost.fpm-waker.sock=
Good. Systemd has successfully created our socket which nginx will connect to. Let's verify that nginx can talk to the socket:
# docker exec -it nginx ls -la /var/run/docker-apps
srw-rw---- 1 www-data www-data 0 Nov 24 21:31 vhost.fpm.sock=
The final step. Given an index.php file in a location available to both the nginx and fpm containers, the following should produce the text "Hello, world" from php:
# curl http://localhost:80/index.php
<p>Sorry, the page you are looking for is currently unavailable.<br/> Please try again later.</p>
<p>If you are the system administrator of this resource then you should check the <a href="http://nginx.org/r/error_log">error log</a> for details.</p>
However, it does not. The fpm container has no log entries but it is running, exactly as expected:
# docker logs vhost
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cacb32d1468c fpm "/usr/sbin/php5-fpm -" 26 hours ago Up 6 minutes 9005/tcp vhost
2f92b7ce5e7b nginx "nginx -g 'daemon off" 28 hours ago Up About an hour 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginx
And what's more, both nginx's writing socket (created by systemd) and fpm's listening socket (created by fpm) are present and correct:
# ls -la /var/run/docker-apps/
total 0
drwxr-xr-x 2 root root 60 Nov 24 21:31 ./
drwxr-xr-x 18 root root 660 Nov 24 20:06 ../
srw-rw---- 1 www-data www-data 0 Nov 24 21:31 vhost.fpm.sock=
srw-rw---- 1 www-data www-data 0 Nov 24 21:31 vhost.fpm-waker.sock=
The only clue is in the nginx logs:
docker logs nginx
172.21.0.1 - - [24/Nov/2016:20:51:04 +0000] "GET /index.php HTTP/1.1" 502 537 "-" "curl/7.38.0" "-"
2016/11/24 20:56:53 [error] 5#5: *15 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /index.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/docker-apps/vhost.fpm-waker.sock:", host: "localhost"
Unfortunately there's precious little about using unix sockets to start docker containers using systemd-socket-proxyd and even less about starting fpm containers this way. At this point, I am completely at a loss for where to go from here.
To see the full project outline, you can download it here and may be able to reproduce the problem with vagrant. However, the systemd service units are not copied into the host automatically at this time:
