0

The setup: An nginx container vhost manager which uses systemd to automatically spawn php-fpm containers when their respective vhosts are activated.

The configuration: A shared volume for sockets in /var/run. systemd-socket-proxyd used to listen on nginx's writing socket then start the fpm container, wait until the fpm container creates a listening socket, then proxy nginx's writing socket to fpm's listening socket.

nginx:/etc/nginx/conf.d/default:

     try_files $uri =404;
        root /usr/share/nginx/html;
        fastcgi_pass unix:/var/run/docker-apps/vhost.fpm-waker.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
        include /etc/nginx/fastcgi_params;
     }

This tells nginx to pass php requests to /var/run/docker-apps/vhost.fpm-waker.sock. First we must ensure nginx and fpm have the same permissions:

nginx:/etc/nginx/nginx.conf:
    user  www-data;

Next we make sure fpm will open a unix socket where systemd expects to find it:

fpm:/etc/php5/fpm/pool.d/www.conf:
     listen = /var/run/docker-apps/vhost.fpm.sock
     listen.owner = www-data
     listen.group = www-data

Now the configuration is ready, we'll start (but not run) our fpm container:

docker create \
--name vhost \
--ipc="container:nginx" \
-v /var/run/docker-apps:/var/run/docker-apps \
fpm

The import part here is the volume mapping which ensures an area of the FS writeable by both of the sockets we're going to use.

host:/vagrant/docker/docker-compose.yml:

 nginx:
  image: nginx
  container_name: switchboard.noflag.org.uk
  ports:
    - 80:80
    - 443:443
  volumes:
    - /var/run/docker-apps:/var/run/docker-apps

Finally, our systemd service unit descriptors on the host:

[Unit]
Description=vhost fpm container

[Service]
ExecStart=/usr/bin/docker start -a vhost
ExecStartPost=/bin/sleep 2

ExecStop=/usr/bin/docker stop vhost

This service descriptor allows systemd to take control of starting and stopping the container. Let's test it:

# systemctl start vhost.service
# docker ps

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                      NAMES
cacb32d1468c        fpm                 "/usr/sbin/php5-fpm -"   26 hours ago        Up 26 minutes       9005/tcp                                   vhost

Yes, this service descriptor works fine.

Now we tell systemd when and how to create a listening socket proxy for nginx to write to:

 # more vhost.fpm-waker.socket 
[Socket]
ListenStream=/var/run/docker-apps/vhost.fpm-waker.sock
SocketUser=www-data
SocketGroup=www-data

[Install]
WantedBy=sockets.target

This descriptor describes the socket systemd will open for nginx to write to. When the socket is activated, so will vhost.fpm-waker.service be.

# more vhost.fpm-waker.service 
[Unit]
Requires=vhost.fpm.service
After=vhost.fpm.service

[Install]
Also=vhost.fpm-waker.socket

[Service]
ExecStart=/lib/systemd/systemd-socket-proxyd /var/run/docker-apps/vhost.fpm.sock

vhost.fpm-waker.service automatically starts vhost.service which, as we have seen, successfully starts the fpm container. Requests to the socket are proxied by /lib/systemd/systemd-socket-proxyd until fpm has made listening socket /var/run/docker-apps/vhost.fpm.sock available.

What happens if we skip systemd-socket-proxyd altogether?

# more vhost.fpm-waker.service
ExecStart=docker start -a vhost
# systemctl start vhost.fpm-waker.service
# docker logs vhost
[24-Nov-2016 13:07:06] ERROR: An another FPM instance seems to already listen on /var/run/docker-apps/vhost.fpm-waker.sock 
[24-Nov-2016 13:07:06] ERROR: FPM initialization failed

So we can confirm that systemd-socket-proxyd is necessary for this process as fpm will not be happy being passed a pre-existing listening socket.

What happens when we put all the pieces together?

# docker ps
CONTAINER ID        IMAGE               COMMAND                 CREATED             STATUS              PORTS                                      NAMES 
2f92b7ce5e7b        nginx               "nginx -g 'daemon off"   28 hours ago        Up About an hour    0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp       nginx
# systemctl enable vhost.fpm-waker.socket 
Created symlink from /etc/systemd/system/sockets.target.wants/vhost.fpm-waker.socket to /etc/systemd/system/vhost.fpm-waker.socket.
# ls -la /var/run/docker-apps
. ..
# systemctl start vhost.fpm-waker.socket
# ls -la /var/run/docker-apps
srw-rw----  1 www-data www-data   0 Nov 24 21:31 vhost.fpm-waker.sock=

Good. Systemd has successfully created our socket which nginx will connect to. Let's verify that nginx can talk to the socket:

# docker exec -it nginx ls -la /var/run/docker-apps
srw-rw----  1 www-data www-data   0 Nov 24 21:31 vhost.fpm.sock=

The final step. Given an index.php file in a location available to both the nginx and fpm containers, the following should produce the text "Hello, world" from php:

# curl http://localhost:80/index.php
<p>Sorry, the page you are looking for is currently unavailable.<br/> Please try again later.</p>
<p>If you are the system administrator of this resource then you should check the <a href="http://nginx.org/r/error_log">error log</a> for details.</p>

However, it does not. The fpm container has no log entries but it is running, exactly as expected:

# docker logs vhost
# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                      NAMES
cacb32d1468c        fpm                 "/usr/sbin/php5-fpm -"   26 hours ago        Up 6 minutes        9005/tcp                                   vhost
2f92b7ce5e7b        nginx               "nginx -g 'daemon off"   28 hours ago        Up About an hour    0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   nginx

And what's more, both nginx's writing socket (created by systemd) and fpm's listening socket (created by fpm) are present and correct:

# ls -la /var/run/docker-apps/
total 0
drwxr-xr-x  2 root     root      60 Nov 24 21:31 ./
drwxr-xr-x 18 root     root     660 Nov 24 20:06 ../
srw-rw----  1 www-data www-data   0 Nov 24 21:31 vhost.fpm.sock=
srw-rw----  1 www-data www-data   0 Nov 24 21:31 vhost.fpm-waker.sock=

The only clue is in the nginx logs:

docker logs nginx
172.21.0.1 - - [24/Nov/2016:20:51:04 +0000] "GET /index.php HTTP/1.1" 502 537 "-" "curl/7.38.0" "-"
2016/11/24 20:56:53 [error] 5#5: *15 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 172.21.0.1, server: localhost, request: "GET /index.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/docker-apps/vhost.fpm-waker.sock:", host: "localhost"

Unfortunately there's precious little about using unix sockets to start docker containers using systemd-socket-proxyd and even less about starting fpm containers this way. At this point, I am completely at a loss for where to go from here.

To see the full project outline, you can download it here and may be able to reproduce the problem with vagrant. However, the systemd service units are not copied into the host automatically at this time:

https://github.com/djcf/wordpress-fpm-docker

  • To me this looks like a bad idea performance-wise. Why cannot you just start PHP-FPM in a container directly and let nginx connect to it via a socket without the daemon starting PHP-FPM? Maybe the daemon doesn't even support Unix sockets... – Tero Kilkanen Nov 25 '16 at 7:13
  • fpm does support unix sockets, as you can see the above. The reason is because on a multi-host system we don't want x number of active LEMP instances per x sites since the RAM allocation would increase directly with the number of sites. But we do want the security and administration benefits of containerization. – DMCoding Nov 25 '16 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.