Using Docker Swarm to build an reverse proxy network

Question

I have a few servers. Each of them has a lot of docker-compose projects for HTTPS Projects which use one auto discovering reverse proxy for HTTPS.

A few days ago I discovered on youtube, that swarm is totally easy to setup and would give me a lot of possibilities to move containers between those maschines without a huge down time.

Now to my two problems:

1.) Syncing configs and letsencrypt certificates

I created a configuration:

version: '3.2'

services:
  nginx:
    image: myFancyImage
    networks:
      - web
    restart: always
    deploy:
      mode: global
    ports:
      - target: 80
        published: 81 # for testing
        protocol: tcp
        mode: ingress
      - target: 443
        published: 444 # for testing
        protocol: tcp
        mode: ingress
    volumes:
      - mainnginx-www:/var/www
      - mainnginx-letsencrypt:/etc/letsencrypt
      - mainnginx-sites-enabled:/etc/nginx/sites-enabled/

volumes:
  mainnginx-www:
  mainnginx-letsencrypt:
  mainnginx-sites-enabled:

networks:
  web:
    external:
      name: web1

And deployed the service via:

docker stack deploy --compose-file docker-compose.yml mainnginx

Now on both maschines an instance of my reverse proxy ran and mounted three volumes, but they are only local not shared.

Of course I am able to use NFS to sync them but what is best practice here? NFS looks like unclean. I read somewhere, that docker is mounting from the manager, but this is not what happens...

How did you solve this?

2.) Using a swarm network to work with local containers

I created a network for my mainproxy:

docker network create --opt encrypted --attachable --driver overlay web1

and added one of my local projects from server1 to the network like shown in the presentation.

I then tried to connect from my server1 reverse proxy and from my server2 reverse proxy to ping my projects container and was only able to do this from server1. Server2 discovered the correct IP, but neither pinged nor ponged. It just waited.

What did I do wrong? Was that not the idea in total?

Answer 1

Related to your first problem. While Swarm is indeed easy to setup and allows you to create replicas of your containers and more stuff, volume sharing is not in the batteries included with Swarm. You're right, volumes are not mounted in the manager. Each container will mount a volume on each worker host where it runs, and those volumes are not shared in Swarm.

You should have a look on Docker docs about volume plugins. From the docs, "a volume plugin might enable Docker volumes to persist across multiple Docker hosts". So if in your case you want to share the same volumes between your swarm hosts then you'll have to choose a volume plugin that best fits your environment selecting a plugin from the list.

The alternative to volume plugins as you mentioned could be of course data sharing with NFS, GlusterFS or Ceph where the worker nodes in the Swarm should share the mount point of the volume. I'd recommend you reading this article about volume persistence and volume sharing, while it's dated and not directly related with swarm it has valuable info and talks about the two strategies mentioned: volume plugin and data sharing. Note that in the article Flocker is mentioned, but Flocker was discontinued although it was forked here https://github.com/ScatterHQ/flocker). As ServerFault is not opinion-based I don't include my preferences, I just mention the existing strategies for your problem.

About your second problem. Swarm indeed allows you to interconnect containers located in different worker hosts thanks to the overlay network. I use load balancers and reverse proxy that connects with other containers flawlessly. You create the network in one of your Swarm managers and your worker hosts will be modified so the same network is created and firewall rules are applied. If you're having problems I recommend you to follow the Swarm tutorial so you can see it working or detect a problem in your setup. I use it for troubleshooting.