Why does docker-compose issue a "No such file or directory" when the file is there?

Question

I am testing a bare metal recovery of my server which basically starts a set of docker container with my services. I recovered from a backup /etc/docker, where I keep all the configuration and persistent volumes.

I then tried to start one of the containers:

root@srv-backup:/etc/docker# docker-compose --verbose -f /etc/docker/docker-compose.d/20-registry.yaml up
compose.config.config.find: Using configuration files: /etc/docker/docker-compose.d/20-registry.yaml
ERROR: compose.cli.main.main: .IOError: [Errno 2] No such file or directory: '/etc/docker/docker-compose.d/20-registry.yaml'

The file is however there:

root@srv-backup:/# ll /etc/docker/docker-compose.d/20-registry.yaml
-rwxrwxr-x+ 1 root root 842 Jan 24 15:19 /etc/docker/docker-compose.d/20-registry.yaml*

root@srv-backup:/# cat /etc/docker/docker-compose.d/20-registry.yaml
services:
  registry:
    container_name: registry
    image: registry
    labels:
      - traefik.http.routers.registry.rule=Host(`registry.example.com`)
      - traefik.http.routers.registry.entryPoints=https
      - traefik.http.routers.registry.tls=true
      - traefik.http.routers.registry.tls.certresolver=le
      - traefik.http.middlewares.lan.ipwhitelist.sourcerange=192.168.10.0/24, 192.168.20.0/24
      - traefik.http.routers.registry.middlewares=lan
      - traefik.enable=true
    restart: unless-stopped
    volumes:
      - /etc/docker/container-data/registry:/var/lib/registry
version: '3'

root@srv-backup:/# file /etc/docker/container-data/registry
/etc/docker/container-data/registry: directory

I tried all kind of incantations with relative and full paths - the issue is the same.

I was wondering whether the docker daemon has access to the file, but it also runs as root:

root@srv-backup:/# ps -ef | grep docker
root      2048     1  0 10:58 ?        00:00:08 dockerd -G docker --exec-root=/var/snap/docker/423/run/docker --data-root=/var/snap/docker/common/var-lib-docker --pidfile=/var/snap/docker/423/run/docker.pid --config-file=/var/snap/docker/423/config/daemon.json
root      2200  2048  0 10:58 ?        00:00:07 containerd --config /var/snap/docker/423/run/docker/containerd/containerd.toml --log-level error

I am quite at loss why it does not work (the docker file works correctly on the server I am trying to replicate in this DRP exercise)

Answer 1

I found a workaround (this is not exactly an answer to the problem, but allows to go forward).

1. I uninstalled the docker snap installation: snap remove docker
2. I installed docker from the repositories: apt install docker.io
3. I installed docker-compose:

curl -L https://github.com/docker/compose/releases/download/1.21.2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

Now the command I used to start the example container works.

There must be something special with the snap version of docker.

Answer 2

It could also be as simple as it was by me: Just start docker.

The error message was a bit irritating.

Answer 3

This actually helped me

pip3 install --upgrade --force-reinstall --no-cache-dir docker-compose && ln -sf /usr/local/bin/docker-compose /usr/bin/docker-compose

Answer 4

the problem could be related to the python version
docker-compose 1.27.x runs for me with Python 3.6.x
for >= 1.28 , one would need 3.7.x
docker-compose >= 1.27 dropped support for Python 2.7

I am not aware of lower version numbers

Answer 5

Ran into this same error, but in my case it could not find the Dockerfile. And it turns out that the file path I had specified was not valid from the context path I had provided. For example, I had:

/myproject
  /myapp
  /docker
    /myapp.Dockerfile
    /docker-compose.yaml

and in docker-compose.yaml:

services:
  myapp:
  build:
    context: ..
    dockerfile: myapp.Dockerfile

and then I was running with:

cd /myproject/docker
docker compose up -d

It would complain the myapp.Dockerfile could not be found, and it turns out it was right.
The context was specified as the parent folder (..) so dockerfile property should be docker/myapp.Dockerfile.