3

this morning I woke up to a 'service down' email I have received from watchdog. After connecting to the server I have found the docker service down. After little investigation, I have found that this morning Ubuntu made an unattended upgrade of the containerd service and after the upgrade, docker service was not started automatically.

How can I make sure that after the upgrade this will not happen again?

Improve this question
1

3 Answers 3

Reset to default
2

We had the same issue. I opted for using a systemd unit file to create a loose dependency between containerd and dockerd.

Here's how:

As root:

Make the override directory (permission 755)

mkdir /etc/systemd/system/containerd.service.d/

Create the override file in the above directory (permissions 644)

vi override.conf

Add the following to the file

[Unit]
Before=docker.service
Wants=docker.service

Restart systemd daemon

systemctl daemon-reload

Check it works

Stop containerd

systemctl stop containerd

Check dockerd has stopped

systemctl status docker

Start containerd

systemctl start containerd

Check dockerd has started

systemctl status docker

++++++++++++++++++++++++++

Some links to other documentation on the subject

https://www.shellhacks.com/systemd-service-file-example/

https://man7.org/linux/man-pages/man5/systemd.service.5.html

Improve this answer
2

I set Docker and its related packages to hold so they are only upgraded on planned upgrades.

sudo apt-mark hold docker containerd

When I want to upgrade Docker I run apt-mark unhold, upgrade the packages, and hold them again. I have an Ansible playbook to automate this process. My playbook to install and configure Docker sets the packages to hold directly after the installation.

Improve this answer
3
  • Thanks for this. This acutally answers the question. The containerd upgrade fiasko was just an example of something that can happen.
    – Krystian
    Dec 1, 2020 at 12:55
  • is your ansible playbook available somewhere, like ansible-galaxy, or whatever?
    – MaxiReglisse
    Dec 7, 2020 at 15:49
  • No, it isn't, sorry. But it's pretty easy, you only need the dpkg_selections module for hold.
    – Gerald Schneider
    Dec 8, 2020 at 6:59
0

The documentation on docker restart policies is helpful, start your containers with the always or unless-stopped options as appropriate for your use.

It also seems reasonable to stop Ubuntu from carrying out unattended upgrades.

Improve this answer
3
  • That doesn't help in this case. I had the same problem this morning, after the upgrade of containerd the docker systemd unit was in the state disabled.
    – Gerald Schneider
    Dec 1, 2020 at 7:38
  • That's a bug in the upgrade - it should be reported.
    – user602730
    Dec 1, 2020 at 7:39
  • It already is
    – Gerald Schneider
    Dec 1, 2020 at 7:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .