3

I have a docker container on ubuntu 16.04 and I want to run a docker command periodically. After checking the command runs successfully from the command line I've setup a crontab with the following:

*/1 * * * * sudo docker run {image-name} python3 /{path-to-scrip}/script.py

This seems to work ok until the script outputs something after which point the crons don't run and it stops docker from working with the following message:

docker: Error response from daemon: connection error: desc = "transport: dial unix /var/run/docker/containerd/docke
r-containerd.sock: connect: connection refused".

I've read that stdout may be the problem and I've tried the following in crontab:

*/1 * * * * sudo docker run {image-name} python3 /{path-to-scrip}/script.py > /home/logs 2>&1

This didn't fix the problem and the logs file remains empty.

This is running on a virtual machine and restarting the instance gets me back to square one where docker runs.

I'm not the most experienced sys admin and suspect I've made a naive mistake, but I don't know how to progress this? What should I have done to setup the cron?

Additional info

I'm not sure if it's relevant, but this is running on a google cloud engine instance and the firewall is set to prohibit all external connections.

Update

After a hunch I've got it running on CentOS7, the cron job worked for a number of hours, but then the script gave an error. The error is something to be expected from the script, but this has lead to the same docker message and cron no longer works.

Further update From this thread I decided to run docker inspect --format '{{json .State }}' {container id for failed run} - this shows the following:

    {"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":8204,"ExitCode":0,"Error":"","StartedAt":"2018-01-02T03:1
5:03.016836367Z","FinishedAt":"0001-01-01T00:00:00Z"}

I guess I just need to figure out how to stop the process from running in order to shut the container down.

  • So, what do you get in /home/logs? – tripleee Jan 1 '18 at 13:06
  • Hi @tripleee - /home/logs is empty – goose Jan 1 '18 at 13:32
  • The error message looks like the Docker service dies. But as a first step, maybe use docker run --rm -it so you can actually capture its output. (The --rm is unrelated, but probably a very good idea. You may find that dcker ps -a reveals a great number of hosed copies for you to clean up.) – tripleee Jan 1 '18 at 13:53
  • 1
    Adding your user to the docker group so you don't need sudo to run simple Docker commands will probably also be a very good idea. – tripleee Jan 1 '18 at 13:55
  • @tripleee turns out I hadn't picked a directory I could write to, the logs now show "the input device is not a TTY". From advice from this thread (stackoverflow.com/questions/43099116/…) I tried --rm -i, but the problem has now come back whereas with the previous settings it seemed to run the command at least once, and although it stopped the connection refused message didn't come back – goose Jan 1 '18 at 16:48
1

I'm not putting this here as the best possible answer, if I could mark this as the correct answer I clearly wouldn't, however for anyone coming across something similar I found that the docker image wasn't very fault tolerant, even when I tried on CentOS 7. The first thing I needed to do was to liberally include try/catch blocks in my script.

Secondly, to unblock docker I created the following bash script:

#!/bin/bash
sudo service docker stop
sudo service docker start
sudo docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

This enabled docker to continue to run, I have no idea why this combination of commands was needed in my case. I may add a cron command for this script if I see the same error again in the future.

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .