16

I run docker container with supervisor like this:

Dockerfile

CMD ["/run.sh"]

run.sh

#!/usr/bin/env bash
exec supervisord -n

supervisor-serf.conf

[group:job]
programs=serf,producer

[program:serf]
command=/start-serf-agent.sh
numprocs=1
autostart=true
autorestart=unexpected
stopasgroup=true
killasgroup=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

start-serf-agent.sh

#!/bin/bash
exec serf agent --join=serf:7946 -tag role=producer

supervisor-servce.conf

[program:producer]
command=/start.sh
numprocs=1
stopasgroup=true
killasgroup=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

start.sh

#!/bin/bash
exec /producer --project=${NAME}

After producer stopped I've got:

producer_1 |     2016/02/29 21:59:50 [INFO] serf: EventMemberLeave: 7c4fbc80af97 172.19.0.2
producer_1 | 2016/02/29 21:59:51 INF    1 stopping
producer_1 | 2016/02/29 21:59:51 INF    1 exiting router
producer_1 | 2016-02-29 21:59:51,281 INFO exited: producer (exit status 0; expected)
producer_1 |     2016/02/29 21:59:51 [INFO] agent: Received event: member-leave

but serf-agent keep container in running state. I want to stop Docker container when producer complete his work properly with status 0. I tried to join processes to one group but seems doesn't work. Guys, what did I skip? Help me please!

1

5 Answers 5

10

I resolved issue with supervisor eventlistener:

[program:worker]
command=/start.sh
priority=2
process_name=worker
numprocs=1
stopasgroup=true
killasgroup=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[eventlistener:worker_exit]
command=/kill.py
process_name=worker
events=PROCESS_STATE_EXITED

kill.py

#!/usr/bin/env python
import sys
import os
import signal

def write_stdout(s):
   sys.stdout.write(s)
   sys.stdout.flush()
def write_stderr(s):
   sys.stderr.write(s)
   sys.stderr.flush()
def main():
   while 1:
       write_stdout('READY\n')
       line = sys.stdin.readline()
       write_stdout('This line kills supervisor: ' + line);
       try:
               pidfile = open('/var/run/supervisord.pid','r')
               pid = int(pidfile.readline());
               os.kill(pid, signal.SIGQUIT)
       except Exception as e:
               write_stdout('Could not kill supervisor: ' + e.strerror + '\n')
       write_stdout('RESULT 2\nOK')
if __name__ == '__main__':
   main()
   import sys
main issue I forgot to point to **process_name**

Also good article process management in docker containers

2
  • What's the content of the kill.py script? Feb 13, 2017 at 10:48
  • @PiotrKieszczyński post has been updated, look at, please Feb 13, 2017 at 13:44
6

Here's a slightly more streamlined version which makes use of a shell script instead of a python script, and also covers multiple services, killing the whole of supervisor if either fails.

supervisord.conf
$ cat /etc/supervisord.conf
[supervisord]
nodaemon=true
loglevel=debug
logfile=/var/log/supervisor/supervisord.log
pidfile=/var/run/supervisord.pid
childlogdir=/var/log/supervisor

[program:service1]
command=/usr/sbin/service1
user=someone
autostart=true
autorestart=true
startsecs=30
process_name=service1

[program:service2]
command=/usr/sbin/service2
user=root
autostart=true
autorestart=true
startsecs=30
process_name=service2

[eventlistener:processes]
command=stop-supervisor.sh
events=PROCESS_STATE_STOPPED, PROCESS_STATE_EXITED, PROCESS_STATE_FATAL
stop-supervisor.sh
$ cat stop-supervisor.sh
#!/bin/bash

printf "READY\n";

while read line; do
  echo "Processing Event: $line" >&2;
  kill -3 $(cat "/var/run/supervisord.pid")
done < /dev/stdin

References

3
  • 1
    Didn't you forget the eventlsteners in the config here? And your config has no reference to stop-supervisor.sh
    – rfay
    Sep 29, 2018 at 13:55
  • where is stop-supervisor.sh triggered?
    – Pieter
    Jan 8, 2019 at 22:53
  • @Pieter - thanks, it got chopped when I pasted it in, I've fixed it.
    – slm
    Jan 9, 2019 at 3:44
4

For those who don't want a separate file.

[supervisord]
loglevel=warn
nodaemon=true

[program:hi]
command=bash -c "echo waiting 5 seconds . . . && sleep 5"
autorestart=false
numprocs=1
startsecs=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0

[eventlistener:processes]
command=bash -c "printf 'READY\n' && while read line; do kill -SIGQUIT $PPID; done < /dev/stdin"
events=PROCESS_STATE_STOPPED,PROCESS_STATE_EXITED,PROCESS_STATE_FATAL
1
  • I confirm this works. Would you please explain how -- why read is appropriate? FWIW, I chose to add the config line startretries=1 (default is 3) to the [program:foo] entries in my supervisord.conf file because retrying with a bad configuration like an invalid password rarely ends well :) May 5, 2022 at 12:03
2

Here's a simple solution for Docker. In your supervisord.conf, replace this:

[program:something]
command = something

with this:

[program:something]
command = sh -c 'something && kill 1'
0

If we want to stop supervisor when some specific services crash but maintain it running in other cases, we can use something like this:

[program:primary_required]
command=sh -c '/path/to/app start || supervisorctl shutdown'

[program:secondary_required_fire_and_forget]
command=sh -c '/path/to/app migrate || supervisorctl shutdown'
exitcodes=0

[program:non_required_service]
command=service non_required start

Using this approach we can use supervisor as Docker entrypoint and raise error on docker start when something required fails, this is useful if we are using services like Elastic Beanstalk, Kubernetes, etc that depends on container status to know if everything is OK.

This also ensure that all services already started will do a gracefull shutdown instead of a kill and without any orphaned process.

Maybe a non intentional behaviour of this approach is that when a service fails it shutdown the supervisor before allow a retry, so retries does not work. In my case this is not a problem.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .