Is there any way supervisord can automatically restart a failed/exited/terminated job and send me a notification email with a dump of the last x lines of log file?

up vote 47 down vote accepted

There is a plugin called superlance.

You install it with pip install superlance or download it at: http://pypi.python.org/pypi/superlance

The next thing you do is you go into your supervisord.conf and add the following lines:

[eventlistener:crashmail]
command=/usr/local/bin/crashmail -a -m email1@example.com
events=PROCESS_STATE

This should be followed by a "supervisorctl update". When a process "exits" you will now get a notification sent to email1@example.com.

If you only want to listen to some selected apps you can exchange the -a for a -p program1 or if it is a group group1:program2 One example would be

[eventlistener:crashmail]
command=/usr/local/bin/crashmail -p program1 -p group1:program2  -m email1@example.com
events=PROCESS_STATE

Regarding the automatic restart: you should make sure that autorestart is set to true (it is set to unexpected by default). This way the package will be restarted 3 times. If after that it still exits, it gives up, but you can change that with startretries.

Example program:

[program:cat]
command=/bin/cat
autorestart=true
startretries=10
  • Despite being inactive for a few years this thread is still valid - I just tested superlance 1.0.0 (dated oct 2016) with supervisor 3.1.4 (CentOS7) and the crashmail worked just fine. – David Ramirez Oct 4 at 19:48

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.