Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is there any way supervisord can automatically restart a failed/exited/terminated job and send me a notification email with a dump of the last x lines of log file?

share|improve this question

1 Answer 1

up vote 24 down vote accepted

There is a plugin called superlance.

You install it with "easy_install superlance" or download it at: http://pypi.python.org/pypi/superlance

The next thing you do is you go into your supervisord.conf and add the following lines:

[eventlistener:crashmail]
command=/usr/local/bin/crashmail -a -m email1@example.com
events=PROCESS_STATE

This should be followed by a "supervisorctl update". When a process "exits" you will now get a notification sent to email1@example.com.

If you only want to listen to some selected apps you can exchange the -a for a -p program1 or if it is a group group1:program2 One example would be

[eventlistener:crashmail]
command=/usr/local/bin/crashmail -p program1 -p group1:program2  -m email1@example.com
events=PROCESS_STATE

Regarding the automatic restart: you should make sure that autorestart is set to true (it is set to unexpected by default). This way the package will be restarted 3 times. If after that it still exists it gives up but you can change that with startretries.

Example program:

[program:cat]
command=/bin/cat
autorestart=true
startretries=10
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.