Is there any way
supervisord can automatically restart a failed/exited/terminated job and send me a notification email with a dump of the last x lines of log file?
There is a plugin called superlance.
You install it with
pip install superlance or download it at: http://pypi.python.org/pypi/superlance
The next thing you do is you go into your
supervisord.conf and add the following lines:
[eventlistener:crashmail] command=/usr/local/bin/crashmail -a -m firstname.lastname@example.org events=PROCESS_STATE
This should be followed by a "supervisorctl update". When a process "exits" you will now get a notification sent to email@example.com.
If you only want to listen to some selected apps you can exchange the
-a for a
-p program1 or if it is a group
group1:program2 One example would be
[eventlistener:crashmail] command=/usr/local/bin/crashmail -p program1 -p group1:program2 -m firstname.lastname@example.org events=PROCESS_STATE
Regarding the automatic restart:
you should make sure that
autorestart is set to
true (it is set to
unexpected by default). This way the package will be restarted 3 times. If after that it still exits, it gives up, but you can change that with
[program:cat] command=/bin/cat autorestart=true startretries=10
I tried installing superlance and running crashmail like this:
sudo apt-get install python-pip sudo pip install superlance
after i do:
sudo nano /etc/supervisor/supervisord.conf
and after i added:
[eventlistener:crashmail] command=/usr/local/bin/crashmail -a -m email@example.com events=PROCESS_STATE
and I do not receive anything....
My crashmail file is:
#!/usr/bin/python -- coding: utf-8 -- import re import sys from superlance.crashmail import main if name == 'main': sys.argv = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv) sys.exit(main())