I have to following problem: We have an Java application which is started by an bash script. This application should run as a daemon, so we have an upstart-job to start it.
start on runlevel [2345]
stop on runlevel [!2345]
#tell upstart we will fork later, so it will mangage the pids.
expect fork
#If the daemon stoppes unexpectedly, restart it!
respawn
script
#The framework will only work, if we start it from this directory.
cd /usr/lib/app-dir
nohup ./appStartScript.sh &> /dev/null &
#send an upstart event, in case we will chain this job later
emit app_running
end script
Sometimes, the application stops working. There is neither a .hprof file, nor a hserr file which is usally created if the VM crashes. Upstart reporst the application as running,
appDeamon start/running, process 1131
But the PID is not listed in ps -aux. (Also, upstart is not able to stop the process with stop appDeamon.)
I'd like to know: a) Why doesn't upstart recognize that the application has crashed? b) Is there a possibility to force upstart to restart the application, even if the process with the given pid is not longer present? (Up to now, we need to restart the whole server.)
Our system is Ubuntu Linux 10.04.1 LTS.