Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have two identical EC2 instances (the second is a replica of the first), running Gentoo. The first instance has monit running which monitors a single process and some system resources and functions great.

In the second instance, monit runs but quits right away. The configuration is similar on both instances so are the versions of monit.

monit.log shows:

[GMT Oct  3 08:36:41] info     : monit daemon with PID 5 awakened

Final lines on strace monit show:

write(2, "monit daemon with PID 5 awakened"..., 33monit daemon with PID 5 awakened ) = 33
time(NULL)                              = 1349252827
open("/etc/localtime", O_RDONLY)        = 4
fstat64(4, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
fstat64(4, {st_mode=S_IFREG|0644, st_size=118, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb773a000
read(4, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0\0"..., 4096) = 118
_llseek(4, -6, [112], SEEK_CUR)         = 0
read(4, "\nGMT0\n", 4096)               = 6
close(4)                                = 0
munmap(0xb773a000, 4096)                = 0
write(3, "[GMT Oct  3 08:27:07] info     :"..., 33) = 33
write(3, "monit daemon with PID 5 awakened"..., 33) = 33
waitpid(-1, NULL, WNOHANG)              = -1 ECHILD (No child processes)
close(3)                                = 0
exit_group(0)                           = ?

No core dumps (ulimit -c shows unlimited)

monit -v shows:

monit: Debug: Adding host allow 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Adding credentials for user 'xxxx'.
Runtime constants:
 Control file       = /etc/monitrc
 Log file           = /var/log/monit/monit.log
 Pid file           = /var/run/monit.pid
 Id file            = /var/run/monit.pid
 Debug              = True
 Log                = True
 Use syslog         = False
 Is Daemon          = True
 Use process engine = True
 Poll time          = 30 seconds with start delay 0 seconds
 Expect buffer      = 256 bytes
 Event queue        = base directory /var/monit with 100 slots
 Mail server(s)     = xx.xxx.xx.xxx with timeout 30 seconds
 Mail from          = (not defined)
 Mail subject       = (not defined)
 Mail message       = (not defined)
 Start monit httpd  = True
 httpd bind address = Any/All
 httpd portnumber   = 2812
 httpd signature    = True
 Use ssl encryption = False
 httpd auth. style  = Basic Authentication and Host/Net allow list
 Alert mail to      = xx@xx.xx
   Alert on         = All events

The service list contains the following entries:

System Name           = xxxx
 Monitoring mode      = active
 CPU wait limit       = if greater than 20.0% 1 times within 1 cycle(s) then alert else if succeeded 1 times within 1 cycle(s) then alert
 CPU system limit     = if greater than 30.0% 1 times within 1 cycle(s) then alert else if succeeded 1 times within 1 cycle(s) then alert
 CPU user limit       = if greater than 70.0% 1 times within 1 cycle(s) then alert else if succeeded 1 times within 1 cycle(s) then alert
 Swap usage limit     = if greater than 25.0% 1 times within 1 cycle(s) then alert else if succeeded 1 times within 1 cycle(s) then alert
 Memory usage limit   = if greater than 75.0% 1 times within 1 cycle(s) then alert else if succeeded 1 times within 1 cycle(s) then alert
 Load avg. (5min)     = if greater than 2.0 1 times within 1 cycle(s) then alert else if succeeded 1 times within 1 cycle(s) then alert
 Load avg. (1min)     = if greater than 4.0 1 times within 1 cycle(s) then alert else if succeeded 1 times within 1 cycle(s) then alert

Process Name          = xxxx
 Group                = server
 Pid file             = /var/run/xxxx.pid
 Monitoring mode      = active
 Start program        = '/etc/init.d/xxxx restart' timeout 20 second(s)
 Stop program         = '/etc/init.d/xxxx stop' timeout 30 second(s)
 Existence            = if does not exist 1 times within 1 cycle(s) then restart else if succeeded 1 times within 1 cycle(s) then alert
 Pid                  = if changed 1 times within 1 cycle(s) then alert
 Ppid                 = if changed 1 times within 1 cycle(s) then alert
 Timeout              = If restarted 3 times within 5 cycle(s) then unmonitor
 Alert mail to        = xx@xx.xx
   Alert on           = All events
 Alert mail to        = xx@xx.xx
   Alert on           = All events

-------------------------------------------------------------------------------
monit daemon with PID 5 awakened

Ran emerge --sync before emerge -va monit which installed monit v5.3.2. When that didn't work i've downloaded v5.5 from their website and compiled from source which did not work either.

share|improve this question
    
what does monit in very verbose and interactive reveal? 'monit -vv -I' –  3molo Oct 3 '12 at 9:04
    
I've posted the results in my question (see the monit -v section) –  Yaniro Oct 3 '12 at 9:08

1 Answer 1

Found out what the problem was:

The file /var/run/monit.pid contained some strange information, it looked like an md5 hash for some reason.

Once i've removed this file and restarted monit, everything worked fine. What alerted my suspicion was the line monit daemon with PID 5 awakened which was very strange because a newly run process should have a much higher PID so i went checking in the .pid file.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.