0

I have been struggling to get monit to monitor my thin cluster for some time now. I am using 3 thin instances. These are my monit and thin config files:

/etc/thin/vocab.yml:
pid: tmp/pids/thin.pid
log: log/thin.log
port: 3000
max_conns: 1024
timeout: 30
chdir: /home/tenjack/apps/Vocab/current
max_persistent_conns: 512
environment: production
servers: 3
address: 0.0.0.0
daemonize: true
require: [] 

thin.monitrc:
check process thin0
  with pidfile tmp/pid/thin.3000.pid
  start program = "/etc/init.d/thin start"
  stop program = "/etc/init.d/thin stop"
  if totalmem > 50.0 MB for 5 cycles then restart
  if failed port 3000 then restart
  if cpu usage > 95% for 3 cycles then restart
  if 5 restarts within 5 cycles then timeout
group thin

I have also tried pointing the pid path in the thin.monitrc file to tmp/pids/thin.pid along with many other variations but none of them seem to work. Anyone have any ideas?

2 Answers 2

1

dunno if you solved this already yet I will reply anyway. =)

The way you do it theres a few things off.

First of all you want to monitor a "cluster" so you need to monitor each node respectively.

Secondly you need to restart the respective thin node, not in /etc/init.d.

it would look something like:

check process thin-81 with pidfile /yourapphere/tmp/pids/thin.81.pid
        start program = "/usr/bin/thin -d -e production  start -p 81"
        stop program = "/usr/bin/thin stop -Ptmp/pids/thin.81.pid"

        if 3 restarts within 5 cycles then timeout
        if totalmem is greater than 150.0 MB for 2 cycles then restart
        if cpu is greater than 80% for 2 cycles then restart
        if failed port 81 protocol http with timeout 30 seconds for 2 cycles then restart
        group thin

check process thin-82 with pidfile /yourapphere/tmp/pids/thin.82.pid
        start program = "/usr/bin/thin start -d -e production -p 82"
        stop program = "/usr/bin/thin stop -Ptmp/pids/thin.82.pid"

        if 3 restarts within 5 cycles then timeout
        if totalmem is greater than 150.0 MB for 2 cycles then restart
        if cpu is greater than 80% for 2 cycles then restart
        if failed port 82 protocol http with timeout 30 seconds for 2 cycles then restart
        group thin

something along those lines depending how you set up your thin "cluster"

2

Here's how I handle Monit watching my cluster of thin servers:

The key for me was to use a newer 'matching' feature of Monit. This was introduced in Monit 5.2 (check your version of monit with 'monit -V'). Instead of watching individual PID files, you can code a Monit stanza to watch for a Regex of a process name. Since your cluster will have 1 process for each node in the cluster, this in effect is watching for if there are no matching processes.

For example, my Monit stanza is :

check process thin
  matching "thin server"
  start program = "/bin/su - myapp -c 'cd /home/myapp/www/current; bundle exec thin start -C config/thin.yml'"
  stop program  = "/bin/su - myapp -c 'cd /home/myapp/www/current; bundle exec thin stop -C config/thin.yml'"
  group thin

Now, if no processes match "thin server", Monit will restart the entire cluster based on the config/thin.yml file in my project directory. The start/stop routine are also coded to do this as user "myapp"

I found this method to be more elegant than coding a stanza for each PID file for each process, especially when my cluster is configured using a yml file.


Additional notes:

To install the latest version of Monit (which has the "matching" feature) over a previously installed version you can install from source. For Ubuntu:

apt-get install libpam-dev 
apt-get install monit

cd /tmp
wget http://mmonit.com/monit/dist/monit-5.3.2.tar.gz
gunzip monit-5.3.2.tar.gz
tar -xvf monit-5.3.2.tar

cd /tmp/monit-5.3.2
./configure --prefix=/usr/sbin --bindir=/usr/sbin --sysconfdir=/etc/monit/
make
sudo make install

Note the extra flags on configure. I needed those to ensure that my 'make install' would overwrite the previous version of Monit installed using apt-get. Your paths may vary.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .