I've been using monit for some time and have really liked it thus far. I recently decided to give m/monit a try and the UI looks great and is super informative. My only hang up is monitoring servers that are autoscaling on amazon AWS and getting a buildup of shutdown servers. Is there a well known way to keep the number of hosts listed to the "living" instances. I know there is an API, but I'm wondering if there are any more in depth solutions out there
2 Answers
Well, this is an interesting question. This has come up on the Monit mailing list before.
I use M/Monit and Monit as well. Each Monit host has a unique ID generated upon the first run of Monit. On my systems, this is set to /var/monit/id. When that host checks in to M/Monit, it's automatically created in the M/Monit dashboard. So scale up in EC2 is automatic.
I would suggest using the API to manage hosts that are either short-lived or shut down for extended periods. You may be able to trigger the hosts' deletion from the host itself!
In the API documentation, there is a method to delete a host with a POST:
POST to http(s)://<mmonit_ip>:<mmonit_port>/admin/host/
with content:
"id=<hostid>&Delete=1"
You can do this with curl.
-
I like your idea, my only issue with it is that there needs to be a signal to the server that it's about to be shutdown. Does AWS use any one particular command when shutting down a server (from autoscaling or in general)? Mar 9, 2014 at 20:34
-
I think I've found an answer to that issue, but I'd like to hear what you think Mar 9, 2014 at 20:39
-
Old comments, but shouldn't just be left hanging. You can get notifications on your autoscale groups and use those to manage your configuration: docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/… Nov 12, 2014 at 18:08
You can change the host ID in /var/lib/monit/id to the hostname of the Linux machine, but you need to manage the naming schema of your servers.
This way when an Instance in your AutoScalingGroup died for some reason, the replacement will be added under m/monit with the same name.
