Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've encountered many distributed system monitoring tools that scale to 1000s of nodes, however, there seem to be none that demonstrate or even claim to be able to handle 10 000s or even 100 000s of nodes. Theoretically this should be possible with a hierarchical clustered network architecture. Has anyone encountered a monitoring system that makes such a claim or a white paper/academic paper that discusses a theoretical implementation?


locked by HopelessN00b Jan 21 '15 at 19:24

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.

closed as off-topic by HopelessN00b Jan 21 '15 at 19:23

This question appears to be off-topic. The users who voted to close gave this specific reason:

If this question can be reworded to fit the rules in the help center, please edit the question.

Assuming that this isn't made up,-Inc/details says that Yahoo uses it for 100,000 machines but has 2000 instances deployed. And I assume that DNX would suit for "management" of the instances.

Also just found Merlin which seems to be able to monitor/check 153000 hosts in ~6s rather than 1hr


I've worked with two tools in the past.

  • Zabbix is a free and open source software. It is claimed on their website that it has been tested with 10.000 nodes.
  • NetIQ Security Manager (or NetIQ Application Manager) is a closed and expensive software. It is very easy to scale up but you will need several servers to do so (database and collectors mainly).

How many hosts you can monitor with a single host will be massively dependant on the kind of checks you are running, how long each check takes, and if the queue can run concurrently.

I've seen Smokeping run against huge amounts of hosts. Same with nagios for simple setups. Guys in my sister company have nagios running against a few hundred machines doing 10-20 checks per hosts, another few hundred routers doing a series of snmp checks, and then some other "network" equipment doing a mix of snmp and custom script monitoring. All in all it is over 10k checks on the machine. Only time there are issues is when the snmp checks start lagging.

Also take a look at Zenoss. There is a few version and it does scale.


Not the answer you're looking for? Browse other questions tagged or ask your own question.