Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I would like to know how sites like the following perform distributed website monitoring (from multiple checkpoints/countries).,,,, etc, etc.

To be exact, what process would occur in checking if a given domain name went down? If the server finds that the site is down, what is the next step? Would it make a REST API request to a separate server to run the same test and report the results?

I have a few theories, including:

  • utilizing host(s) from different countries
  • utilizing proxies from different countries

I'm looking for the most proper or correct way to handle this, which can include the usage of servers from multiple countries/hosts.

share|improve this question
Can you please clarify what you're asking? At first it looks like you want to know how others do it (unanswerable) and then you seem to want to know how to implement it. Which one is it? – John Gardeniers Mar 17 '10 at 3:13
@John, I'd really just like to know how to implement it. – cballou Mar 17 '10 at 10:52

The vast majority of that is going to be implementation dependent. For example, it's up to each provider how they implement the internal communications. Perhaps REST, perhaps XMLRPC, perhaps Pyro, and so on.

From a feature standpoint, it looks as though they all handle it slightly different. Watchmouse appears to randomly grab an available tester and run a test on one of it's configured nodes, which happen to be geographically dispersed. Others appear to assign to a single geographic location.

The "most correct" way is relative, depending on what you want to monitor. What exactly do you want to monitor, and how important are international checks to you? Can you do anything about it if tests from Canada are fast and checks from the Ukraine are slow?

If I were doing it personally, I'd have remote polling nodes report back to centralized collectors for data collation, but alerts would go from the edge systems in order to speed things up.

All depends on what it is you want to do, though. HTH.

share|improve this answer
So this begs the question... would I have a number of servers hosted in each of the countries? Or is there another way to ping a site from different countries/locations without the necessity of servers in those countries? – cballou Mar 17 '10 at 2:22
You would need a server or a method of originating an outbound connection. Maybe a small collection of VPS accounts would suffice. I've always thought it would be cool to use iPhones for this purpose. – McJeff Mar 17 '10 at 4:21

Ideally you would want client machines scattered globally to run whatever kind of check you want. I'm guessing you just want to know if the site is up, which can be checked with a simple GET. It would be possible to do this using proxies but you would then also need to determine whether it's the web site or the proxy that is not responding. That should be easy enough if you're using multiple proxies.

Depending on how much can be automated on the web server end will dictate, to at least some extent, how you go about using the results of the monitoring system. As I see it, the basic system would go something like this:

  • The sites are checked periodically (5 minutes?) to see if they're up
  • A site that fails to respond will be rechecked every half minute or so and a check counter is incremented for that site
  • If the site responds the check time and fail counter are reset
  • If the site fails to respond after X number of retries an alert is sent and, if possible, the service and/or server is restarted
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.