0

I would like to know how sites like the following perform distributed website monitoring (from multiple checkpoints/countries).

pingdom.com, site24x7.com, uptrends.com, siteuptime.com, etc, etc.

To be exact, what process would occur in checking if a given domain name went down? If the server finds that the site is down, what is the next step? Would it make a REST API request to a separate server to run the same test and report the results?

I have a few theories, including:

  • utilizing host(s) from different countries
  • utilizing proxies from different countries

I'm looking for the most proper or correct way to handle this, which can include the usage of servers from multiple countries/hosts.

2
  • Can you please clarify what you're asking? At first it looks like you want to know how others do it (unanswerable) and then you seem to want to know how to implement it. Which one is it? Mar 17, 2010 at 3:13
  • @John, I'd really just like to know how to implement it. Mar 17, 2010 at 10:52

2 Answers 2

1

The vast majority of that is going to be implementation dependent. For example, it's up to each provider how they implement the internal communications. Perhaps REST, perhaps XMLRPC, perhaps Pyro, and so on.

From a feature standpoint, it looks as though they all handle it slightly different. Watchmouse appears to randomly grab an available tester and run a test on one of it's configured nodes, which happen to be geographically dispersed. Others appear to assign to a single geographic location.

The "most correct" way is relative, depending on what you want to monitor. What exactly do you want to monitor, and how important are international checks to you? Can you do anything about it if tests from Canada are fast and checks from the Ukraine are slow?

If I were doing it personally, I'd have remote polling nodes report back to centralized collectors for data collation, but alerts would go from the edge systems in order to speed things up.

All depends on what it is you want to do, though. HTH.

2
  • So this begs the question... would I have a number of servers hosted in each of the countries? Or is there another way to ping a site from different countries/locations without the necessity of servers in those countries? Mar 17, 2010 at 2:22
  • You would need a server or a method of originating an outbound connection. Maybe a small collection of VPS accounts would suffice. I've always thought it would be cool to use iPhones for this purpose.
    – McJeff
    Mar 17, 2010 at 4:21
0

Ideally you would want client machines scattered globally to run whatever kind of check you want. I'm guessing you just want to know if the site is up, which can be checked with a simple GET. It would be possible to do this using proxies but you would then also need to determine whether it's the web site or the proxy that is not responding. That should be easy enough if you're using multiple proxies.

Depending on how much can be automated on the web server end will dictate, to at least some extent, how you go about using the results of the monitoring system. As I see it, the basic system would go something like this:

  • The sites are checked periodically (5 minutes?) to see if they're up
  • A site that fails to respond will be rechecked every half minute or so and a check counter is incremented for that site
  • If the site responds the check time and fail counter are reset
  • If the site fails to respond after X number of retries an alert is sent and, if possible, the service and/or server is restarted

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .