2

Say I've got a web application running on a Linux VPS box. In order to apply kernel upgrades and some security fixes, I'll need to reboot the server. During this period, the server isn't reachable. I'm wondering if it's possible to display a maintenance notice to the user (via HTTP) at this time. The notice will be sent by a PHP script, which will handle requests to provide maintenance notice in either HTML format, a picture, or web service XML to clients. Or if possible, provide a partially running service.

What kinds of hardware and software do I need for this to happen (minimum budget please, and I'm hoping that a typical VPS/dedicated hosting provider can support these)?

Now I already knew that I could use software like squid to provide load balancing between 2 HTTP servers. But what do I do if squid itself needed an upgrade, or the CentOS server that's hosting squid? Plus, I don't actually need load balancing, it's fine if I have to "switch" to maintenance mode manually. Any ideas?

Thanks in advance.

0

6 Answers 6

2

If it is important to you to have this much uptime (ie. during a <5 minute server reboot) then I would recommend setting up a second virtual server. It could be very minimal, running lighttp and a single static page - or it could be a duplicate of your live site.

Then set up heartbeat2 on both servers, and give them a shared IP address. While the main machine is up, it will respond to the address, but if it goes offline the IP address will automatically be brought up on the second machine within seconds.

If you decide to put a full version of your site on both servers, I would install ldirectord in addition to heartbeat2 which is a load balancer application that will run on the same machines as your webservers. In this case there will not even be seconds of downtime when either server is down.

Then, if you want to get fancy you can mount your website content and configs from an nfs share on another server (or server cluster), so that no synching or duplication between webservers is necessary.

7
  • Thanks for the recommendations, I'll checkout the documents of the programs you mentioned. But one thing I'm not sure, is sharing the same IP among two servers. If that's possible, then I won't have any problems. The hosting provider I contacted says it's not possible to have two servers using the same IP. Even if you could configure a static IP on both, their router/switch simply do not use the second server.
    – He Shiming
    May 28, 2009 at 14:35
  • Sorry, had to down vote since this solution would cause possible bad data. If users connect to a clone system while the primary is down for repairs/upgrade/etc. they could be inputting data into the system not thinking twice. If you went two servers architecture your best bet is to cluster (allows you to bring one down completely at a time without issue)
    – SQLChicken
    May 28, 2009 at 14:37
  • Also the only way to use same IP on two servers is to use a network load balancer that had one IP which would subsequently point to two or more IPs (one for each server).
    – SQLChicken
    May 28, 2009 at 14:38
  • @SQLChicken, many thanks for the tips. I was hoping there's something between blackout and cluster. But I can see that cluster is my best option.
    – He Shiming
    May 28, 2009 at 14:42
  • Voting this up as it's the best solution for maintaining site uptime across reboots. If the second server is a static maintenance page (as suggested) then there's no possibility of data corruption. If it's a duplicate of the site, then the site needs to be able to deal with writes from multiple servers - but that's probably worth doing anyway. May 28, 2009 at 16:12
3

What we typically do for upgrades is to build a new machine with the upgraded components (software, hardware, etc.). We then load production data onto the system and test it thoroughly. Once we're ready, we reload the latest production data, and point DNS to the new machine.

That way, there's constant availability between the old box and the new one.

2

If you need to do an upgrade on a server that requires a reboot, you need at least one alternate machine to host the temporary site while the original machine is offline. One option is to use DNS with a low TTL to point to the new server while the original website is offline.

Another like you said is using squid to proxy. I believe squid can also work in a round-robin DNS situation with multiple squid servers.

However, any DNS propagation you do, if choosing to go that way, may be cached by the downstream servers. You can try to alleviate this with a 30 second TTL or something ridiculously small, but some DNS server will ignore that and still cache the value for longer.

If you are switching to maintenance mode anyway, why not just weather the 5 min it takes to reboot and send out a notice to users beforehand? This should work fine in everything except the extremely high-availability situations.

1

Some DNS providers (such as dnsmadeeasy.com) provide a failover dns service.

This service will regularly check to ensure your primary IP address is up, and if it is not, it will start responding with a backup IP address - which could be pointed to a "Server Unavailable - please try again later" page on another host somewhere.

This would probably not be suitable for reboots however, as the TTL on dns entries would prevent people from getting the failover page during the 2 or 3 minutes it takes to reboot. It works great for longer (or unplanned) outages however.

1
  • Thanks! I'll keep this failover dns service as a backup plan.
    – He Shiming
    May 28, 2009 at 14:20
1

Check out Zoneedit DNS. For a very low cost they offer DNS fail over.

0

I would suggest just giving a notice to the users on the site informing them of the blackout times. Then during the blackout period they would get the 404 or 500 errors.

edit: They would not get the 404/500 but rather a timeout error.

2
  • Hmm... actually they won't get 404 500 errors. Because the entire server is offline, so all they've got is connection timeout. As if the service disappeared.
    – He Shiming
    May 28, 2009 at 14:18
  • of course, that is correct...so the users time out, but they should still know it's coming because of the warning.
    – northpole
    May 28, 2009 at 14:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .