Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm currently running a single EC2 instance, and plan to move to a fault tolerant architecture eventually. Something that will help me decide how urgent this migration is is EC2 MTBF.

Is there any data about how often EC2 machines fail?

share|improve this question

3 Answers 3

up vote 3 down vote accepted

There are no published MTBF statistics. "More often than you would like" is about the best that you are going to get. Beyond that, the other posters have provided excellent answers about how to handle architecting your application.

share|improve this answer
    
Sadly, I have to accept. I was't expecting official numbers from Amazon, but perhaps for some of their more heavy users, or at least a ballpark figure. –  ripper234 Nov 2 '11 at 8:49

This is something I've researched for a company project and, unfortunately, its not really possible to quantify. Because there are such huge numbers of nodes in EC2, and cluster computing is inherently unstable due to the large numbers of machines in play, its really a factor of the following: can your application handle failures?

To note, the biggest issues seem to be single points of failures (obviously). Don't host your single database in the cloud, a single file store, etc.. Disk failures on EC2 aren't exactly common place, but I've seen experiences from 0.0001% to 2% disk failure rates. Googling around (and checking the EC2 boards) will yield you more evidence of this. For long term storage--or "more reliable" storage--check out Amazon S3.

Overall, you shouldn't view EC2 instances as stand in replacements for servers in your own data center or co-lo. Rather, you should view them as part-time workers--many will show up, most will do a good job, but every once in a while, one of them will call in sick or quit. When that happens, your application needs to be able to handle the loss, be it data corruption or a server going ofline. If it can (like you say), then cloud computing is a good idea.

share|improve this answer
    
I'm asking because a previous toy website I had lasted > 6 months on a single EC2 instance. I know we should move to something more tolerant, but I'm "confident enough" to host a production app there for a few weeks, until we can setup something more serious (startup, high availability is not the #1 priority right now). Your post does't really answer the question I asked - I know it's a bad idea for the long run, but I just want to know the MBTF so I can make a prioritize moving to multiple servers. –  ripper234 Nov 1 '11 at 14:47

I generally expect the MTBF to be higher for EC2 instances than for high end hardware I would buy and put in a data center.

The big difference is that I can design my EC2 setup so that when an instance fails, I can bring up a new one within minutes of being alerted and getting to an Internet connection. This is a huge contrast for what I used to have to do when a server failed in a colo 40 minutes away where I had to drive down there, debug the hardware issues, install replacement parts if I happened to have them on hand.

For example, if an instance's underlying hardware fails, you can throw it away and switch to new hardware with a couple commands:

A Simpler Way To Replace Instance Hardware on EC2
http://alestic.com/2011/02/ec2-move-hardware

So, though I sometimes design for replication and automated recovery or failover, other times I tend to find myself living with the risk of a little downtime because it's so easy to recover manually.

Document/script instance setup (software installation/configuration) so you can reproduce it at a moment's notice. Take regular snapshots. Make regular backups of your data (in addition to snapshots). Keep copies of backups offsite (outside of EC2).

If you need extra nines of uptime, go for the more complicated replicated, redundant, failover, autoscaling architectures, which AWS also makes easier than with physical hardware.

share|improve this answer
    
That's mean time to recovery, not mean time between failures. Having a low MTTR can offset a high MTBF, as you say, but it doesn't answer the question. –  Ben Jencks Nov 2 '11 at 2:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.