Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a growing stock photo website. It's currently hosted on a pretty basic dedicated server from WebFusion and I now need to expand, both storage and performance, and to lose their rubbish support team. We have 500k photos online now and reckon we could make the million or two mark quite soon.

I'm now stuck on what options there are for expanding, whether to just upgrade the server we have to hold more files, use a third-party storage service, like Amazon, buy our own server or something else.

Any suggestions would be amazing, thanks.

share|improve this question
1  
Impossible to answer without a five hour conversation and then homework of a half dozen books. Have you considered convening with a consultant? =) –  Wesley Apr 26 '12 at 2:32

2 Answers 2

up vote 5 down vote accepted

I run photoblog.com and I can give you my experience.. We started just before amazon's aws offering. So our way to scale was bringing a new small server online with raided drives and selecting a random server to place the images on.. this was great but it sucked to scale.

I switched to putting all images on s3 and couldn't be happier.. I don't have to worry about losing a whole image server and all the images.. I don't have to worry about backups at all. If you want to serve images quicker but at around 2x the cost you can serve them from cloudfront.

We also see about a 20% cost savings from using s3 over storing them on managed servers.

share|improve this answer
    
That's cool, thanks. If using CloudFront, would I store the entire web application on there, just like a normal server, and point a domain name at it? Then obviously store the photos on S3? –  PaparazzoKid Apr 26 '12 at 2:48
    
No.. cloudfront just turns your s3 bucket into a true cdn that mirrors your data around the world. –  Mike Apr 26 '12 at 3:22
    
Oh, I see. Many thanks. I'm literally about to sign up to S3, it looks perfect so far. –  PaparazzoKid Apr 26 '12 at 3:47

Yes!

No!

Maybe?

It all depends on what level of performance you need and how secure you want your data. If you need hundreds of reads per second, you'll want a storage system that can cache reads and that will cost you a nice sum of money and some of your own hardware. If all that you need is raw storage, then that simplifies things at least as far as architecture is concerned.

Data protection is likely important for you so take a double amount of storage into consideration to hold backups. You might want to look into deduplication with ZFS (BSD / Illumos) or SDFS (Linux). Replication, file versioning, security, SANs, BANs, bit level, block level, byte level...

Break it down into three levels:

Performance. Failure resilience. Backup resilience.

Start with performance. Figure out how many reads and writes you're dealing with. Move on to failure resilience. Figure out just how redundant you want to be. Geo redundant? Or is it acceptable to be confined to one datacenter? Do you want to have your content delivered to people in far reaches of the globe and have it be local to them? NExt is backup resilience. How much history do you want and how much versioning do you want. They all cost bytes.

TL;DR

There is no answer.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.