4

Once you move beyond 100k hits/month, what would people in this community say is the biggest hurdle?

My situ: Tons of static media (audio/video/images) being served off of S3/CDN, but being stored locally as backup (not served though). Everything that can be cached, is cached, with about 8 gigs of memory scalable to 32.

We're currently handling about 100k hits without a problem, and would love to know what problems others have run into: load balancing? memory issues? disk i/o?

Thanks for any tips. I've looked through the related questions, and they answered them well, but just wanted to get some more feedback.

2
  • You'll probably get a better answer over at serverfault.com
    – musicfreak
    May 31, 2009 at 6:46
  • 7
    100,000 hits per month is about 125 requests per hour, or about 2 per minute. You shouldn't even need to think until you've got 100 times that traffic unless each request is really heavy.
    – Jon Skeet
    May 31, 2009 at 7:42

8 Answers 8

5

Hardware is so cheap these days that increasingly few people even get to the point where they need more than one machine. $10,000 will buy you:

  • 16-32GB of RAM;
  • 2 quad core Xeons;
  • RAID5 disk array.

That kind of machine can serve over 10,000 concurrent users on an even mildly optimized site for all but the most resource intensive applications.

Basically there are two approaches to scalability:

  • Vertical: basically buying the biggest machine you can so you don't need more than one;
  • Horizontal: doing things in a way that lends itself to parallelism. Only needed on the most intensive of applications.

Look at StackOverflow: its basically run on a Web server plus a database server and it does in excess of 6 million hits a month.

That being said, scalability is about finding and addressing your bottleneck.

  • If your database is slowing things down either give it more resources or use some form of in-memory caching to take the load off;
  • If disk I/O is your problem then the same applies;
  • If you're running out of memory to the point where it's causing too many page faults and thus causing a disk I/O problem, add more memory;
  • Does your application and it's data lend itself to partitiioning across servers? If so, that's one way of scaling horizontally;
  • If bandwidth is an issue and you're delivering large files then perhaps a CDN is the answer;
  • And so on.

Ultimately though, 100k hits/month is not that large. I suspect that these days on a typical Web site you'd need to get beyond 10 million/month before you had real problems, assuming you don't do things badly (eg if you don't index your database searches then of course you'll be having problems with that but they're nothing to do with hits/month).

I would say redundancy is a far bigger headache than scalability. The issues involved in having redundant links, monitoring processes for system failure, having and maintaining a DR (disaster recovery) site, dealing with the issues that entails (like split-brain clustering), etc are far more difficult and tedious.

4
  • Thanks for the in-depth write up. The static media is already coming off a 3rd party cdn, so that def. won't be the issue. I assumed our current setup could handle 2-3m without too big of an issue, but I wanted to be fully aware of what I could expect when we got beyond that. Our needs are entirely db based. Our hits are to very dynamic, very active pages, which are similar to message boards in the type of content being spit out.
    – onassar
    May 31, 2009 at 7:08
  • That being the case, in-memory caching will probably be your next hurdle I would guess. That is assuming you're already effectively caching and serving static content like Javascript/CSS/images.
    – cletus
    May 31, 2009 at 7:12
  • You should be warned... Bigger machines = bigger SPOF :-) May 31, 2009 at 12:50
  • 1
    Upvoted for bringing up reliability and redundancy -- arguably more important that capacity. Well done.
    – pboin
    Sep 27, 2009 at 10:56
2

Use Varnish or Squid. Both are web accelerators and both are excellent for static media files.

If you scale out, you can even have 1 machine for web cache and 1 machine for the dynamic content

Alternatively, you could try tweaking apache by using the mods: mod_expires, mod_headers, mod_cache, mod_file_cache, mod_mem_cache.

1

Depends a little bit on what sort of database handling your doing - if you are serving static content or near static content; you will scale well beyond 100K/mo with very modest hardware.

Complex database driven sites such as forums can be a larger problem, you will need to look into database replication, reverse proxies on the main website to act as additional caches, and additional load-balanced webservers. Most 'heavy' DB-using software also supports things such as memcached, which can be used to reduce the load on the database on common requests.

Frankly though, I think you are probably overestimating demand right now - 100K is within the realm of handling on a single machine just fine. Once you break 10M, then you need to look at more complex solutions such as the ones I have outlined above.

My personal advice is to always favour parallel machines over single complex ones - not only does it end up being cheaper, but it gives you a lot more room to grow when the time comes. Expanding already expensive hardware gets you into the realm of $25,000 servers - where 'bang for buck' disintegrates.

1

Once you move beyond 100k hits/month

Is NOTHING!!! If you have troubles serving 100k hits per month you are in trouble. Generally you should not have problems with most dumbest systems serving 100K per day.

This is about LAMP... and CMS systems

My situ: Tons of static media (audio/video/images) being served off of S3/CDN

There are very good web servers for these purposes: lighttpd -- it serves YouTuble media streaming, nginx is good as well.

These are lightweight web servers that scale extremely well with high loads.

Even switching Apache module from mod-prefork to mod-worker would help (even lighttpd are still much better).

Bottom line: these loads are far from being high.

1

Other have pointed out that 100K hits is not a problem though I suspect they missed your point that you were already serving that many hits without issue...

Given that your content includes video I would suggest that network bandwidth would be your first problem, followed by I/O requests once you have enough concurrent requests for large objects. If you have more RAM than data then I/O will be less of a problem, once everything is sat in cache (though it'll take a while to prime the cache after any server shutdown).

One thing that you must specify when trying to size requirements like this is what distribution your hits have. Are they evenly distributed over the month or more bursty? For instance a site hosting information about a national lottery might see 80% or more of its weekly traffic in a two hour window after the draw time - 80% of 100K requests in two hours is 2.7 per second (assuming a 4 week month), still not massive but getting there depending on the size/complexity or the responses. Other less predictable bursts are possible too, such as what happens if a video on your site hits the front-page of Digg or similar.

0

The biggest hurdles are probably

  • getting your application in shape for a loosely coupled architecture which drops ACID and other strong constraints to allow for asynchronous operation and short-term inconsistencies. See Brewer's CAP Theorem for a broad overview of this topic.

  • Deal with proper management of many nodes. This includes basic system administration tasks like Package management as well as testing and rolling out change requests. When you've got 50 webservers you can't just "restart all the apache servers"; some thought has to be brought into these tasks

  • which brings me to my last point - understand the effects which can be seen in large systems. Thundering Herd is one of the things you're going to see. This article on Audiogalaxy also describe a few problems of large systems.

Scalable Internet Architectures and The art of capacity planning are two books on this topic which I can warmly recommend.

0

I'm a scaling guy, I have one rule: always leave yourself the option of buying yourself out of a performance problem.

Anything that can't be fixed by spending money has no fixed 'time to resolve', so always try to code/confit your way out of a problem but always leave yourself with the option to throw cores/cycles/memory/storage/bandwidth/whatever at the issue - they can always be FedEx'ed :)

0

Once you move beyond 100k hits/month, what would people in this community say is the biggest hurdle?

Over thinking the problem, over engineering the platform, and over complicating the system.

100k/month is nothing. Your entire goal at that point should be spending as little of your time as humanly possible focusing on it.

You must log in to answer this question.