Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The title probably makes little sense, so here is an example.

I have a file hosting site, that serves a large amount of semi-randomly accessed files.

The setup is as follows:

  • High horsepower front-end +DB server that also does encoding for files that need encoding
  • Fresh file server, which stores newly uploaded content, thats probably (and usually) rapidly accessible, which has 500GB of raided SSD storage, that can push over 3GBit of traffic.
  • 3 cheap node servers, containing 2 x 750GB SATA drives in raid1, where files older than 2 weeks are archived, from the SSD server (mentioned above).

Files on each server are accessed via subdomains (via modsec) in a straight forward fashion (,, etc)

Where I have the problem is this. I introduced a "premium" service where people pay a small fee every month, and get ad-free, quick accesses to stuff on the site. Once they are logged in, they access same files via via a different modsec script, with a different pass phrase. That all works fine and dandy.... except the cheap node servers are all IO bound, so accessing the files on them via a different, unsaturated network makes no difference, since it cannot read off the drive fast enough.

What would be a good way to make files on the site be accessible via 2 different network routes, 1 of which will be saturated (the "free network") while all other files are on an un-saturated "premium" network?

share|improve this question
up vote 1 down vote accepted

Hang on, your question pertains to something you already answered half-way through your explanation. Like you said, your problem isn't with saturation of the network adapters, you're limited by IO on the SATA drives. Or am I misreading?

Assuming that's right, you may be able to make some improvements depending on the access patterns for your older files. If you tend to get 'runs' on a single file at once (i.e. a link to the file is posted on a blog, then suddenly you're getting 500 unique IPs requesting the same file) then you should move that file into either a memory or pagefile cache, or stage it across to the SSD server before you serve it out.

A similar question was asked recently and I explored other possible solutions:

share|improve this answer
Well, what i thought I could do is either: put SAS drives instead of sata, so it could push ~600-700mbit instead of 250 with sata... or install a cache server between the SSD box, and the archive boxes, that would cache rapidly accessed files from the archive boxes. Except I dont know how to implement that. – user11350 Mar 31 '10 at 4:52
What kind of hardware are the node servers running, and what drive controllers? SAS should net you a performance gain, but it seems to be more dependant on the controller than the drives. – Chris Thorpe Mar 31 '10 at 5:07
Its not the controller. Its 2 7200rpm SATA drives in raid1, using some rocketraid card. Completely random IO, sustaining about 250mbit with lots and lots of IO. – user11350 Mar 31 '10 at 5:12
Are your files duplicated across the node servers, or spread? Sounds like you'd be better off with a spread where multiple clients hitting the same file would only be touching a single node server. – Chris Thorpe Mar 31 '10 at 5:53
Get proper hardware. 2x7200RPM on a low end card is not going to cut it. Upgrade to a proper SAS card and get in a proper storage subsystem - many discs, 10k RPM. WD has nice velociraptors, and Supermicro has a cse that can fit 23 of them into a 2 rack units high. You will be surprised of the IO load that can handle. – TomTom Jul 23 '10 at 23:55

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.