up vote 0 down vote favorite
share [g+] share [fb]

I want to make a server for my static content.
I need to serve some 3-10 mb files - a lot. (I will also put on this server some .js and .css and images from my websites).
I thought of nginx and G-WAN ( http://trustleap.com/ ).
What I don't know is what resources are needed for serving static content? How much RAM is used for each file transfer?
If I will go with a 256 mb (or 512 mb) VPS with good port and huge bandwhich how many hits /seconds will I be able to serve (3-10 mb files)? (I know "it depends" - but please give me a rough estimation based on experience or theory).
There are not a lot of files, just often downloaded - should I consider caching, or this will only use my memory needed for serving hits?

link|improve this question
3  
Question: How much RAM is needed? Answer: Enough. – joeqwerty Sep 17 '10 at 12:08
As much as you can afford. – Tom O'Connor Aug 13 '11 at 12:23
feedback

2 Answers

up vote 4 down vote accepted

If you're using nginx, then you're talking just a few KB of overhead per active connection. If you're using something like Apache, you'll have one thread per connection, which means hundreds of KB or even megabytes per connection.

However, nginx does not support asynchronous disk IO on Linux (because async disk IO on Linux is basically horribly broken by design). So you will have to run many nginx worker processes, as every disk read could potentially block a whole worker process. If you're using FreeBSD, this isn't a problem, and nginx will work wonders with asynchronous disk and network IO. But you might want to stick with Apache if you're using Linux for this project.

But really, the most important thing is disk cache rather than the web server you choose. You want lots of free RAM so that the OS will cache those files and make reads really fast. If the "hot set" is more than say 8 GB, consider getting less RAM and an inexpensive SSD instead, as the cost/benefit ratio will likely be better.

Finally, consider using a CDN to offload this, and getting a really cheap server. Serving static files is what they do, and they do it very fast and very cheaply. SimpleCDN has the lowest prices, but MaxCDN, Rackspace, Amazon, etc. all are big players at the low end of the CDN space.

link|improve this answer
Thanks for all the info. So, if I chose FreeBSD or other UNIX should be ok to serve what I need (1 - 10 TB) from a vps with less memory using nginx? What about windows? – cripox Sep 21 '10 at 15:46
feedback
up vote 2 down vote

If the OS can cache the hot part of the content into ram, it will not use the disk and will serve things really quickly. Hundreds of request per second should be possible on a VPS, you will most likely saturate the network well before you run into CPU limits.

If the content does not fit into ram, then disk IO (seek, throughput, filesystem fragmentation) will come into play and the equation changes.

The webserver will add a memory overhead per client, but nginx can do that in a few kilobyte per connection.

Hope these pointers can help you.

link|improve this answer
Thanks, but what is not clear to me is if there is memory overhead per connection: i.e. if nginx or gwan consumes memory for every hit? If I have 10 request of a 5 mb file in the same time, will this mean there will be 50mb memory used for serving it? Maybe + memory for threads (I don't know if nginx or gwan uses threads for evey connection). – cripox Sep 17 '10 at 11:44
Per open connection they require some memory. 10 concurrent requests (at any time there are 10 tcp connections open sending/receiving the file) will require 10 times a few kilobytes. This has nothing to do with the contents, so the 5MB does not apply here. – Joris Sep 17 '10 at 12:59
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.