2

What options are there to serve static files very fast?

Are there any options that allow caching the file in the memory of the server, etc? For example caching the js files in the memory.

I know of nginx but it still reads the file of the hard disk every time and what i want to do is to reduce the I/O from the disk by storing them in the memory.

I have about 1000 requests on js files every second and they are slowing down the server, even crashing apache sometimes.

What other alternatives are there to nginx for running along a LAMP stack on CentOS 5.6

5
  • This is a question for Serverfault. BTW: Its surprising you're seeing performance bottlenecks serving static files (are you really? and why is it reading from the hard disk each time, have you disabled the buffer cache?)
    – derobert
    Aug 4, 2011 at 17:24
  • Yes, I am serving around 1000 js requests per second. How can I increase the buffer cache?
    – user658911
    Aug 4, 2011 at 17:28
  • 1
    Buffer-cache automatically uses RAM that is otherwise free. To increase it, add more RAM or reduce RAM usage by other things, but I'm not convinced that's really your bottleneck. Once this gets moved to Serverfault, it'll be seen by a crowd that more routinely deals with this stuff.
    – derobert
    Aug 4, 2011 at 17:33
  • @user658911 let us continue this discussion in chat
    – derobert
    Aug 4, 2011 at 17:43
  • Consider using a CDN / external block storage.
    – Avatar
    Mar 25, 2020 at 5:13

4 Answers 4

2

RAM is definitely faster than disk (if you need details, check numbers everyone should know).

If you have the memory available on the webserver, you can make a RAM disk. If you've got an array of systems, maybe something like memcached would be better.

1

Zeus is the fastest webserver I know of. On boot copy everything to a RAM disk...

If you want/need to stick with a LAMP stack there are acceleration modules for Apache for static content, and a front end reverse proxy like Squid, Vanrish, or HAProxy can help too.

Depending on why you need to server "faster" (a meaningless term in this context) you might benefit from a load balancing farm or a different Internet connection.

1

I know of nginx but it still reads the file of the hard disk every time

No it does not read the file every time from the disk. Since you are using CentOS and thus a Linux kernel, it sure as death does cache frequently accessed files and directory entries to RAM.

Either you have much more "hot" static files than your server does have RAM or something else is eating up your disk I/O. Couple of thousands of requests to static files should be peanuts to every reasonable web server software. Even Apache should handle it OK, though lighttpd or nginx are probably better suited for that task, especially if the static files are large.

Perhaps the log level is so high that the constant writing tortures HD? If it's about that and you need to store the logs, consider sending the logs to a separate central syslog server over the network.

0

I second the suggestion to use a caching system like Varnish. Depending on your needs, if you have some dynamic content, I would suggest do a setup like this:

Use Varnish as the front end, with both Apache AND Nginx in the back end. When something is requested that isn't cached, it (Varnish) then passes the request off to Apache for the dynamic content, and passes the request off to Nginx for the static content.

This way, most of your stuff is stored in RAM (Varnish cache). Varnish is an incredibly fast caching server / reverse proxy (and can also be configured to be a nice load balancer for multiple servers).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .