0

After searching the web for hours and couldn't find what I was looking for, I thought someone here might have an idea ;-)

I'm looking for Linux a program that will cache in memory a designated folder. The cache will then be accessible via HTTP. (I don't want to rely on the OS/filesystem cache)

Correct me if I'm wrong, but from what I've understood most of the proxy servers (Squid, Varnish, Traffic Server, Polipo, Lusca) use HTTP for their input. I can't use Memcache because it's not really designed for that kind of usage...

Any thoughts ?

Thanks & Regards

Matt

1

I don't want to rely on the OS/filesystem cache

Why?

Implementing this outside the kernel means the code will have to hook into inotify to ensure cache consistency. It'll also need code for loading and refreshing the cache. It'll be horibly complicated and in most cases will be no faster than relying on the filesystem cache. In some cases it will actually slow the system down by duplicating content.

| improve this answer | |
  • Seconded. Can the OP give some more details around the actual problem he is trying to solve? – mfinni Oct 6 '10 at 13:49
  • I'm using a module on the web-servers to send only a few parts of video files (ie changing the start and end times). There is about 4500 videos files, but out of those only 100 are requested frequently. As for now the web server is getting the video files directly from the filesystem which is slow (the server is a cloud server) and use a lot of IO. My idea was to use a cache, like a proxy-cache, so the most demanded videos can be stored in memory and then delivered faster, but also to be able to share the cache between the servers (if a cache doesn't have a file it's sent from another one). – Mathieu D Oct 6 '10 at 21:44
  • It is because I need to be able to send only parts of the video that I can't use a proxy in front of the web-servers... Hope it all makes more sens now. Thanks for your inputs ! – Mathieu D Oct 6 '10 at 21:46
  • With varying offsets using a reverse proxy offers little benefit - but that's not what you asked: I still stand by my query and statement - this is not going to improve performance. – symcbean Oct 7 '10 at 11:51
0

It seems to me that it wouldn't take very long to throw together a script in <language of choice> that would hold a hash of location to content mapping.

| improve this answer | |
  • Could you give more details please? Don't you think having a script doing the interface between a cache and the web-server will be another thing to maintain/break? – Mathieu D Oct 6 '10 at 21:48
  • 1. The script can be the cache if done right. 2. A pre-packaged program to do this can also need maintenance/break down. – Ignacio Vazquez-Abrams Oct 6 '10 at 21:53
0

Is a RAM disk what you are looking for? http://www.vanemery.com/Linux/Ramdisk/ramdisk.html

| improve this answer | |
  • The problem with a ram-disk is to "link" it to my existing folder in order to store only the most requested files (all the files won't fit in memory + if the server restart everything will be lost). – Mathieu D Oct 6 '10 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.