0

After searching the web for hours and couldn't find what I was looking for, I thought someone here might have an idea ;-)

I'm looking for Linux a program that will cache in memory a designated folder. The cache will then be accessible via HTTP. (I don't want to rely on the OS/filesystem cache)

Correct me if I'm wrong, but from what I've understood most of the proxy servers (Squid, Varnish, Traffic Server, Polipo, Lusca) use HTTP for their input. I can't use Memcache because it's not really designed for that kind of usage...

Any thoughts ?

Thanks & Regards

Matt

Mathieu D
  • 1
  • 1

3 Answers3

1

I don't want to rely on the OS/filesystem cache

Why?

Implementing this outside the kernel means the code will have to hook into inotify to ensure cache consistency. It'll also need code for loading and refreshing the cache. It'll be horibly complicated and in most cases will be no faster than relying on the filesystem cache. In some cases it will actually slow the system down by duplicating content.

symcbean
  • 21,009
  • 1
  • 31
  • 52
  • Seconded. Can the OP give some more details around the actual problem he is trying to solve? – mfinni Oct 06 '10 at 13:49
  • I'm using a module on the web-servers to send only a few parts of video files (ie changing the start and end times). There is about 4500 videos files, but out of those only 100 are requested frequently. As for now the web server is getting the video files directly from the filesystem which is slow (the server is a cloud server) and use a lot of IO. My idea was to use a cache, like a proxy-cache, so the most demanded videos can be stored in memory and then delivered faster, but also to be able to share the cache between the servers (if a cache doesn't have a file it's sent from another one). – Mathieu D Oct 06 '10 at 21:44
  • It is because I need to be able to send only parts of the video that I can't use a proxy in front of the web-servers... Hope it all makes more sens now. Thanks for your inputs ! – Mathieu D Oct 06 '10 at 21:46
  • With varying offsets using a reverse proxy offers little benefit - but that's not what you asked: I still stand by my query and statement - this is not going to improve performance. – symcbean Oct 07 '10 at 11:51
0

It seems to me that it wouldn't take very long to throw together a script in <language of choice> that would hold a hash of location to content mapping.

Ignacio Vazquez-Abrams
  • 45,939
  • 6
  • 79
  • 84
  • Could you give more details please? Don't you think having a script doing the interface between a cache and the web-server will be another thing to maintain/break? – Mathieu D Oct 06 '10 at 21:48
  • 1. The script can *be* the cache if done right. 2. A pre-packaged program to do this can also need maintenance/break down. – Ignacio Vazquez-Abrams Oct 06 '10 at 21:53
0

Is a RAM disk what you are looking for? http://www.vanemery.com/Linux/Ramdisk/ramdisk.html

Porch
  • 680
  • 5
  • 12
  • The problem with a ram-disk is to "link" it to my existing folder in order to store only the most requested files (all the files won't fit in memory + if the server restart everything will be lost). – Mathieu D Oct 06 '10 at 21:51