1

I work for a large internet site where people can share large pieces of text with each other. We store most information in InnoDB databases, but the actual pieces of text are stored somewhere in text based files. These text based files vary between a few KB up to 10MB each. There are millions of these files, and we have setup a good folder/file structure so that there are never too many files in one folder.

The webserver (DB is on another one) where these files are store is a powerful machine with 4x 15k SAS drives in RAID10 and has 24GB ram. We run Nginx as a webserver and Xcache to speed up PHP. This all works perfectly, and the load usually varies between 0.7 and 1.5.

Now, I only plan on using Memcached for storing the 'text based' files in the ram. So instead of having to read them from disk every time someone requests a page where one of these 'text based' files has to be loaded. In PHP I use file_get_contents(); to load the 'text based' files into a variable, and then display it somewhere on the page afterwards.

My question is, would it actually lower the load you think implementing Memcached for this particular feature, or does Linux already have some kind of internal file caching by itself that is used when I request a certain file via file_get_contents very often?

Mr.Boon
  • 1,471
  • 4
  • 24
  • 43

3 Answers3

2

The obvious answer is that you have to test the various options yourself in your specific scenario.

That said, Linux uses all available RAM as disk cache, but only as long as it not used for other, more important purposes.

Sven
  • 98,649
  • 14
  • 180
  • 226
  • Thanks for your comment. The server currently only uses about 65% of the available ram, so I guess everything is being cached already then right? – Mr.Boon Jan 05 '12 at 10:03
  • 1
    Yes, very likely. Since you don't currently experience performance issues, I doubt the benefit of something like Memcached would be very high. – Sven Jan 05 '12 at 10:10
2

I would guess that reading a small text file from a disk (or, for popular files, from the cached RAM) would be faster than having a conversation with memcached.

You'll have to test that, but remember that Linux very aggressively caches the files and directory structures, so most likely you are already serving most of the stuff from RAM.

Are you observing heavy, slow disk I/O at your web server?

Janne Pikkarainen
  • 31,852
  • 4
  • 58
  • 81
1

I assume you have munin to monitor server statistics (via iostat/vmstat). So you can check whether your system is using a lot of disk IO or not. If it does, then the system is not efficient in caching. You might check out Varnish, if the pages you serve are mostly static contents.

Also take a look at Linux-ftools. That's exactly what you need, when you want to tinker around system level file caching. Check it out

Jauzsika
  • 651
  • 1
  • 4
  • 10