1

I am running an Nginx server to serve video content to users. This nginx server picks content from an upstream server, caches it, also serves to users. All future requests are served from this cache. I have set aside 500GB of space on cache for this. When the cache is full, the nginx cache manager is able to delete old, unused files as per my proxy_cache directive if the cache folder is on the hard drive. When I mount this cache folder on the RAM (tmpfs), nginx is unable to drop the old files. I get an error saying '28: No space left on device'. I have checked the permission of the cached folder both while on the RAM and on the hard drive. It has the same permissions. 

proxy_cache_path    /cache/12054 keys_zone=a12054:100m levels=1:2 max_size=500g inactive=7d;

If I unmount from RAM it starts working fine again.

Manjunath Bhat
  • 11
  • 3
  • after mounting on tmpfs, no content has completed 7 days yet. But my 500 GB of storage is full. So I want the earliest content to be deleted and new file to use that space. – Manjunath Bhat Aug 16 '16 at 07:28

1 Answers1

2

You have too little RAM. You need 500 GB of virtual RAM (physical RAM + swap space) for that configuration to work in tmpfs. Nginx will not try to purge files until they expire or you have reached max_size. From the manual:

The special “cache manager” process monitors the maximum cache size set by the max_size parameter. When this size is exceeded, it removes the least recently used data.

Confirm how large your tmpfs partition is by running df -h.

You might not need tmpfs for this at all. If you are running a modern operating system all free RAM should automatically be used as a disk cache anyway. Just write the files to disk. They will have to be written sooner or later anyway so it might be more efficient to write them directly instead of waiting for them to be swapped out.

Emil Vikström
  • 90,431
  • 16
  • 141
  • 175
  • I have 500 GB of RAM set aside for this directory. My df -f output: ' tmpfs 500G 500G 0 100% /cache/12054 ' I have reached max_size on my tmpfs. Once I hit the max_size nginx is unable to remove the least used content from tmpfs which has not yet reached both inactive and proxy_cache_valid. nginx removes files which have crossed either inactive and proxy_cache_valid. – Manjunath Bhat Aug 16 '16 at 08:43
  • Also, I do not have that much memory on disk, but have enough space on RAM. – Manjunath Bhat Aug 16 '16 at 09:00
  • 1
    What happens if you shrink the Nginx max_size to, say, 450 GB? – Emil Vikström Aug 16 '16 at 09:15
  • 1
    It worked. I increased the mount size of tmpfs to be greater than my max_size. Now nginx is able to purge out files as and when required. Is there any thumb-rule for how much extra space I should give on tmpfs? Thanks a ton. Saved my day. – Manjunath Bhat Aug 16 '16 at 10:16