3

We have a very dynamic and high-traffic application and are using NGINX caching with dynamic URLS. We want to cache these URLs for repeated requests.

We have run into an issue a couple times while scaling where we reach the max entries in the keys_zone and then the server then starts throwing 500 errors and blank responses from the server for new URLs/files that have never been requested before. See Nginx: 'no memory in cache keys zone', despite plenty of space for reference

Example errors:

[crit] 30568#0: ngx_slab_alloc() failed: no memory in cache keys zone "PAGES"`
[alert] 27697#27697: could not allocate node in cache keys zone "PAGES"

Our cache:

proxy_cache_path /usr/share/nginx/cache levels=1:1:2 keys_zone=PAGES:50m inactive=7d max_size=900m;

Expanding keys_zone size or restarting temporarily fixes the issue. But we are looking to maintain a fixed cache that purges old unused entries versus expanding the keys_zone to support every file/URL we could serve.

Questions if anyone has context of how the NGINX inner-workings:

  1. I understand that we are creating too many keys in the keys_zone over time. If we make the max_size small enough will it stop creating new entries in keys_zone, and instead overwrite existing ones?
  2. Does the inactive param have anything too due with keys_zone? Since our inactive set pretty long, does this keep the entries in keys_zone?
  3. Is there a way to purge the keys_zone records without restarting NGINX?
dlrust
  • 715
  • 2
  • 8
  • 12

1 Answers1

0

All the documentation is here

Answers:

  1. No, the cache manager is responsible for that

The special “cache manager” process monitors the maximum cache size set by the max_size parameter. When this size is exceeded, it removes the least recently used data. The data is removed in iterations configured by manager_files, manager_threshold, and manager_sleep parameters (1.11.5). During one iteration no more than manager_files items are deleted (by default, 100). The duration of one iteration is limited by the manager_threshold parameter (by default, 200 milliseconds). Between iterations, a pause configured by the manager_sleep parameter (by default, 50 milliseconds) is made.

  1. yes

Cached data that are not accessed during the time specified by the inactive parameter get removed from the cache regardless of their freshness. By default, inactive is set to 10 minutes.

  1. I guess deleting cached files will do the trick but it's no recommended. The recommended way is to configure nginx for your needs

You should play a little with the different parameters of the proxy_cache_path directive until it feel your needs.

Suggestions:

  1. inactive=7d is probably way to high and saves unnecessary cache for a very long time. Maybe your use case requires it, but note that the default is 10 minutes.

  2. If you create more than 100 new keys in the period between the “cache manager” runs than it won't be able to keep up and you should change the manager_files and/or manager_threshold and/or manager_sleep parameters

ofirule
  • 143
  • 7