As mentioned in the observation, we are getting evictions when we use set() operation for updating a known key. The frequency we update our key is really unpredictable and very high. You can relate our case to some kind of lock implementation for a contending resource.
After running experiments with some keys, which are highly frequently getting updated. We observed replace() is not causing any evictions, but set() does cause evictions.
After going through memcache docs here and here, came to a conclusion that
- set() operation always invokes a memory allocation, irrespective. This is causing the evictions on slab where this key is allocated.
- replace() operation is not doing any memory allocation.
Hence, for the question of 'update a frequently existing key' replace is better option at-least for our use case. It helped us avoid evictions.