0

I am using the FUSE file system fuse.py and plan to use memcache to store the files. I want to have a small amount of memcache space so that I know what files are being stored and what files are being replaced.

I understand that Memcache has slabs which stores data of that particular size. But is it possible to have only one slab? For example have one slab of 5mb, with each page size 1mb so that I can store maximum of 5 files. And if I need to add the 6th one the LRU policy will remove a file and make space for it.

Basically I want to use memcache as a cache for files and once a file is removed from memcache, store the change in db server.

TheFallenOne
  • 1,598
  • 2
  • 23
  • 57
  • 2
    this is impossible because caches can expire at any time. – Zig Mandel Oct 18 '15 at 00:11
  • 1
    In addition none of the capabilities for managing page size exist in the appengine memcache service. Why you would want to add another layer (fuse) for db entities is hard to understand. Also if you use ndb the caching layer is done for you when you use `key.get()` – Tim Hoffman Oct 18 '15 at 00:57

1 Answers1

0

Don't make assumptions on the implementation of a system based on how this or other systems might be implemented; the only thing you can rely on is what's in the published API documentation, and in this case, Google App Engine does not provide a callback when entries expire (which will either expire when the entry reaches its TTL or is replaced due to LRU), nor does it back them up to other storage systems.

You shouldn't use memcache for durable storage such as a file system; please consider one of the following systems instead for durable storage:

You can, of course, cache their data in Memcache, but since it can expire, you should store the originals in a more durable storage system, and cache the frequently-accessed chunks/files/metadata in Memcache.

Misha Brukman
  • 12,938
  • 4
  • 61
  • 78