3

I am doing a POC with Aerospike. It seems to fit all my needs except for one. When I set a TTL on a record, it expires after the TTL expires as expected, but I have a requirement where I want aerospike to notify me when a record is being evicted. Is that even possible?

Vivek Kothari
  • 462
  • 6
  • 20

3 Answers3

4

You can get the record TTL with a UDF and compare to the current time

A skeleton UDF is given here:

local function get_metadata(rec) return record.ttl(rec); end

https://discuss.aerospike.com/t/how-to-query-the-metadata-such-as-ttl-of-record-from-tools/916

I suspect this would be pretty resource heavy if you are doing it for every single record periodically but it's possible to have your application run that UDF every so often and ping a notification when the record is close to expiry.

If you are looking at a more general level you can grep the logs for NSUP (this is the subsystem that handles expiration and eviction) and there is a clear log message indicating when NSUP is removing records.

Ben Bates
  • 707
  • 3
  • 7
4

Aerospike will not do this for you out of the box. You can do it by running a periodic scan job. In the scan policy, you can set includeBinData to false. This will be a very efficient scan as it will only return the meta data of the records. In the meta data, you can check the ttl and read the full records that are about to expire.

sunil
  • 3,507
  • 18
  • 25
2

Thanks for your interest in Aerospike. How do you want Aerospike to notify you?

We have had this request for a long time, and have considered implementations. There are a number of problems in a distributed system.

For example, we could write a log file entry to a special "expiration file". Of course, this data can be lost when a server crashes, so is not redundant.

Another mechanism would be to insert onto a queue, or perform a network event ( REST call ). This also requires a distributed log to make sure notifications don't get dropped ( which Aerospike does with it's cross datacenter feature ).

Thus, I am interested in the kind of notification mechanism you'd like.

The method Ben Bates suggests is using the Scan UDF function to move the expiration into this kind of UDF scanning. Then, in the UDF, you can also notify. This should work, but you'll need to put the TTL in a bin and not use the standard TTL mechanism. This will certainly be less efficient than the standard TTL mechanism in C, but may provide what you need.

Brian Bulkowski
  • 866
  • 1
  • 7
  • 10
  • A REST call would be great. – Vivek Kothari Apr 09 '16 at 06:22
  • 1
    Please compare to Redis. You can start redis with Event notifications on and you can configure which types of events you want your Redis server to notify (like write, read, expiration, etc.,), So you can subscribe to your server and when something goes on, your event handler gets triggered. Something like this would be very useful for aerospike. I am considering using aerospike but this is a requirement for me and I cannot pay the price of making queries everytime to check what is about to expire, neither parsing logs. that is not good in perfomance and it is an amateur solution. – Cesar Villasana May 10 '16 at 18:43
  • @CesarVillasana well, someone has to check periodically whether a key is about to expire, and it's super non-optimal no matter if it's done on DB or on client side. Redis implements semi-passive way of keys expiration, which is why you only get an expiration event once a key was deleted, and not when it was expired. – Michael Spector Nov 29 '17 at 14:00
  • "super non-optimal". Not really true, Aerospike expires keys efficiently. If you read or write it's checked then, if you don't read or write, it's checked periodically. Those are efficient RAM indexes and running them isn't too bad, even when you have a few billion keys per user. Aerospike's event notification system is just entering Early Access; you should reach out to them if you're interested. – Brian Bulkowski Jul 08 '18 at 22:57