3

Aerospike is blazingly fast and reliable, but expensive. The cost, for us, is based on the amount of data stored.

We'd like the ability to query records based on their upsert time. Currently, when we add or update a record, we set a bin to the current epoch time and can run scan queries on this bin.

It just occurred to me that Aerospike knows when to expire a record based on when it was upserted, and since we can query the TTL value from the record metadata via a simple UDF, it might be possible to infer the upsert time for records with a TTL. We're effectively using space to store a value that's already known.

Is it possible to access record creation or expiry time, via UDF, without explicitly storing it?

Alex Woolford
  • 4,433
  • 11
  • 47
  • 80

2 Answers2

4

At this point, Aerospike only stores the void time along with the record (the time when the record expires). So the upsert time is unfortunately not available. Stay tuned, though, as I heard there were some plans to have some new features that may help you. (I am part of Aerospike's OPS/Support team).

Meher
  • 2,804
  • 9
  • 7
  • 1
    Thanks @Meher. If we know the void time and the TTL we should be able to infer the upsert time providing the TTL is not 0 (i.e. "forever"). So I'm curious to know if it's possible to access the void time via a UDF? void - TTL = upsert; where TTL > 0. – Alex Woolford Jan 28 '16 at 23:16
1

void time : This tracks the life of a key in system. This is the time at which key should expire and is used by eviction subsystem.

so ttl is derived from the void time.

As we get ttl from a record, we can only calculate the void time (now + ttl)

Based on what you have, I think you can evaluate the upsert time from ttl only if you add same amount of expiration to all your records, say CONSTANT_EXPIRATION_TIME.

in that case 

upsert_time = now - (CONSTANT_EXPIRATION_TIME - ttl)

HTH

Amod Pandey
  • 1,336
  • 3
  • 14
  • 22
  • Thanks @AmodPandey. I appreciate the response. The goal is to minimize the amount of data stored. We could certainly store the upsert time in a bin, but that means we're effectively storing data that already exists in Aerospike (for TTL > 0). The part that's missing is the void time for records that already exist: Aerospike stores this but, looking through the documentation, it's not obvious how that information can be accessed. – Alex Woolford Jan 29 '16 at 16:58
  • Hey @AlexWoolford, Aerospike does not store both void time and TTL. It only stores the void time (time at which the record should expire). Say I add a record and set expiration to 1 hour and the record was inserted at 4PM. Aeropsike will store the void time as 5PM and that is it. The evict system when run at 5 PM will clear the index (to remove the object). – Amod Pandey Jan 29 '16 at 20:37
  • So, if you are adding same amount of expiration every time you create/update an object (say one hour, call it CONSTANT_EXPIRATION_TIME). then upsert_time = Current time - (CONSTANT_EXPIRATION_TIME - TTL). TTL is obtained from record. HTH – Amod Pandey Jan 29 '16 at 20:44
  • Thanks again, @AmodPandey. Since current time isn't a constant we'd need to record it. This means that we're recording a value that can already be derived from data that's inside Aerospike (for TTL > 0). An alternative title to this question could be "how can I retrieve the void time for a record?" Do you know if that's possible? – Alex Woolford Jan 29 '16 at 21:26
  • I think I am not helping here. But I will give a last shot. Java - long now = System.currentTimeMillis() / 1000; long voidTime = now + record.getTimeToLive() // record.expiration gives to time from seconds from Jan 01 2010 00:00:00 GMT Lua local void_time = os.time() + record.ttl(topRecord) -- where topRecord is the first argument of the record udf – Amod Pandey Jan 29 '16 at 21:42
  • @AmodPandey has a workable solution. I'll try to simplify the explanation: (1) When inserting/upserting a record, set its TTL value to 1000 years from the time of input. So, if inserting on Jan. 1, 2016, 12:00am, set its TTL value to Jan. 1, 3016, 12:00am. (2) Now, whenever you retrieve a record, also get its TTL value. Subtract 1000 years from that, and you'll have your insert time. Of course, this means you lose TTL functionality. And it's a bit hack-ish and prone to bugs. – RavenMan Feb 01 '16 at 07:39