1

We are having below aerospike set connfiguration and want that entry from the Map should be auto deleted after 'x' days. How should the same be achieved using Spring data aerospike ?

@Document(collection = "cust")
public class Customer {

   @Id
   @Field(value = "PK")
   private String custId;

   @Field(value = "mobileNumber")
   private String mobileNumber;

   @Field(value = "creationTime")
   private String creationTime;
 
   @Field(value = "corrDetails")
   private HashMap<String, Object> corrDetails;
 
}

Moreover there are around 10 - 12 <K,V> pairs in my map with each having different structure, and I want only 1 Particular <K,V> to expire after x days !!

Dependencies looks like :-

<dependency>
            <groupId>com.aerospike</groupId>
            <artifactId>aerospike-client</artifactId>
            <version>4.1.3</version>
        </dependency>

        <dependency>
            <groupId>com.aerospike</groupId>
            <artifactId>spring-data-aerospike</artifactId>
            <version>${aerospike.data.version}</version>
            <scope>system</scope>
            <systemPath>${basedir}/lib/spring-data-aerospike-2.0.0.RELEASE.jar</systemPath>
        </dependency>

        <dependency>
            <groupId>com.aerospike</groupId>
            <artifactId>aerospike-client</artifactId>
        </dependency>
        <dependency>
            <groupId>com.aerospike</groupId>
            <artifactId>aerospike-helper-java</artifactId>
            <version>1.2.2</version>
        </dependency>
Aditya Goel
  • 201
  • 1
  • 15

2 Answers2

3

There is no native feature/ability within Aerospike server to assign a TTL to a K:V pair in a map data type. You will have to build it from the application. i.e. store the desired expiration timestamp when you insert the K:V pair, with the K:V data, and then periodically scan the entire database to remove expired KV pairs or remove expired K:V pairs on reads (less intensive but carries expired K:V pairs until read) - no elegant option really. Perhaps revisit the data model.

Aerospike does have the ability to expire and remove entire records i.e. assign TTL to an entire record - and then it has an internal thread to do the expiration and removal for you. So if you can store this K:V pair that you want to keep for x days as a separate record, then that is straightforward. But anything at bin level or at bin data level (like map data type), you have to handle from the application.

pgupta
  • 5,130
  • 11
  • 8
  • Thank you for answering to this post. Yeah we realised the same. We don't have fixed model for the K,V pair that we are gonna inputting to the Map. For different K,V pairs, this can be different structure. But was just curious and wondering, if I had to define & fix the structure of my SET upfront (i.e. at the beginning of the project), does not that defeat the purpose of In-memory data structure like AS !!? – Aditya Goel Jul 22 '20 at 15:40
  • Aerospike is not a sub-set of structured data store - its the other way around. You can store both structured - predefined data model or unstructured data - just the same. i.e. you can have every record in a set in aerospike have the same data structure or each record can have a different data structure - Aerospike is record centric - not "table or set" centric. SET is just a metadata on a record in aerospike and not a data structure definition. – pgupta Jul 22 '20 at 16:57
  • Thanks for your response. But question is, How do I maintain the "unstructured data" into the aerospike ? Say I have 5 records of different structure, How would they be mapped to a single SET format ? Do you mean, we should define the "Object" type of field in java and then leave it on whatever data is arriving into it ? This is what exactly we want to achieve i.e. We must NOT be mandated to specify the structure of the set at the very beginning, as you see above in my question we are defining the definition of the set. Any references to examples shall be greatly helpful ! – Aditya Goel Jul 28 '20 at 10:59
  • Again, in Aerospike, set name is just a tag on a record. So data in Aerospike across records for example can be like so: – pgupta Jul 28 '20 at 18:20
0

Again, in Aerospike, set name is just a tag on a record. So data in Aerospike across records for example can be like so:

  1. Rec Key=1, data: bin1=5, bin2="abc" and you can specify (optional) -> belongs to "myset1",
  2. Rec Key=2, data: bin1="cdg", bin2=7 and you can specify -> belongs to "myset1",
  3. Rec Key="abc" data: bin1=[list data type], bin2={map data type}, bin3="xyz", bin4=4 and you specify -> belongs to "myset2"

... hope you get the drift. Data schema is purely at the record level, Set name is just a modeling convenience - if you don't specify set, record goes to default "null" set - set name is attached as a "tag" to that particular record. You can then work on all records whose tag (set name) is "myset1". But within Aerospike, all records belonging to different sets in the same namespace are distributed randomly on the storage space for that namespace.

pgupta
  • 5,130
  • 11
  • 8