-3

I’ve been developing a Java based Enterprise application on Linux platform whose one of the functions is to generate/store a record whenever there’s a request (https) and delete the record after an expiration period. This application may have to maintain as many as 200K records (which expire at different times) at any given time. Is launching a Timer at the time of each record creation and deleting the record in the Timer callback (when the timer expires) the best solution to delete each record at its expiration ? If it is the best solution, can as many as 200K Timer instances be running at a given time ? If it’s not the best solution, what’re the possible alternatives ?

Thanks

Sathish Rao
  • 75
  • 11
  • 4
    Before re-inventing the wheel, you should check existing cache implementations where there is time-based expiration. E.g. Guava Caches etc. – RealSkeptic Dec 25 '18 at 16:57
  • Thanks for the suggestion on expiration time. I’m currently exploring this option. – Sathish Rao Dec 26 '18 at 16:24

2 Answers2

3

Is launching a Timer at the time of each record creation and deleting the record in the Timer callback (when the timer expires) the best solution to delete each record at its expiration ?

Almost certainly NO! Instead, run a single background task which periodically (e.g. once every five seconds) searches for expired records and deletes instead. Which is essentially what a mark and sweep garbage collector (like Java uses) does.

Elliott Frisch
  • 198,278
  • 20
  • 158
  • 249
  • Thanks. I’m not sure about the performance of the background task if it has to scan through 200K+ records periodically. I guess I’ll have to explore this option further. – Sathish Rao Dec 26 '18 at 16:22
  • Better than the performance if you have 200K+ threads waiting to execute and blocking each other (assuming you can even get 200K+ threads to run at once). – Elliott Frisch Dec 26 '18 at 17:16
0

No, a queue is the way to go. It is not as real-time (maybe) as theoretical timers. But is scales better, and gives better control. Simple message queues are usable.

Maybe doing one periodic task on the database would be feasible for simple cases; DELETE ... WHERE create_time < ?.

Joop Eggen
  • 107,315
  • 7
  • 83
  • 138
  • Thanks for the suggestion on Message queues. Regarding periodic task, I’m not sure about the real-time performance if it has to scan through 100k+ records, say once every 5 seconds. – Sathish Rao Dec 26 '18 at 16:19
  • I would not take an SQL DELETE-on-indexed-date out of consideration. Test it (benchmark times should be linear to the number of records). Also possible DB technologies like table partitioning (on the date field, Oracle has it commercially) can improve times. – Joop Eggen Dec 27 '18 at 09:05