1

I currently have a chat bot setup to store records onto MongoDB. The object that's stored in Mongo looks like... 

{ ..., expiration_time: 12451525, ... }

Where expiration_time is a number represented in minutes.

My initial approach was use setInterval on the web application to query the database to delete all records that matched the criteria of current time being greater or equal to expiration time. However, I feel like there would be a lot of additional queries to the database from the web application, when there are already other operations such as reading and writing data.

I read about storing functions onto Mongo, but I'm sure how to automate the process of invoking the function to self delete the records.

I would definitely love any feedback, approach, or guidelines for best practices.

Thanks in advance!

mralanlee
  • 479
  • 5
  • 15
  • 1
    there is a built-in mechanism for this purpose in MongoDB called [TTL index](https://docs.mongodb.com/manual/tutorial/expire-data/). – Andrew Aug 16 '17 at 06:49

2 Answers2

1

IMHO there are 2 options:

  1. One can use a cronjob to remove the out-of-date entries
  2. One can use the Capped Collections. It's like a ring buffer, so that the oldest entry will be overwritten. Here one must choose the right fix-size of the capped Collections. I.e, size = 24 * 60 = 1440 if the chat bot writes every minute to the collection.
Minh Tuan Nguyen
  • 1,026
  • 8
  • 13
1

You can use a TTL index: 

db.messages.createIndex( { "expiration_time": 1 }, { expireAfterSeconds: 0 } )

The only requirement is that expirtion_time has to be a date instead of an integer.

To expire documents at a specific clock time, begin by creating a TTL index on a field that holds values of BSON date type

Lukasz Wiktor
  • 19,644
  • 5
  • 69
  • 82
  • Had a question, it seems that this only works if I add new records in. It does not seem to self-expire on it's own. – mralanlee Aug 17 '17 at 23:59
  • @user3593810 Have you converted the `expiration_time` to date for the old documents? Also take a look at https://stackoverflow.com/questions/18727743/mongodb-ttl-not-removing-documents – Lukasz Wiktor Aug 18 '17 at 09:15