12

I have been thinking about using the build in TTL feature, but it's not easy to dynamically changing the expiration date. Since mongodb is using a background task purging the data. Is there any downside just coding my own purging function based on "> certain_date" and run say once a day? This way, I can dynamically changing the TTL value, and this date field won't have to be single indexed. I can reuse this field as part of the complex indexing to minimize number of indexes.

tshepang
  • 12,111
  • 21
  • 91
  • 136
user2833162
  • 149
  • 1
  • 7
  • why do you say it's not easy to dynamically change the expiration date? See Daniel's answer - it's actually quite simple. – Asya Kamsky Feb 01 '14 at 07:07

2 Answers2

20

There are 2 ways to set the expiration date on a TTL collection:

  1. at a global level, when creating the index
  2. per document, as a field in the document

Those modes are exclusive.

Global expiry

If you want all your documents to expire 3 months after creation, use the first mode by creating the index like the following:

db.events.ensureIndex({ "createdAt": 1 }, { expireAfterSeconds: 7776000 })

If you later decide to change the expiry to "4 months", you just need to update the expireAfterSeconds value using the collMod command:

db.runCommand({"collMod" : "events" , "index" : { "keyPattern" : {"createdAt" : 1 } , "expireAfterSeconds" : 10368000 } })

Per-document expiry

If you want to have every document has its own expiration date, save the specific date in a field like "expiresAt", then index your collection with:

db.events.ensureIndex({ "expiresAt": 1 }, { expireAfterSeconds: 0 })
Daniel Coupal
  • 815
  • 6
  • 8
  • 1
    Thanks for the info. I should clarify. I didn't mean it's not easy. I mean it's expensive to touch every records if I decided to change the duration. I have million of records. The mongo is already taking up lots of CPU time whenever I delete records (with index). It seems to me that the internal implementation is not friendly for use case where changing expiration is the norm. – user2833162 Feb 01 '14 at 22:22
  • 3
    These aren't two different 'modes' that are mutually exclusive. This is the same technique applied with different semantics. One is relative to the created date, the other requires setting an expiry beforehand, and of course those two can be combined by setting a time delta on the index and applying it to a date field with 'expires' semantics. – mnemosyn Feb 03 '14 at 09:02
  • Is it possible to run the collMod command in the background? Whether or not that's possible, what impact does the command have on the collection while it's being executed? I have a rather large collection (>250m documents) for which I'd like to reduce the TTL from 60 days to 30 days. – Matt Passell Dec 15 '15 at 17:58
5

I have been thinking about using the build in TTL feature, but it's not easy to dynamically changing the expiration date

That's odd. Why would that be a problem? If your document has a field Expires, you can update that field at any time to dynamically prolong or shorten the life of the document.

Is there any downside just coding my own purging function based on "> certain_date" and run say once a day?

  1. You have to code, document and maintain it
  2. Deleting a whole lot of documents can be expensive and lead to a lot of re-ordering. It's probably helpful to run the purging more often

Minimizing the number of indexes is a good thing, but the question is whether it's really worth the effort. Only you can give an answer to this question. My advice is: start with something that's already there if any possible and come up with something better if and only if you really have to.

mnemosyn
  • 45,391
  • 6
  • 76
  • 82
  • 1
    Thanks. so it sounds like there's no real benefit. Changing the expires field seems expensive. What I need is to have a way to adjust how long too keep the documents. So let's say it's configured to be 3 months now. But I want to change it to be 4 months. There's no reason to touch every document in my opinion. If I maintain my own purging routine, then my query criteria changes to "> 4 months". This is where my question comes from. Using TTL feature will require me to touch every document in the collection. But implementing it myself will not. It seems counter intuitive. – user2833162 Oct 01 '13 at 18:19
  • 1
    why would you have to change the date on the documents if you use TTL? – Asya Kamsky Jan 31 '14 at 19:47