mongodb TTL not removing documents

Question

I have a simple schema like:

{
    _id: String,      // auto generated
    key: String,      // there is a unique index on this field
    timestamp: Date() // set to current time
}

Then I set the TTL index like so:

db.sess.ensureIndex( { "timestamp": 1 }, { expireAfterSeconds: 3600 } )

I expect the record to removed after 1 hour but it is never removed. I flipped on verbose logging and I see the TTLMonitor running:

Tue Sep 10 10:42:37.081 [TTLMonitor] TTL: { timestamp: 1.0 } { timestamp: { $lt: new Date(1378823557081) } }
Tue Sep 10 10:42:37.081 [TTLMonitor] TTL deleted: 0

When I run that query myself I see all my expired records coming back:

db.sess.find({ timestamp: { $lt: new Date(1378823557081) }})

...

Any ideas? I'm stumped.

EDIT - Example document below

{ "_id" : "3971446b45e640fdb30ebb3d58663807", "key" : "6XTHYKG7XBTQE9MJH8", "timestamp" : ISODate("2013-09-09T18:54:28Z") }

score 5 · Accepted Answer · answered Sep 10 '13 at 20:35

5

Can you show us what the inserted records actually look like?
How long is "never"? Because there's a big warning:

Warning: The TTL index does not guarantee that expired data will be deleted immediately. There may be a delay between the time a document expires and the time that MongoDB removes the document from the database.
Does the timestamp field have an index already?

answered Sep 10 '13 at 20:35

jcollum

43,623
55
191
321

3

The TTLMonitor runs every 60 seconds so I presume it should be getting picked up in at least 1 minute of the expiration. By never I mean never (e.g. days pass and nothing is ever deleted). There is only 2 indexes - the TTL index and the unique index I pointed out on the key field. Example document in the edit. Thanks for your help. – jckdnk111 Sep 10 '13 at 20:44
3

No I'm wondering what you see when you do `db.sess.find().pretty()` -- I just want to confirm the timestamp field is there and doesn't look funny. For the indexes, you should do `db.sess.getIndexes()` to confirm that's all the indexes that are there. – jcollum Sep 10 '13 at 20:49
`db.sess.getIndexes()` showed me the error of my ways. I was using `db['system.indexes'].find({expireAfterSeconds: {$ne: null}})` -- I had a typo in the collection name while creating the index and I didn't notice it in the output of my `system.indexes` query. Thanks so much for your help! – jckdnk111 Sep 10 '13 at 20:55
I'm using Node.js mongoose and it seems sometimes indexing just doesn't get through. No code changes. Just repeating executions and it has indexing sometimes and sometimes not. Weird. – huggie Sep 11 '14 at 05:03
@huggie you should post that as a separate question – jcollum Sep 11 '14 at 15:25
@jcollum: I don't mean it as a question. I just want people to watch out for it. – huggie Sep 12 '14 at 00:49

score 3 · Answer 2 · answered Jul 27 '16 at 19:54

This was my issue: I had the index created wrong like this:

{
    "v" : 1,
    "key" : {
        "columnName" : 1,
        "expireAfterSeconds" : 172800
    },
    "name" : "columnName_1_expireAfterSeconds_172800",
    "ns" : "dbName.collectionName"
}

When it should have been this: (expireAfterSeconds is a top level propery)

{
    "v" : 1,
    "key" : {
        "columnName" : 1
    },
    "expireAfterSeconds" : 172800,
    "name" : "columnName_1_expireAfterSeconds_172800",
    "ns" : "dbName.collectionName"
}

mongodb TTL not removing documents

2 Answers2

Linked

Related