MongoDB: Capped collection findAndModify

Question

I'm trying to atomically insert an empty document if the capped collection is empty or return the last naturally sorted document if not empty. Can I do this with findAndModify?

db.collection.findAndModify({
    query: { _id: { $exists: true }},
    sort: { $natural: -1 },
    update: {},
    upsert: true,
    new: true
});

I would have expected this to either return the latest document (if the collection is non empty) or insert a new document if none exist, however, it inserts a blank document (without an _id) every single time it's called. Does findAndModify work with capped collections? I need the added document to have an _id.

Thanks.

-Scott

Same situation when performing an upsert: `db.collection.update({ _id: { $exists: true }}, {}, true);`...adds an empty object `{}` to the collection every time. — scttnlsn, Jul 28 '12 at 17:03
Of course it would your sending a empty object.....I dont mean to be mean here but get a brain cell then come back, this isnt MySQL... — Sammaye, Jul 28 '12 at 18:03
To expand on last comment since I realised I just called you a noob and walked off; allowance of empty objects is quite good and is not really a bug. If you are sending in empty objects when you don't need to then check for this empty object before you run the function. MongoDB only requires an _id which the JS driver (in console) can actually make itself on new objects. You can also take away that `new` and `upsert` flag but I suspect it is there for a reason you have them there. — Sammaye, Jul 28 '12 at 18:11
Ok. I'll try generating the `_id` in the client and changing the update to `{ _id: new ObjectId() }`. — scttnlsn, Jul 28 '12 at 18:20
Wait, you can insert a Document without _id? What Mongo version is this? — Sammaye, Jul 28 '12 at 18:31
What you can do is an empty $set `update: {$set: {}}` but you ARE right. There is something wrong here...I completely misread your question at first. I wonder what 10gen would say about this, the _id is supposed to be immutable...can you post this to mongodb-user? — Sammaye, Jul 28 '12 at 18:34
@Sammaye: capped collections have some [usage exceptions](http://www.mongodb.org/display/DOCS/Capped+Collections#CappedCollections-UsageandRestrictions) in relation to `_id`. By default capped collections do not have an index on `_id` and the `_id` can also be updated (in fact, capped collections are also allowed *not* to have an `_id` field!). If you are using replication with a capped collection, you do need to add a unique index on `_id`. — Stennie, Jul 29 '12 at 13:57

score 1 · Answer 1 · answered Jul 29 '12 at 14:36

1

I'm trying to atomically insert an empty document if the capped collection is empty or return the last naturally sorted document if not empty. Can I do this with findAndModify?

There is a flaw in your query logic. A findAndModify() with:

query: { _id: { $exists: true }},
sort: { $natural: -1 },
update: {},
upsert: true,
new: true

... will:

do an update on the last inserted record with an _id set

OR
insert a new (empty) document if no existing document with an _id is found.

The update is going to replace your last inserted record with an empty one .. which presumably is not the intended outcome :).

You are seeing a completely empty document (no _id field) because capped collections have some exceptions to the behaviour for standard collections.

In particular:

there is no requirement for an _id field by default; you can have one generated on the server by including the autoIndexId:true option to createCollection()
there is no index on the _id field (note: you will want a unique index if using replication with a capped collection)

Also note that documents in a capped collection must not grow in size or the update will fail

Refer to the Capped Collection Usage & Restrictions on the wiki for more info.

answered Jul 29 '12 at 14:36

Stennie

63,885
14
149
175

Come to think of his requirements he might actually be looking for TTL collections rather than the capped edition. – Sammaye Jul 29 '12 at 14:46
@Sammaye: Whether TTL or capped, the use case for inserting an empty document with an ObjectID (versus more meaningful data) isn't clear to me yet. Could also be a case for values that may be better tracked in a document instead of a collection .. perhaps using [$exists](http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24exists) to work out if a field is set. The OP would have to clarify the intended outcome given this findAndModify() isn't the right approach. – Stennie Jul 29 '12 at 15:06
Using `update: { _id: new ObjectId() }` is doing the trick. If there's already a doc with an `_id` then the `findAndModify` does nothing (presumably because we can't change a doc in a capped collection). – scttnlsn Jul 30 '12 at 15:23
@scttnlsn: You can change a doc in a capped collection, you just can't *grow* it. I suspect your update doesn't do what you think it does because the semantics will still be as described above. If there is a doc with an _id in your capped collection and your query is on $exists:true, you are likely changing this to a new ObjectID(). – Stennie Jul 30 '12 at 21:30
@Stennie: Perhaps `update: { _id: new ObjectId(), foo: 'bar' }` will do the trick then. I just need to atomically ensure that there's at least a single document in the collection (but I don't care what it is) because a tailable cursor will be immediately closed if the collection is empty. – scttnlsn Jul 30 '12 at 22:25
@scttnlsn: if it's for a tailable cursor and a doc is required, I would just do an `insert` before you start tailing .. should be a more predictable outcome. What driver and version are you using? It may be a bug that the cursor is closed if the connection is empty ;) – Stennie Aug 01 '12 at 11:40
@Stennie: I agree- it's probably better to just insert a document before hand. The findAndModify seems hairy when used like this. I'm using the node-mongodb-native driver. – scttnlsn Aug 01 '12 at 15:04
@Stennie: I was hoping to have the insertion of that document abstracted away (before opening the tailable cursor) but also needed it to be atomic in case there are multiple clients. It's probably better to make this an explicit step that must be taken before opening _any_ cursors. – scttnlsn Aug 01 '12 at 15:48

MongoDB: Capped collection findAndModify

1 Answers1