When does node-mongodb-native hits the database?

Question

I have trouble understanding when exactly the database is hit when using node-mongodb-native. Couldn't find any reference on that. As everything is callback based, it gave me the feeling that every single call hits the database ... For example, are those two snippets any different in terms of how many times the database is hit :

// ---- 1
db.collection('bla', function(err, coll) {
    coll.findOne({'blo': 'bli'}, function(err, doc) {
        coll.count(function(err, count) {
            console.log(doc, count)
        })
    })
})

// ---- 2
db.collection('bla', function(err, coll) {
    coll.findOne({'blo': 'bli'}, function(err, doc) {
        db.collection('bla', function(err, coll) {
            coll.count(function(err, count) {
                console.log(doc, count)
            })
        })
    })
})

I was basically wondering whether I can cache instances of collections and cursors. For example, why not fetch the collections I need only once, at server start, and reuse the same instances indefinitely ?

I'd really like to understand how the whole thing work, so I'd really appreciate a good link explaining stuff in details.

`coll.count(function(err, count) {` will hit the DB multiple times, MOngoDB cannot say that from one second to the next that the count of the two queries is the same — Sammaye, Feb 13 '13 at 09:56
Yes ... `count` I kind of guessed it hits the db at least once, but calls that don't really fetch a result, just returning an instance of something, it is kind of hard to guess. — sebpiq, Feb 13 '13 at 10:09
Just because it doesn't fetch a result doesn't mean it does not run a query. Since you are not iterating the `find` the only queries performed there should be on the count. The query for the `find` will run on the count but not before hand — Sammaye, Feb 13 '13 at 10:11
Oops, I made a mistake, I meant `findOne`. I believe here the `find` will never be run since `count` is on the collection and not the cursor. — sebpiq, Feb 13 '13 at 10:19
But, from what you say, sounds like queries are kind of cached until you actually ask for a result, then they are run at once. So why all those callbacks ??? Couldn't it be just `db.collection('bla').find({'blo': 'bli'}).count(function(err, count) { /*blabla*/ })` ? — sebpiq, Feb 13 '13 at 10:21
Ah yea I didn't notice that, my bad. Ok yea the findone will run once and then the count will come from the collection meta which is still a query technically cos it does go to the server, so I would say you are looking at 4 queries for those snippets together. No, queries are not cached, it is the way the cursor works in client side drivers. They don't actually perform the query until you iterate the results. As for the callbacks, I think that is just a dynamic of node.js — Sammaye, Feb 13 '13 at 10:23
OK ... so there's actually no problem in, for example, creating a cursor at server start and reusing it indefínitely (maybe using rewind) ? Or a collection ? And if not, do you know why in all the snippets you can find, and in the docs there's no mention of all this ? — sebpiq, Feb 13 '13 at 10:29
Ah I am wrong, the callbacks are because node.js queries the database physically each time. So you are looking at 3 queries for the first snippet and then 4 for the second, of course the calls are async so you have callbacks for them. — Sammaye, Feb 13 '13 at 10:30
Do you have a reference, in the docs or elsewhere where I can read about that? — sebpiq, Feb 13 '13 at 10:34
Yea so long as you are using that cursor with the query you inited it with it should be fine, as shown here: https://github.com/mongodb/node-mongodb-native/blob/master/Readme.md#find where they use rewind etc to do this. And also yes, I cannot see any problem reusing collection objects either — Sammaye, Feb 13 '13 at 10:34
One thing that tipped me off about what I said about node.js querying the db each time was that it support strict mode: https://github.com/mongodb/node-mongodb-native/blob/master/Readme.md#strict-mode where by it forces a collection to exist when you create it in the driver. The only way it can do that is to ping MongoDB directly, I will look around some more on the subject — Sammaye, Feb 13 '13 at 10:35
Ok thanks! If you have enough material, maybe you could write a short answer as well :) — sebpiq, Feb 13 '13 at 10:38
Ok I actually looked in the source code here: https://github.com/mongodb/node-mongodb-native/blob/master/lib/mongodb/db.js#L446 It will ping the server if you have strict on but not if you don't it will just init the callback. Also find doesn't seem to immediateley ping MongoDB: https://github.com/mongodb/node-mongodb-native/blob/master/lib/mongodb/collection.js#L790 so what I saw must have been specific to strict mode — Sammaye, Feb 13 '13 at 10:42

score 1 · Answer 1 · answered Feb 13 '13 at 10:50

Looking at the source code for the node.js driver for collection it seems it will not ping MongoDB upon creation of the collection unless you have strict mode on: https://github.com/mongodb/node-mongodb-native/blob/master/Readme.md#strict-mode

The source code I looked at ( https://github.com/mongodb/node-mongodb-native/blob/master/lib/mongodb/db.js#L446 ) reinforced the idea that if strict was not on then it would just try and create a new node.js collection object and run the callback.

However findOne and count will break the "lazy" querying of node.js and will force it to query the database in order to get results.

Note: The count being on the collection won't enforce a "true" count of all items in the collection. Instead it will garnish this information from the collection meta.

So for the first snippet you should see two queries run. One for the findOne and one for the count and two for the second snippet as well since creating the collection after the findOne should not enforce a query to MongoDB.

score 1 · Answer 2 · answered Feb 13 '13 at 12:07

After some googling, I have find this link about best practices for node-mongodb-native. It is answered by Christian Kvalheim who seem to be the maintainer of the library. He says :

"You can safely store the collection objects if you wish and reuse them"

So even if the call to collection might hit the database in case it is made in strict mode, the actual client-side collection instance can be reused.

When does node-mongodb-native hits the database?

2 Answers2