Copied MongoDb from cloud9 to Mlab and new documents are saved in odd order to DB

Question

I am trying to use databases on a website for the first time and running into something odd. I did a webDev course that showed how to create a blog type site that use cloud9 to do all the development, and then deployed the finished site to heroku using a MongoDb databse and using Mlab to host the database once you have the site on heroku.

Everything works out great, however I added a bunch of collections and documents to the database while it was still on cloud9 and the instructor in the webDev course never went into how to move this database to Mlab. I found some info and used the mongodump command to save out the entire database and on mlab found this code, I used this basic command to get the mongodump onto mlab:

  mongorestore -h ds153637.mlab.com:53637 -d nttw_db -c travels -u <user> -p <password> travels.bson

The database seems to have transferred fine, and my site when testing it on heroku, seems to have all the data I initially stored in the DB when it was created on cloud9. The problem I run into is now when I add to the database on the live deployed site hosted on heroku, database entries are being added to the collections in odd orders on Mlab. This is a blog type site with hiking trails and campgrounds and reviews on gear, so when i added a new test entry to the trails, the new document was added after the first entry in the collection, but before the other 4 entries. Usually the new data I enter gets appended to the end of the collection, at least that's how it worked when I was adding entries when the DB was on cloud9. Here is the code that is used in my app.js on the server to add a new entry posting from the form on my site:

router.post("/trail_new", isLoggedIn, function(req, res){

    var name = req.body.trail.name;
    var tags = req.body.trail.tags;
    var splash = req.body.trail.splashText;

    var thl     = req.body.trail.thl || "somewhere on earth.!";
    var fees    = req.body.trail.fees;
    var tl      = req.body.trail.tl;
    var info    = req.body.trail.info;

    var body = req.body.trail.body;
    var adRight = req.body.trail.adRight || "none";
    var photos = req.body.reviewPhoto;

    tags = tags.split(",");

    //Assemble a variable to send all the review data into the create function for the SCHEMA pattern
    var newTrail = { name: name, tags: tags, splashText: splash, thl: thl, fees: fees, tl: tl, info: info, body: body, adRight: adRight, photos: photos };

    Trail.create( newTrail, function(err, travel){ 
        if(err){
            console.log(err);
            res.redirect("trail/new");
        }else{
            console.log(travel);
            res.redirect("/trail");
        }
    });
});

I am sure there are some less-than-elegant lines in the above code, but it all appears to work fine for adding entries into the database. Is there any thing I am overlooking about how Mlab adds to the database.? I don't know why new entries are being inserted into the db after the first entry in each collection and before the other ones that were all there before copying the mongodump over to Mlab. The version of mongo running on the cloud9 development environment is v3.6.9 and the Mlab version is 3.6.6 (MMAPv1). Not sure if this matters or not.

I would really hate to have to recreate all the entires I have already made in the database from when it was on cloud9, but If I have to wipe the database and start over with it on Mlab, I can do it. It may not even be an issue, but It worries me slightly that the new entries are not being put into the DB in the correct order and don't want it to somehow corrupt the database down the road if there is an issue that I don't know about.

Any information would be great. This is the first attempt at using databases and they are so cool, but confusing for a first timer.!

Thanks

E

Unless you specify an explicit sort order (or use a capped collection) the return order of documents is not guaranteed. You haven't included a code snippet showing how you find entries, but that should include `sort()` criteria. Without sort criteria the natural order may coincidentally appear to be insertion order, but is definitely not guaranteed to remain so as documents are modified or deleted. — Stennie, Feb 02 '19 at 00:22

score 0 · Accepted Answer · answered Feb 01 '19 at 05:41

0

The key here that you need to know IMO is that in MongoDB, the data is stored as a big JSON blob. And JSON by definition is a loosely bounded blob of key-value pairs. The order of objects doesn't matter in JSON and you should keep this in mind. So when you dump your data in C9 & upload in Mlab, neither one cares about the order of JSON objects and you see the order messed up. Think of this particular point as a slight drawback of using the NOSQL databases and move on.

P.S Did you take the Udemy Bootcamp course by Steele?

answered Feb 01 '19 at 05:41

shiv

165
1
20

Ah Ok, it surprisingly keeps the order of all the entires when I bring the database into Mlab, but new additions are then not "in order" when they get added into the db on Mlab. Yes, that is the bootcamp I did on udemy. It was my first exposure to web dev and I thought it was very good, but I jumped the gun once the course showed how to setup the mongoDB in cloud9 and started populating the database and then towards the end when he showed how to setup the Mlab db, didn't mention anything about transferring your db from cloud9 to Mlab. – E-Bling Feb 01 '19 at 17:42
Yeah man. I faced the same issue when I migrated my DB. The order of entries doesn't matter so don't sweat over it. Also, accept the answer if it helped :) – shiv Feb 01 '19 at 18:05
Done.! I was looking at the database entries in Mlab, which is cool and nicer than looking at them in the mongo console, and the new entries I make look identical to the entries that were made when the DB was in cloud9. So I guess it's no issue. Thanks. -E – E-Bling Feb 01 '19 at 18:11
1

Actually, MongoDB does *not* store data as JSON blobs, although JSON is a convenient way to think about serialisation from a client point of view. MongoDB's underlying format for storage and wire protocol is [BSON](http://bsonspec.org/), which is a JSON-like binary format that supports additional data types. For some more context on JSON vs BSON, see: [Why does MongoDB check for order of keys when matching embedded documents?](https://softwareengineering.stackexchange.com/questions/319468/). The order of keys in documents isn't related to how those documents are ordered at the storage layer. – Stennie Feb 02 '19 at 00:34
1

@Stennie the link you shared has one of the best articles I've ever read! Thank you. – shiv Feb 05 '19 at 14:43

Copied MongoDb from cloud9 to Mlab and new documents are saved in odd order to DB

1 Answers1