How to import json into MongoDB using Mongoose

Question

I have a few problems with this, which is whats making it tricky, so...

I am using Mongoose and MongoLab, I can store data and retrieve it just fine, but I want a system that allows me to do a base seed of the database.

I have the schemas created for the collections, but none are ran because there is no data, so I can't seem to run a normal mongoimport as the collection isn't yet created.

I want to add something to my node server so that if the collection doesn't exist or is empty, it loads a schema for a collection and then inserts the json for the seed data.

so I have this...

var Club = require('./schemas/Club');

I normally use Club.find, or Club.save etc, thats working fine.

I want to just run a save against an array of Objects to the Club collection which it needs to create.

I did look into mongoose-fixture but its not been updated in years, and there is probably a way of doing this without needing so much extra code, as I have the schema defined, and the array of json ready.

This is the success event I listed for when I guess I want to do the check and import.

mongoose.connection.on('open', function () {
  console.log('mongoose.connection.opened');
});

Also, to consider, if I wanted to create two collections, and when it generates the ObjectId() for the items in the first collection, I can imagine wanting to use those in the second collection as a ref.

Just assume Club objects only have one string property for now.

// contents of data/club.json
[
  { 'name' : 'Barcelona' },
  { 'name' : 'Real Madrid' },
  { 'name' : 'Valencia' }
]

Any help much appreciated

Answer 1

If I understand it well, all you need is to upload a JSON document to your MongoDB collection from Mongoose. Given that your model is named Club, you can access raw driver methods through Club.collection. And use insertMany to achieve what you want.

Here is a stand alone example (the interesting stuff is at the end):

> var mongoose = require('mongoose')
> var assert = require('assert')

> mongoose.connect('mongodb://localhost/test');

> var Schema = mongoose.Schema
> var clubSchema = new Schema({
...   name: String,
... })

> var Club = mongoose.model('Club', clubSchema)

// Now, the interesting part:
> data = [
...   { 'name' : 'Barcelona' },
...   { 'name' : 'Real Madrid' },
...   { 'name' : 'Valencia' }
... ]
> Club.collection.insertMany(data, function(err,r) {
...       assert.equal(null, err);
...       assert.equal(3, r.insertedCount);
... 
...       db.close();
... })

And check from the Mongo Shell:

> db.clubs.find()
{ "_id" : ObjectId("5574b464b680174d79e37601"), "name" : "Barcelona" }
{ "_id" : ObjectId("5574b464b680174d79e37602"), "name" : "Real Madrid" }
{ "_id" : ObjectId("5574b464b680174d79e37603"), "name" : "Valencia" }

Answer 2

Sometimes shell scripts may help:

for filename in *; do mongoimport -d mydb -c $filename --jsonArray done

See also: MongoDB Bulk import using mongoimport from Windows folder