Move Mongo Embeded Document into Own Collection

Question

Can someone point me in the right direction, i have a Collection (Forms) each Form has an embedded document array (Responses). Responses for each form have gotten massive and in hindsight embedding them was a bad idea (mongo documents including embedded have a maximum size limit).

Is there a way i can quickly and easily move all of these embedded Responses into their own collection? is there such a thing like the old SQL select into? I have had a look around in rails console but it is inaccessible with so many embedded documents, so i imagine it'll have to be a complex find and insert query in the mongo console? (just guessing there)

My Model is fixed but this migration (and the mongo docs) are stumping me.

TIA Dougle

score 5 · Accepted Answer · answered Mar 21 '11 at 16:48

So here's a start... This is in the mongo shell

db.questions.insert({name:"jwo", responses:[{question:"your name?", answer:"yomamma"}, {question:"your name?", answer:"pappa"}]});

This created a document json structure like so:

> db.questions.findOne();
{
    "_id" : ObjectId("4d877e89b75dc42c4709278d"),
    "name" : "jwo",
    "responses" : [
        {
            "question" : "your name?",
            "answer" : "yomamma"
        },
        {
            "question" : "your name?",
            "answer" : "pappa"
        }
    ]
}

Now loop through the responses, and set their question_id with the questions' _id, and then insert it into the new responses collection

> for(i=0; i<question.responses.length; ++i){
... question.responses[i].question_id = question._id;   
... db.responses.insert(question.responses[i]);                                                                      
... }

> db.responses.findOne();
{
    "_id" : ObjectId("4d878059b75dc42c4709278e"),
    "question" : "your name?",
    "answer" : "yomamma",
    "question_id" : ObjectId("4d877e89b75dc42c4709278d")
}

You'll want to change the db.questions.findOne to find all of them and loop over. If this does take a while, you may need to switch to a map-reduce function.

This worked a treat thanks, you're right it did take a while :oD — Question Mark, Mar 24 '11 at 14:53

score 1 · Answer 2 · answered Mar 18 '13 at 09:41

Here is the code we ended up with, based off Jesse Wolgamott's answer.

var count = 0;
db.transactions.find().sort({_id: 1}).forEach(function(t){
  if(count % 10000 == 0)
    print(""+t._id+" "+count);

  count += 1;

  for(i=0; i<t.inputs.length; ++i){
    t.inputs[i].transaction_id = t._id;
    db.input2s.insert(t.inputs[i]);
  }
});

Move Mongo Embeded Document into Own Collection

2 Answers2