0

I have the following collection:

{
  "_id" : ObjectId("51f1fcc08188d3117c6da351"),
  "cust_id" : "abc123",
  "ord_date" : ISODate("2012-10-03T18:30:00Z"),
  "status" : "A",
  "price" : 25,
  "items" : [{
      "sku" : "ggg",
      "qty" : 7,
      "price" : 2.5
    }, {
      "sku" : "ppp",
      "qty" : 5,
      "price" : 2.5
    }]
}

My map function is:

 var map=function(){emit(this._id,this);}

For debugging purpose I overide the emit method as follows:

var emit = function (key,value){
  print("emit");
  print("key: " + key + "value: " + tojson(value));
  reduceFunc2(key, toJson(value));
}

and the reduce function as follows:

var reduceFunc2 = function reduce(key,values){
  var val = values;
  print("val",val);
  var items = [];
  val.items.some(function (entry){
    print("entry is:::"+entry);
    if (entry.qty>5 && entry.sku=='ggg'){
      items.push(entry)
    }
  });
  val.items = items;
  return val;
}

But when I apply map as:

var myDoc = db.orders.findOne({
  _id: ObjectId("51f1fcc08188d3117c6da351")
});
map.apply(myDoc);

I get the following error:

emit key: 51f1fcc08188d3117c6da351 value:
{
  "_id":" ObjectId(\"51f1fcc08188d3117c6da351\")",
  "cust_id":"abc123",
  "ord_date":" ISODate(\"2012-10-03T18:30:00Z\")",
  "status":"A",
  "price":25,
  "items":[
    {
       "sku":"ggg",
       "qty":7,
       "price":2.5
    },
    {
       "sku":"ppp",
       "qty":5,
       "price":2.5
    }
  ]
}

value:: undefined
Tue Jul 30 12:49:22.920 JavaScript execution failed: TypeError: Cannot call method 'some' of undefined

you can find that their is an items field in the value as printed which is of array kind, even then it is throwing error cannot call some on undefined, if someone can tell where i am going wrong.

Chris Heald
  • 61,439
  • 10
  • 123
  • 137
Phalguni Mukherjee
  • 623
  • 3
  • 11
  • 29

1 Answers1

2

You have an error in your reduceFunc2 function:

var reduceFunc2 = function reduce(key,values){
  var val = values[0]; //values is an array!!!
  // ...
}

Reduce function meant to reduce an array of elements, emitted with the same key, to a single document. So, it accepts an array. You're emitting each key only once, so it's an array with a single element with it.

Now you'll be able to call your MapReduce normally:

db.orders.mapReduce(map, reduceFunc2, {out: {inline: 1}});

The way you overridden emit function is broken, so you shouldn't use it.

Update. Mongo may skip reduce operation if there is only one document associated with the given key, because there is no point in reducing a single document.

The idea of MapReduce is that you maps each document into an array of key-value pairs to be reduced on the next step. If there is more than one value associated with the given key, Mongo runs a reduce operation to reduce it to the single document. Mongo expects reduce function to return reduced document in the same format as the elements which was emitted. It's why Mongo may run reduce operation any number of times for each key (up to the number of emits). There is also no guarantee that reduce operation will be called at all if there is nothing to reduce (e.g. if there is only one element).

So, it's best to move map logic to the proper place.

Update 2. Anyway, why are you using MapReduce here? You can just query for the documents you need:

db.orders.find({}, {
  items: {
    $elemMatch: {
      qty: {$gt: 5},
      sku: 'qqq'
    }
  }
})

Update 3. If you really want to do it with MapReduce, try this:

db.runCommand({
  mapreduce: 'orders',
  query: {
    items: {
      $elemMatch: {
        qty: {$gt: 5},
        sku: 'ggg'
      }
    }
  },
  map: function map (){
    this.items = this.items.filter(function (entry) {
      return (entry.qty>5 && entry.sku=='ggg')
    });
    emit(this._id,this);
  },
  reduce: function reduce (key, values) {
    return values[0];
  },
  verbose: true,
  out: {
    merge: 'map_reduce'
  }
})
Leonid Beschastny
  • 50,364
  • 10
  • 118
  • 122
  • it gives only "{" on doing values[0] – Phalguni Mukherjee Jul 30 '13 at 08:19
  • I get the following result: – Phalguni Mukherjee Jul 30 '13 at 08:57
  • { "_id" : ObjectId("51f1fcc08188d3117c6da351"), "value" : { "_id" : ObjectId("51f1fcc08188d3117c6da351"), "cust_id" : "abc123", "ord_date" : ISODate("2012-10-03T18:30:00Z"), "status" : "A", "price" : 25, "items" : [{ "sku" : "ggg", "qty" : 7, "price" : 2.5 }, { "sku" : "ppp", "qty" : 5, "price" : 2.5 }] } } I am getting the following result,whereas items should have only one object as per the reduce logic. – Phalguni Mukherjee Jul 30 '13 at 09:04
  • It shows clearly it does not use reduce: ], "timeMillis" : 1, "counts" : { "input" : 3, "emit" : 3, "reduce" : 0, "output" : 3 }, "ok" : 1, } – Phalguni Mukherjee Jul 30 '13 at 09:12
  • I am going MR as I have to query multiple collection and than club the result and return to user. – Phalguni Mukherjee Jul 30 '13 at 09:36
  • But you neither maps your data, nor reduces it, you only projects it. Why can't you simply join the results of multiple queries? – Leonid Beschastny Jul 30 '13 at 09:43
  • this is what I am doing:jing.$cmd { "mapreduce" : "orders" , "map" : "function map(){emit(this._id,this);}" , "reduce" : "function reduce(key,values){for(var i=0;i5&&entry.sku=='ggg'){items.push(entry)}});val.items=items;};return values;}" , "verbose" : true , "out" : { "merge" : "map_reduce"} , "query" : { "$where" : "return this.items.some(function(entry){return entry.qty>5})&&this.items.some(function(entry){return entry.sku=='ggg'})"}}, map & reduce fnctnare gtng crtd dynmicly based on usr requir – Phalguni Mukherjee Jul 30 '13 at 09:50
  • `$where` is slow, it's best to avoid it where possible. See my update. – Leonid Beschastny Jul 30 '13 at 10:06
  • Even with the above approach the problem exists. so when I am querying the collection it is picking the right document, but in the sub document "items" even other object which do not match criteria exists.like { "sku":"ppp", "qty":5, "price":2.5 } – Phalguni Mukherjee Jul 30 '13 at 10:22
  • Sorry, I missed `this.items = ` in my example. Try updated version. – Leonid Beschastny Jul 30 '13 at 10:25
  • { "_id" : ObjectId("51f1fcc08188d3117c6da351"), "value" : [{ "sku" : "ggg", "qty" : 7, "price" : 2.5 }, { "sku" : "ppp", "qty" : 5, "price" : 2.5 }] } – Phalguni Mukherjee Jul 30 '13 at 10:26
  • Just tried it myself and it worked fine. Are you sure that you tried the last version of my code> – Leonid Beschastny Jul 30 '13 at 10:45
  • { "_id" : ObjectId("51f1fcc08188d3117c6da351"), "value" : { "_id" : ObjectId("51f1fcc08188d3117c6da351"), "cust_id" : "abc123", "ord_date" : ISODate("2012-10-03T18:30:00Z"), "status" : "A", "price" : 25, "items" : [{ "sku" : "ggg", "qty" : 7, "price" : 2.5 }, { "sku" : "ppp", "qty" : 5, "price" : 2.5 }] } }, this is the output, but it also contains "{ "sku" : "ppp", "qty" : 5, "price" : 2.5 }" inside "items", which do not match our criteria. – Phalguni Mukherjee Jul 30 '13 at 10:50
  • Copy my code and paste it to mongo console. I just tried it and it works fine. – Leonid Beschastny Jul 30 '13 at 10:57
  • > db.runCommand({ ... mapreduce: 'orders', ... query: { ... items: { ... $elemMatch: { ... qty: {$gt: 5}, ... sku: 'ggg' ... } ... } ... }, ... map: function map (){ ... this.items = this.items.filter(function (entry) { ... return (entry.qty>5 && entry.sku=='ggg') ... }); ... emit(this._id,this); ... }, ... reduce: function reduce (key, values) { ... return values[0]; ... }, ... verbose: true, ... out: { ... merge: 'map_reduce' ... } ... }) – Phalguni Mukherjee Jul 30 '13 at 11:01
  • result i gt:{ "result" : "map_reduce", "timeMillis" : 51, "timing" : { "mapTime" : 3, "emitLoop" : 49, "reduceTime" : 0, "mode" : "mixed", "total" : 51 }, "counts" : { "input" : 1, "emit" : 1, "reduce" : 0, "output" : 1 }, "ok" : 1 } – Phalguni Mukherjee Jul 30 '13 at 11:01
  • Actual output in "map_reduce":{ "_id" : ObjectId("51f1fcc08188d3117c6da351"), "value" : { "_id" : ObjectId("51f1fcc08188d3117c6da351"), "cust_id" : "abc123", "ord_date" : ISODate("2012-10-03T18:30:00Z"), "status" : "A", "price" : 25, "items" : [{ "sku" : "ggg", "qty" : 7, "price" : 2.5 }, { "sku" : "ppp", "qty" : 5, "price" : 2.5 }] } } – Phalguni Mukherjee Jul 30 '13 at 11:02
  • What version of `mongodb` are you using? It works fine in `Mongo v2.4.5` on my Ubuntu. – Leonid Beschastny Jul 30 '13 at 11:30
  • I'll try it on my Windows PC when I'll be near it. Try to play with `map` function to ensure that `items` is filtered. For example, you may try to create new `var doc = this` first. I don't think I'll be able to provide further help to you unless I'll find Windows PC. – Leonid Beschastny Jul 30 '13 at 12:20
  • Ok but I am using 2.4.1, but data is inconsistent,in windows please verify once if you get the same we need to inform the mongodb team. – Phalguni Mukherjee Jul 30 '13 at 12:24
  • Tested it with `Mongo 2.0.4` on my windows PC, everything worked fine. – Leonid Beschastny Jul 30 '13 at 16:09
  • Tested with `Mongo 2.4.5` on windows, everything is fine. – Leonid Beschastny Jul 30 '13 at 16:38
  • I tried again, by restarting mongod, but it dosen't come on windows. I am using windows 7,mongo 2.4.3 – Phalguni Mukherjee Jul 30 '13 at 17:28