MongoDB good practices to organize my js codebase : javascript preprocessor?

Question

I'm looking for good ideas to organize my js code around mongo with the difficulty that map reduce functions cannot use the client scope so all functions must be sent (or saved) to the server (hard to reuse existing code). I could use server stored function but as we have a db/client system is a too large scope and db scoped function does not seem to be possible.

Other idea is to use a javascript preprocessor, something like that :

var mapFunction = function() {
    // @include lib.js
    Lib.foo(this.bar);
}

lib.js :

var Lib = {
    foo : function(bar) {...}
}

and run the generated mapFunction within a mapReduce command.

Is there any good practices/ideas to achieve good code reuse (without system stored functions) inside mongodb map/reduce javascript codebase?

score 0 · Accepted Answer · answered Jul 13 '12 at 06:08

0

Server-side functions in MongoDB are DB scoped. However, they are generally discouraged in favour of code that is maintained with the rest of your application in a version control system.

If you're sending the map/reduce command from an application, a pre-processor shouldn't be required .. you could call a variable or function which inserts the appropriate code.

You haven't mentioned a specific language, but here's an example borrowed from the MongoDB PHP documentation:

// construct map and reduce functions
$map = new MongoCode("function() { emit(this.user_id,1); }");
$reduce = new MongoCode("function(k, vals) { ".
    "var sum = 0;".
    "for (var i in vals) {".
        "sum += vals[i];". 
    "}".
    "return sum; }");

$sales = $db->command(array(
    "mapreduce" => "events", 
    "map" => $map,
    "reduce" => $reduce,
    "query" => array("type" => "sale"),
    "out" => array("merge" => "eventCounts")));

answered Jul 13 '12 at 06:08

Stennie

63,885
14
149
175

Thank you, I didn't talked about the hosting code cause we plan to make very complex map/reduce functions. During the conception/devel. of those functions the developper won't work in the hosting app but directly with mongo shell. If the function is accepted it will be included in that main app which is JAVA. Maybe I should have noted that we are in the process of determining if mongo mr is the good tool for that. I'm afraid that the mongo mr function is not well designed for complex mr devel. – nomoa Jul 13 '12 at 07:20
1

@nomoa depending on your application, you may find Hadoop a better fit for complex MR .. Hadoop is multi-threaded and functions can be written in Java. See also: [Hadoop vs Mongo Map/Reduce](http://stackoverflow.com/questions/9287585/hadoop-map-reduce-vs-built-in-map-reduce). There is a [MongoDB adapter for Hadoop](https://github.com/mongodb/mongo-hadoop/) which also may be of interest. – Stennie Jul 13 '12 at 07:38
Thanks, I will introduce hadoop to the team (that's a big but necessary move). Until we migrate the current functions (hopefully we don't have too much many for the moment) to it I use gpp a very simple and language agnostic preprocessor which do the job well. IMHO you should put your last comment in the answer as it is more appropriate. – nomoa Jul 13 '12 at 07:47
@nomoa: your question was around organizing M/R code for MongoDB so I think that is still the right approach for an answer. My comment was in reply to yours about complexity and using Java. Choosing between competing M/R solutions is a different question with other considerations, eg: [SO: Hadoop vs MongoDB M/R](http://stackoverflow.com/questions/9287585/hadoop-map-reduce-vs-built-in-map-reduce). There isn't a "one tool best solves every problem" answer (and even on that SO answer, the last 3 points are very subjective). The MongoDB adapter for Hadoop actually allows you to use both :). – Stennie Jul 15 '12 at 11:54

MongoDB good practices to organize my js codebase : javascript preprocessor?

1 Answers1