4

I have a single-page application that is feeding on an API I wrote in JavasScript running on Node.js and using MongoDB for data storage. The API exposes several different content types, each of which is stored in a separate collection in my MongoDB database. My single-page application has a search bar consisting of a drop-down to select content type (each one corresponding to a distinct collection in my MongoDB database) and an input field to specify a search query that will be applied.

What I would like to do is: add an option in my dropdown, called "All", and, when this is selected, my API will return the five most-recently created documents, regardless of the collection that contains those documents.

So, as an example, I might have the following collections:

  • Automobiles
  • Airplanes
  • Boats
  • Bicycles

I would like to write JavaScript in my API that returns the most recent five of any of these. As such, if the user searches for "G", the JSON response that I generate might contain several documents from each collection, as follows:

 [
   {
     "_id": "123",
     "name": "Golf,
     "collection: "Automobile",
     "createdAt": "2014-06-20T01:45:00.0000Z"
   },
   {
     "_id": "234",
     "name": Gulfstream",
     "collection": "Airplane",
     "createdAt": "2014-06-19T01:45:00.0000Z"
   },
   {
     "_id": "345",
     "name": "Glastron",
     "collection": "Boat",
     "createdAt": "2014-06-18T01:45:00.0000Z"
   },
   {
     "_id": "456",
     "name": "Gary Fisher",
     "collection" "Bicycle",
     "createdAt": "2014-06-17T01:45:00.0000Z"
   }
   {
     "_id": "567",
     "name": "Grand Prix",
     "collection": "Automobile",
     "createdAt": "2014-06-16T01:45:00.0000Z"
   }
 ]

Question: Is searching across multiple collections possible in Mongo, and, if so, how do I do it?

Please note:

  1. This is a very different problem than trying to do a JOIN in Mongo -- or, at least it could be. (See: MongoDB - Search on multiple collections. I'm not trying to correlate documents from different collections as I would in a JOIN. (E.g., it cannot be said that documents in the Boats collection contain details about documents in the Automobiles collection.)
  2. There appears to be a poor solution out there that would involve running the query for each collection (i.e., query five Automobiles, five Airplanes, five Boats, and five Bicycles; merge the results into an array, sort by createdAt, and then splice the first five off the front of the array). See: Meteor.js - ways to do user search over multiple collections. However, if possible, I'd like to avoid this because (i) it will make the queries 5x less efficient; and (ii) it will make pagination VERY difficult.
  3. I am using Sails.js as my framework (and, hence, Waterline as my ORM). So, even if this isn't possible at the Mongo layer, I theorize that it may have been implemented at the ORM layer. Is that so, and, if so, how would I use it?
Community
  • 1
  • 1
Morris Singer
  • 1,715
  • 2
  • 18
  • 34
  • 3
    In MongoDB you cannot query more than one collection at a time. generally if that is a requirement you're better off storing all of the data in a single collection and separating out the different types thru a "type" field. Don't know if anyone has done this at the ORM layer. – John Petrone Jul 10 '14 at 00:08

1 Answers1

5

Though John Petrone's comment is very helpful (as it points out that I am running up against a limitation in Mongo), I did want to put together a comprehensive list of possible solutions:

  1. Running the query for each collection (i.e., query five Automobiles, five Airplanes, five Boats, and five Bicycles; merge the results into an array, sort by createdAt, and then splice the first five off the front of the array). (See more details of this solution in the question, above).
  2. Storing all of these kinds of objects in the same collection, per John Petrone's comment, above.
  3. Denormalizing the data by storing a copy of only the data from each document needed to generate a search result; all such copies, regardless of the collection in which their source documents are stored, could be kept in one collection (called, e.g., SearchResults), and the search in my SPA could make requests of an API endpoint that queries documents in that collection. This is an adaptation of John Petrone's proposed solution (in the comments, above), which takes into account the practical limitation of the JavaScript/Mongo ORMs that I have worked with (i.e., Waterline and Mongoose), which all make life really miserable if you want to store documents constructed with different models in the same collection. Drawbacks: requires extra queries on Create, Update, and Delete; more data to store; must keep the SearchResult data in sync with the source documents. Benefits: efficient Read queries; easily compatible with JavaScript/Mongo ORMs.
Morris Singer
  • 1,715
  • 2
  • 18
  • 34