1

I have a MongoDB collection with objects, the _id property of which is a string.

I want to perform a .find() operation on that collection, perform a .sort() to the returned cursor. Then, an operation of .toArray() would give me an array of documents in a particular order. However, I do not want to extract all those documents, but only the ones between two of them.

Now, I understand that I can perform a .toArray(), get all my results in an array, and then select the interval between two documents, but that would cost too much resources. I want, if possible, to select that interval BEFORE performing an .toArray(), or in general, in the most resource-saving way.

What I am trying to accomplish is an alternative of .limit() and .skip() (paging), but pinpointing an interval by providing _id's of "starting" and "ending" document.

E.g. given that .find().sort({_id: 1}).toArray() gives:

{_id: "a", a: "First entry"},
{_id: "bar", a: "Third entry"},
{_id: "bar2", a: "Fourth entry"},
{_id: "bar3", a: "Fifth entry"},
{_id: "bar4", a: "Sixth entry"},
{_id: "foo", a: "SEcond entry"}

I need something like find().sort({_id: 1}).allAfter({_id: "bar"}).allBefore({_id: "bar4"}).toArray(...); To give me:

{_id: "bar2", a: "Fourth entry"},
{_id: "bar3", a: "Fifth entry"},
Scott Hernandez
  • 7,452
  • 2
  • 34
  • 25
Ivo
  • 1,673
  • 3
  • 19
  • 28
  • This makes little sense since before and after implies an expected order (or sort). If you want to sort once to find the documents between two values then you would have to sort in your client to return them in another order to the user. – Scott Hernandez Jan 07 '12 at 22:54
  • No, I want to return to the client in the same order. And I do not want to select "the documents between two values" but the documents between two other documents. – Ivo Jan 07 '12 at 23:16
  • Please provide an example of documents and the ones you want returned, and how. Your "description of between two other documents" is meaningless without a sort/order; maybe you mean insertion order... – Scott Hernandez Jan 08 '12 at 00:50
  • I rewrote the question, I hope it is cleaner now. – Ivo Jan 08 '12 at 01:18

2 Answers2

1

What you are talking about is range based pagination (where the range is on the sort). You can do this easily by keeping track of the edge of the previous and next pages. Here are some existing answers:

Range based paging mongodb

How to do pagination using range queries in MongoDB?

http://groups.google.com/group/mongodb-user/browse_thread/thread/dc46592112735bf0/a5c700b1283537d8

Let me know if this is not sufficient for your needs.

Community
  • 1
  • 1
Scott Hernandez
  • 7,452
  • 2
  • 34
  • 25
  • I guess it is. The problem with this is that the ID's are strings, which means, if I'm going to use range querying, I have to always sort alphabetically. If I want to sort by another key, given that I only have the ID's of the 'start with' and 'end with' elements, I would have to first query them and find their values for that certain key so I can pass them to $lt and $gt. But it works. – Ivo Jan 09 '12 at 19:33
  • But if you are sorting the ids then you are sorting alphanumerically whether you like, or know it, or not. It will be consistent which is the main goal of range based pagination. – Scott Hernandez Jan 09 '12 at 22:07
0

Seems like you're implying insertion ordering: "bar" was inserted before "a", and "foo" but after "bar2".

You can't query by insertion order in mongodb (maybe because mongo sometimes reorders documents in a collection, for example if you do an update to "bar2" and there's not enough allocated space, then mongo could put it physically behind "bar4").

Your problem would be easily solvable if you could generate integer/long ids, or alternatively by adding a field for insertion time/date.

milan
  • 11,872
  • 3
  • 42
  • 49
  • That's irrelevant. I need to select between two results after I already have the cursor and I have applied a .sort() on it. $lt and $gt will not work because I might sometimes have different sort order (what I pass to .sort() is unknown). – Ivo Jan 06 '12 at 02:57
  • aha, I didn't understand your question then. you need to search within search. is this part of a single search 'session' or you search & sort first and let users select a subset of results and then you search again? – milan Jan 06 '12 at 08:32
  • Search, sort (like a usual MongoDB query) and THEN select a subset of those results. If you think of a more understandable explanation, please edit my post. – Ivo Jan 06 '12 at 17:01
  • i don't understand then why don't you make that a part of the query, this selection of a subset. – milan Jan 06 '12 at 18:00
  • Because, imagine I have 30000 results returned from the query and loaded in the memory. Besides the fact this is a heavy operation, I have to make it even heavier by iterating through all the records in order to select a subset (after one, before another) of them. – Ivo Jan 07 '12 at 23:15
  • what i was asking is, instead of doing this in 2 steps (query + selecting a subset), why don't you make it all part of the query? – milan Jan 08 '12 at 09:33
  • Because I don't know how, that's what I'm asking. The problem is that I don't want to select a subset with .limit() and .step() but instead via two other documents (the one marking the start, the other the end) found via the same query. "start" and "end" imply that an order has been established (via .sort()) – Ivo Jan 08 '12 at 11:43