28

TL;DR: which of the three options below is the most efficient for paginating with Redis?

I'm implementing a website with multiple user-generated posts, which are saved in a relational DB, and then copied to Redis in form of Hashes with keys like site:{site_id}:post:{post_id}.

I want to perform simple pagination queries against Redis, in order to implement lazy-load pagination (ie. user scrolls down, we send an Ajax request to the server asking for the next bunch of posts) in a Pinterest-style interface.

Then I created a Set to keep track of published posts ids, with keys like site:{site_id}:posts. I've chosen Sets because I don't want to have duplicated IDs in the collection and I can do it fastly with a simple SADD (no need to check if id exists) on every DB update.

Well, as Sets aren't ordered, I'm wheighting the pros and cons of the options I have to paginate:

1) Using SSCAN command to paginate my already-implemented sets

In this case, I could persist the returned Scan cursor in the user's session, then send it back to server on next request (it doesn't seem reliable with multiple users accessing and updating the database: at some time the cursor would be invalid and return weird results - unless there is some caveat that I'm missing).

2) Refactor my sets to use Lists or Sorted Sets instead

Then I could paginate using LRANGE or ZRANGE. List seems to be the most performant and natural option for my use case. It's perfect for pagination and ordering by date, but I simply can't check for a single item existence without looping all list. Sorted Sets seems to join the advantages of both Sets and Lists, but consumes more server resources.

3) Keep using regular sets and store the page number as part of the key

It would be something like site:{site_id}:{page_number}:posts. It was the recommended way before Scan commands were implemented.

So, the question is: which one is the most efficient / simplest approach? Is there any other recommended option not listed here?

Rafael Beckel
  • 2,199
  • 5
  • 25
  • 36
  • 2
    "best" is totally subjective. What's best for me is not necessarily good for you. – Sergio Tulentsev Aug 18 '15 at 15:40
  • 2
    +1 re "best". Using a list isn't a good choice because LRANGE is O(N). Go with Sorted Sets - simplest and cleanest. – Itamar Haber Aug 18 '15 at 16:04
  • @SergioTulentsev Yes, maybe "best" is not the... best word. I'll think in something less subjective and edit the question. Thanks – Rafael Beckel Aug 18 '15 at 16:45
  • @ItamarHaber I'm studying the other Sorted Sets commands and structure. You're right, that's definetely the cleanest way to go, and queries are blazing fast. Thanks! Could you post this as an answer, so I can accept it? – Rafael Beckel Aug 18 '15 at 18:16

1 Answers1

17

"Best" is best served subjective :)

I recommend you go with the 2nd approach, but definitely use Sorted Sets over Lists. Not only do the make sense for this type of job (see ZRANGE), they're also more efficient in terms of complexity compared to LRANGE-ing a List.

F.O.O
  • 4,730
  • 4
  • 24
  • 34
Itamar Haber
  • 47,336
  • 7
  • 91
  • 117
  • 4
    Your comment (and answer) made me study more deeply the difference between LRANGE [ O(S+N) ] and ZRANGE [ O(log(N)+M) ], and figure out that it can be huge! Also, it has leaded me to this [old but gold post from Reddit](https://www.reddit.com/r/programming/comments/1f2ml3/what_does_olog_n_mean_exactly/), which explains O(log n) in a very clear way. The comments there are damn funny!! Well, answer accepted. Cheers. – Rafael Beckel Aug 18 '15 at 19:58