1

I'm having trouble thinking of how to implement an online friends list with Ruby and Redis (or with any NoSQL solution), just like any chat IM i.e. Facebook Chat. My requirements are:

  • Approximately 1 million total users
  • DB stores only the user friends' ids (a Set of integer values)

I'm thinking of using a Redis cluster (which I actually don't know too much about) and implement something along the lines of http://www.lukemelia.com/blog/archives/2010/01/17/redis-in-practice-whos-online/.

UPDATE: Our application really won't use Redis for anything else other than potentially for the online friends list. Additionally, it's really not write heavy (most of our queries, I anticipate, will be reads for online friends).

John K. Chow
  • 1,651
  • 16
  • 24
  • Please do not cross-post on stackoverflow and the Redis ML. It is considered as impolite for people providing Redis support. – Didier Spezia May 04 '12 at 09:16
  • Sorry about that @DidierSpezia, I wasn't aware that I was being inconsiderate to the Redis community. I can see that this is like spamming. – John K. Chow May 04 '12 at 15:44

3 Answers3

4

After discussing this question in the Redis DB Google Groups, my proposed solution (inspired by this article) is to SADD all my online users into a single set, and for each of my users, I create a user:#{user_id}:friends_list and store their friends list as another set. Whenever a user logs in, I would SINTER the user's friends list and the current online users set. Since we're read heavy and not write heavy, we would use a single master node for writes. To make it scale, we'd have a cluster of slave nodes replicating from the master, and our application will use a simple round-robin algorithm to do the SINTER

Josiah Carlson suggested a more refined approach:

  1. When a user X logs on, you intersect their friends set with the online users to get their initial online set, and you keep it with a Y-minute TTL (any time they do anything on the site, you could update the expire time to be Y more minutes into the future)
  2. For every user in that 'online' set, you find their similar 'initial set', and you add X to the set
  3. Any time a user Z logs out, you scan their friends set, and remove Z from them all (whether they exist or not)
John K. Chow
  • 1,651
  • 16
  • 24
0

what about xmpp/jabber? They built-in massive concurrent use and highly reliable, all you need to do is the make an adapter for the user login stuff.

c2h2
  • 11,911
  • 13
  • 48
  • 60
  • I'm not familiar with XMPP at all, and it seems like it has a ton of features that doesn't fit our needs. Have you tried using XMPP before? – John K. Chow May 04 '12 at 15:47
0

You should consider In Memory Data Grids, these platforms are targeted to provide such scalability. And usually can easily be deployed as a cluster on any hardware of cloud. See: GigaSpaces XAP, vMware GemFire or Oracle Coherence. If you're looking for free edition XAP provides Community Edition.

Guy Korland
  • 9,139
  • 14
  • 59
  • 106
  • Unfortunately my tech stack is all Ruby and most of your suggestions have Java/.NET drivers. Additionally, the community support behind them aren't as strong as Redis (at least from initial glance). – John K. Chow May 09 '12 at 16:21
  • XAP provides many standard APIs which can be used by Ruby developers. e.g. memcached, REST and even Native API (c++) – Guy Korland May 10 '12 at 12:43