9

I'm looking for a pub/sub engine, with the following requirements:

  1. Very low latency < 0.5 sec
  2. Scalable
  3. Shardable (based on geo localisation)

I'd like to be able to have multiple pub/sub servers and be able to publish or subscribe to channels from any server, no matter on which server the channel is declared.

For example:

If user A is connected to server SRV1 and user B connected to server SRV2, If user B subscribe to "MyChannel" and user A publish something on channel "MyChannel", user B will get the message even if he's not connected to the same server.

I don't know if Redis is able to do that. I didn't find anything about the subject.

Jason Aller
  • 3,541
  • 28
  • 38
  • 38
Vinc
  • 91
  • 1
  • 2
  • Try activeMQ? http://activemq.apache.org/ – Royce Sep 19 '11 at 23:52
  • 1
    This sounds like the infrastructure that we have built at [Pusher](http://pusher.com). We use WebSocket servers for low latency connection between clients and servers, AWS ELB for loadbalancing and a combination of queuing technologies. Point 3. hasn't been a big issue for us as yet but it's something we are looking into. – leggetter Sep 20 '11 at 09:33
  • hi! Check out [Realtime.co][1] [1]: http://framework.realtime.co/ – Frade Jan 15 '14 at 15:23

4 Answers4

4

We've been using ZeroMQ and it's pub/sub features for while now and we're very happy with what we're seeing.

It's also worth looking at what's coming up in the next version (reducing network bandwidth by pushing subscription requests upstream)

Alan
  • 13,510
  • 9
  • 44
  • 50
3

I suggest you look at Data Distribution Service for Real Time Systems (DDS) standard. It's specifically designed to be a scalable pub/sub middleware both for real-time and non-real-time systems.

It has a few mature implementations all of which has it's own strength points, but generally the implementations are scalable and low latency.

These are the implementations I would suggest you look at (If you need them to work on a WAN environment, I guess the first two ones have great support for that):

Ertan D.
  • 822
  • 1
  • 6
  • 22
1

nanomsg is a successor to ZeroMQ, written by the same author, and with many language-bindings.

It is written in C and uses zero-copy mechanisms. If you're looking for exceptional latency, and are willing to get your hands dirty ( which you should if you aim for something extreme ), I would recommend one of those two.

If you're looking for exceptional throughput, go with Kafka.

Note, none of those solutions implement geolocation out of the box, Redis 3.2 will have something : http://antirez.com/news/89

If you are looking for an easy, "good enough" solution, I'd go with Redis ( be sure to read this blog post from aphyr first ).

user3666197
  • 1
  • 6
  • 50
  • 92
nha
  • 17,623
  • 13
  • 87
  • 133
  • 1
    Yes,**`nanomsg`** is a younger sister of smart & already widespread **`ZeroMQ`** masterpiece.For real insights,the more for soft-real-time ecosystem design, **worth reading Martin Sustrik's marvelous blog posts and publications** >>> http://250bpm.com on co-routines and other smart notes **+ Pieter Hintjens' book** in pdf : **"Code Connected, Vol.1"**.(Btw. 500 ms latency is a very non-real-time-ish criterion, an infrastructure is the issue,where a poor Layer3 transport service may invalidate meeting such limit. Both `ZeroMQ` and `nanomsg` have excellent latency-avoidance,shaving-off ns(!s)) – user3666197 Aug 18 '15 at 16:22
1

It seems you are looking for some sort of messaging. Try RabbitMQ, with shovel or federation plugins

André Laszlo
  • 15,169
  • 3
  • 63
  • 81
xelibrion
  • 2,220
  • 1
  • 19
  • 24