13

I'm looking for a way to implement basic Publish / Subscribe between applications written in different languages, to exchange events with JSON payloads.

WebSocket seems like the obvious choice for the transport, but you need an (arguably small) layer on top to implement some of the plumbing:

  • aggreeing on messages representing the pubsub domain "subscribe to a topic", "publish a message"
  • aggreeing on messages for the infra ("heartbeat", "authentication")

I was expecting to find an obvious standard for this, but there does not seem to be any.

  • WAMP is often refered to, but in my (short) experience, the implementations of server / clients libraries are not great
  • STOMP is often refered to, but in my (even shorter) experience, it's even worse
  • Phoenix Channels are nice, but they're restricted to Phoenix/Elixir world, and not standard (so the messages can be changed at any phoenix version without notice.)

So, is everyone using MQTT/WS (which require another broker components, rather than simple servers ?) Or gRPC ?

Is everyone just re-implementing it from scratch ? (It's one of those things that seems easy enough to do oneselves, but I guess you just end up with an half-baked, poorly-specified, broken version of the thing I'm looking for...)

Or is there something fundamentally broken with the idea of serving streams of data from a server over WS ?

phtrivier
  • 13,047
  • 6
  • 48
  • 79

1 Answers1

13

There are two primary classes of WebSocket libraries; those that implement the protocol and leave the rest to the developer, and those that build on top of the protocol with various additional features commonly required by realtime messaging applications, such as restoring lost connections, pub/sub, and channels, authentication, authorization, etc.

The latter variety often requires that their own libraries be used on the client-side, rather than just using the raw WebSocket API provided by the browser. As such, it becomes crucial to make sure you’re happy with how they work and what they’re offering. You may find yourself locked into your chosen solution’s way of doing things once it has been integrated into your architecture, and any issues with reliability, performance, and extensibility may come back to bite you.

ws, faye-websockets, socket.io, μWebSockets and SocketCluster are some good open-source options.

The number of concurrent connections a server can handle is rarely the bottleneck when it comes to server load. Most decent WebSocket servers can support thousands of concurrent connections, but what’s the workload required to process and respond to messages once the WebSocket server process has handled receipt of the actual data?

Typically there will be all kinds of potential concerns, such as reading and writing to and from a database, integration with a game server, allocation and management of resources for each client, and so forth.

As soon as one machine is unable to cope with the workload, you’ll need to start adding additional servers, which means now you’ll need to start thinking about load-balancing, synchronization of messages among clients connected to different servers, generalized access to client state irrespective of connection lifespan or the specific server that the client is connected to – the list goes on and on.

There’s a lot involved when implementing support for the WebSocket protocol, not just in terms of client and server implementation details, but also with respect to support for other transports to ensure robust support for different client environments, as well as broader concerns, such as authentication and authorization, guaranteed message delivery, reliable message ordering, historical message retention, and so forth. A data stream network such as Ably Realtime would be a good option to use in such cases if you'd rather avoid re-inventing the wheel.

There's a nice piece on WebSockets, Pub/Sub, and all issues related to scaling that I'd recommend reading.

Full disclosure: I'm a Developer Advocate for Ably but I hope this genuinely answers your question.

Srushtika Neelakantam
  • 1,427
  • 9
  • 15