2

I am developing a Twitter app that (on the backend) consumes Tweets does some fairly intense processing and then stores the data in a database for use later by the client. All of my servers are running node.js.

I am going to have a server connected to the Twitter Streaming API using nTwitter for node.js. I want to then have this server pass the Tweets along to worker servers and distribute the load based on the Tweet ID (the last digit in the ID would be used).

Right now, I am using Socket.io (and Socket.io-client) which seems to run pretty well. It seems like the Websocket protocol is ideal for this. I am wondering if there are there any reasons not to use Socket.io in this manner?

Loading Balancer Using Socket.io

Justin Elkow
  • 2,833
  • 6
  • 28
  • 60
  • 2
    I can't think of any. Any complexity here would be the same as with a RESTful service, and socket.io will save you some connection time. – Tim Brown Apr 01 '14 at 01:30
  • 1
    It seems pretty silly to use `socket.io`, since that's an abstraction meant to cover up the differences between transports. If all of the clients are `node.js` servers, why not just use raw WebSockets? It'll be more performant (ripping out a layer of abstraction tends to do that) and you won't lose much that I can think of. – Aaron Dufour Apr 01 '14 at 02:51
  • @AaronDufour you are right, it does come with overhead, but it offers nice features like heartbeat, reconnect, etc. Right now, I can get 13,000 messages per second from the LB to the Worker in the same datacenter on a 256GB machine which is enough for me at this point. – Justin Elkow Apr 01 '14 at 21:02
  • 1
    @JustinCloud Interesting. It didn't do heartbeats when I last used it (been quite awhile now). Most of the extra overhead is when connecting, which is probably a much lower percentage of your traffic than for a typical webapp. I'm glad you're happy with your setup - but if it ends up not working, the `ws` npm module has good implementations of both a WebSocket server and client. – Aaron Dufour Apr 02 '14 at 02:03
  • This is not really about load balancing. – Farid Nouri Neshat Apr 15 '14 at 18:39
  • This may not be about Load Balancing in the conventional sense, but conceptually it IS about balancing a load of data that needs processing using the WS protocol. – Justin Elkow Apr 16 '14 at 17:37

0 Answers0