42

So I am using node.js and socket.io. I have this little program that takes the contents of a text box and sends it to the node.js server. Then, the server relays it back to other connected clients. Kind of like a chat service but not exactly.

Anyway, what if the user were to type 2-10k worth of text and try to send that? I know I could just try it out and see for myself but I'm looking for a practical, best practice limit on how much data I can do through an emit.

josh3736
  • 139,160
  • 33
  • 216
  • 263
cbmeeks
  • 11,248
  • 22
  • 85
  • 136
  • 2
    You can send as much data as you want. Even big video files. The only issue here is the scalability, i.e. to how many users you want to broadcast it? But you will hit these kind of problems anyway. – freakish Oct 19 '12 at 15:42
  • when ever i am emiting 20Mb file data as base64encoded string then other person within the room able to receive but if it is more then 23MB file data is not able receive.. is there any solution for me – Nalla Srinivas Apr 11 '16 at 05:05
  • @NallaSrinivas might want to consider streaming the file with something like socket.io-stream – Sanjay Oct 25 '22 at 18:51

3 Answers3

112

As of v3, socket.io has a default message limit of 1 MB. If a message is larger than that, the connection will be killed.

You can change this default by specifying the maxHttpBufferSize option, but consider the following (which was originally written over a decade ago, but is still relevant):

Node and socket.io don't have any built-in limits. What you do have to worry about is the relationship between the size of the message, number of messages being send per second, number of connected clients, and bandwidth available to your server – in other words, there's no easy answer.

Let's consider a 10 kB message. When there are 10 clients connected, that amounts to 100 kB of data that your server has to push out, which is entirely reasonable. Add in more clients, and things quickly become more demanding: 10 kB * 5,000 clients = 50 MB.

Of course, you'll also have to consider the amount of protocol overhead: per packet, TCP adds ~20 bytes, IP adds 20 bytes, and Ethernet adds 14 bytes, totaling 54 bytes. Assuming a MTU of 1500 bytes, you're looking at 8 packets per client (disregarding retransmits). This means you'll send 8*54=432 bytes of overhead + 10 kB payload = 10,672 bytes per client over the wire.

10.4 kB * 5000 clients = 50.8 MB.

On a 100 Mbps link, you're looking at a theoretical minimum of 4.3 seconds to deliver a 10 kB message to 5,000 clients if you're able to saturate the link. Of course, in the real world of dropped packets and corrupted data requiring retransmits, it will take longer.

Even with a very conservative estimate of 8 seconds to send 10 kB to 5,000 clients, that's probably fine in chat room where a message comes in every 10-20 seconds.

So really, it comes down to a few questions, in order of importance:

  1. How much bandwidth will your server(s) have available?
  2. How many users will be simultaneously connected?
  3. How many messages will be sent per minute?

With those questions answered, you can determine the maximum size of a message that your infrastructure will support.

josh3736
  • 139,160
  • 33
  • 216
  • 263
  • 4
    This is an awesome answer. Thanks for explaining it at a lower level. But where did the 8 packets per client come from? Also, for some reason I was thinking that there was a limit of like 512 bytes or something like that with sockets. Not sure where I got that. lol. Thanks – cbmeeks Oct 19 '12 at 17:40
  • 2
    With a Ethernet [MTU](http://en.wikipedia.org/wiki/Maximum_transmission_unit) of 1500 B, each packet's maximum payload size is 1500 B - 40 B (TCP+IP overhead) = 1460 B. To send 10,240 B of data, a minimum of 8 packets must be sent (1460*7 = 10220 -- not enough; there's 20 more bytes to send. 1460*8 = 11,680). – josh3736 Oct 19 '12 at 18:11
  • Awesome. Would those 8 packets be sent at the same time or serially? Or in some random order? I know this is a little OT but I appreciate the answer. :-) – cbmeeks Oct 19 '12 at 19:38
  • 4
    By their very nature, packets are sent serially -- all data must be sent as a stream of bytes. Because of the nature of IP routing, those packets may not *arrive* in the correct order, but you never have to worry about that because it is TCP's job to ensure that the packets are reassembled into the correct order before handing the data off to the application. – josh3736 Oct 19 '12 at 20:06
  • FALSE , There is a limit 1M, Read more here: https://socket.io/docs/v4/troubleshooting-connection-issues/#you-are-trying-to-send-a-huge-payload – Eyni Kave Jan 05 '22 at 20:21
  • That has changed since this answer was written 10 years ago. I'll update it. – josh3736 Jan 06 '22 at 02:34
8

Limit = 1M (By Default)

Use this example config to specify your custom limit for maxHttpBufferSize:

const io = require("socket.io")(server, {
    maxHttpBufferSize: 1e8, pingTimeout: 60000
});
  • 1e8 = 100,000,000 : that can be good for any large scale response/emit
  • pingTimeout : When the emit was large then it take time and you need increase pingtime too

Read more from Socket.IO Docs

After these setting, if your problem is still remaining then you can check related proxy web server configs, like client_max_body_size, limit_rate in nginx (if u have related http proxy to socketIO app) or client/server Firewall rules.

Eyni Kave
  • 1,113
  • 13
  • 23
4

2-10k is fine, there arn't any enforced limits or anything, it just comes down to bandwidth and practicality.. 10k is small though in the grand scheme of things so you should be fine if that's somewhat of an upper bound for you.

hexist
  • 5,151
  • 26
  • 33
  • 2
    CPU could be a bottleneck too. I am assuming everyone ignored the cpu because his messages are very big and infrequent. Causing the network bottleneck to appear first... – Joe Yahchouchi Jan 29 '14 at 12:19