2

Assuming we have a network with n nodes and there is a coordinator elected that sends commands to nodes. Let's further assume that the coordinator has horrible bandwidth(upload speed) and he wants to send a large file 10 GB in nodes in o(n) assumption time.

My idea now to optimize performance is to use the erasure coding technique to split the large files into chunks and send one chunk per node so that later on, nodes communicate with each other in o(n^2) assumption time to retrieve the whole block. Hence the bandwidth is shared between validators so the leader does not need to upload huge data and limit his bandwidth and the performance of consensus. Will this incur higher throughput performance? Or I make a hole in the water?

One more advantage of using erasure coding is that if some nodes are considered malicious/byzantine nodes and refuse to communicate and send their chunks the non-faulty nodes will still be able to retrieve messages with lesser chunks with the power of erasure coding

Panagiotis Drakatos
  • 2,851
  • 4
  • 31
  • 43
  • @Ron Maupin i respect your skills and your experience so an answer from you would be ideal – Panagiotis Drakatos Jan 20 '23 at 10:11
  • @Ron Maupin how throughput will be affected can you explain? – Panagiotis Drakatos Jan 24 '23 at 17:13
  • @swineone so correct me if i am wrong throughput will be increased because the sender does not need to send the whole message multiple times to all but only a portion and the bandwidth is shared with the whole network. right? – Panagiotis Drakatos Jan 31 '23 at 10:54
  • I’ll revise my answer if you can address this question: Are all your nodes **in a locally managed network** or **Internet nodes**. The answer depends on this and I assumed local. – James Risner Feb 01 '23 at 14:29

1 Answers1

2

Coordinate with the clients to listen to a multicast stream over your local network. Once you are sure everyone you need to receive the 10 GB file is listening, send the file as a multicast.

The advantage is that IP multicast can send information to a group of interested listeners over a local network with a single transmission. It is and ideal solution to your problem.

You will need to handle some form of flow control, to catch up any that lost a packet during the transfer.

There are a couple of caveats:

  • Consumer-grade networking equipment may not support multicast.
  • Make sure your WiFi infrastructure supports multicast over WiFi. For example, Cisco Meraki supports Multicast.
  • For networks involving multiple switches, you will need to enable multicast routing.
James Risner
  • 5,451
  • 11
  • 25
  • 47
  • Assuming we have one publisher and 3 subscribers with ip multicat the publisher send a message once? – Panagiotis Drakatos Feb 01 '23 at 11:08
  • 1
    Yes. All three machines receive the same packets / transmission at the same time. – James Risner Feb 01 '23 at 11:10
  • 1
    If sending a 10 GB file, it will cost 10 GB plus a percent or so additional over head on the wire. You may send to any number of receivers on the same network at the same time. – James Risner Feb 01 '23 at 11:12
  • @PanagiotisDrakatos does this meet your needs? – James Risner Feb 01 '23 at 13:19
  • 1
    *Yes. All three machines receive the same packets / transmission at the same time* Not if you're using UDP packets. UDP packets can be dropped for any reason, and there's almost no way that 10 GB can be transmitted via UDP over a connection with "horrible bandwidth" without significant packet loss. Just dumping 10 GB of multicast packets without flow control will probably cause the bulk of the data to be summarily dropped. – Andrew Henle Feb 01 '23 at 14:05
  • @AndrewHenle I’ll clarify there needs to be some work done to catch up any that lost / missed. But this is a solved problem (e.g. video broadcasts over multicast handles lost packets). – James Risner Feb 01 '23 at 14:07
  • @JamesRisner Video broadcast tends to ignore dropped packets - there's no force the user to rewind to play the missed bits. And https://en.wikipedia.org/wiki/Multicast#IP "Achieving IP multicast service over a wider area requires multicast routing. Many networks, including the Internet, do not support multicast routing. Multicast routing functionality is available in enterprise-grade network equipment but is typically not available until configured by a network administrator. The Internet Group Management Protocol is used to control IP multicast delivery. " – Andrew Henle Feb 01 '23 at 14:11
  • Caveat: multicast over WiFi works very poorly compared to unicast – Jeremy Friesner Feb 02 '23 at 05:27
  • @AndrewHenle thanks, I've added additional information to address your point. – James Risner Feb 02 '23 at 13:24
  • @James Risner i think i have a similar question would you mind take a quick look and answer it if possible? https://stackoverflow.com/questions/75301944/zmq-pub-sub-pattern-does-the-publisher-need-to-upload-multiple-times-in-nom-m – Mixalis Navridis Feb 02 '23 at 15:53