1

Scenario:
Using Boost Asio 1.69 I have a C++ server running on Linux Fedora 4.9 and a client running on an Android 10 phone using a peer to peer connection. There is continuous sending of data from server to client and occasionally a packet from client to server.

Problem:
Things work well but due to my Server sending many packets at a high rate, my client is unable to catch up. This causes packets to get dropped. Digging deep and reading about the problem, I learnt that there is something called UDP packets pacing. Another link here. This seems to me as a potential solution to my problem. At least something I want to try to avoid the burst of flow of UDP packets and rather try smoothen the flow.

So I tried the following firstly:

uint32_t bytes_per_second = 1000000;
if(setsockopt(udp_socket, SOL_SOCKET, SO_MAX_PACING_RATE, &bytes_per_second, sizeof(bytes_per_second)) < 0) {
    std::cout << "Unable to set socket max pacing rate" << std::endl;
}

But above does not seem to have any affect. I different numbers set for bytes_per_second with no helpful effect and the problem stayed the same.

Question:
How can I effectively exercise UDP packets pacing? Or how can I ensure a slight gap between the packets I am sending from my Linux server side?

Is it a linux configuration I could do or is it something I could do by calling setsockopt on the udp socket? Any suggestions towards potential investigations are also welcome!

TheWaterProgrammer
  • 7,055
  • 12
  • 70
  • 159
  • I do not know what "smoothen" is, but UDP is unreliable, and packets containing UDP datagrams will be lost on the network. That happens all the time, for various reasons. UDP is unreliable, and any application using it must be prepared to lose datagrams, or have as application-layer protocol that requests lost data to be resent. – Ron Maupin Jul 02 '20 at 10:09
  • 1
    I edited my question to point out that its a P2P connection I have. I understand that I loose packets in UDP but this does not happen if I slow down my data rate on the server side. Its really strange. UDP is supposed to be fast right? But, it cant keep up speed at high data rates? – TheWaterProgrammer Jul 02 '20 at 10:20
  • It's "fast" exactly because it doesn't have many of the safety mechanisms built into TCP, like Flow Control. What you're seeing is the result of not having flow control. – Malt Jul 02 '20 at 10:33
  • Do you mean "smother" when you write "smoothen," which is not an English word? Even if you smothered UDP down to one datagram per second, you can still lose IP packets on the network that contain your UDP datagrams. Granted, it will happen a lot less, but the chance is that it will happen once in a while. – Ron Maupin Jul 02 '20 at 10:58
  • 1
    @RonMaupin Sorry for my bad english. I meant to make it smooth. In other words, to make the UDP data flow "smooth". Not smother of course :D – TheWaterProgrammer Jul 02 '20 at 12:38

2 Answers2

1

What you're describing is a Flow Control problem. The problem in flow control is not how to pace the rate of packets, but how to determine the right rate in the first place. You've set bytes_per_second = 1000000;, but why? Why not half that rate? Why not one tenth? Perhaps you're not seeing any improvement because the rate is still too high.

Your protocol needs some way of finding an appropriate rate for a specific client. This can only be done using information from the client. For instance you could add sequence numbers to the packets and have the client report its last processed sequence number. If that number is too old then you need to slow down.

TCP has a built-in flow control mechanism in which the client reports the amount of free space it has in the receive buffer. This information is reported in every TCP segment, so the sender always knows how much additional data it can send. Perhaps it's worth considering switching to TCP for your application?

Malt
  • 28,965
  • 9
  • 65
  • 105
  • 1
    This is really good information @Malt. Hoping to get some more like these from this discussion. Sure. I am playing with that number to see if I can cause an effect. Is there anything other than `SO_MAX_PACING_RATE` that I could use to try affect things in this perspective? – TheWaterProgrammer Jul 02 '20 at 12:45
  • Well, you could do rate limiting in userspace. Just have your code wait between packets – Malt Jul 02 '20 at 13:17
  • @programmer_of_the_galaxies Did you check to see whether `SO_MAX_PACING_RATE` works for you? It requires the network interface to use the `fq` qdisc. – Malt Jul 02 '20 at 13:26
  • 1
    I dont think `SO_MAX_PACING_RATE` makes any difference. I have tried all ranges of values. What is `fq`? A linux tool that allows me to control pacing? – TheWaterProgrammer Jul 02 '20 at 13:43
  • https://man7.org/linux/man-pages/man8/tc-fq.8.html talks about TCP not UDP? – TheWaterProgrammer Jul 03 '20 at 10:01
0

You need to add fq discipline to the interface first

tc qdisc add dev eth0 root fq (1)

Mario Lau
  • 56
  • 2