Can Linux buffer UDP packets to correct the sequence

Question

Good Day,

I am transmitting MP3 audio using UDP through the Internet. Yes, it's most likely a bad idea for various reasons, however my use case limits me to this setup. To expand further based on comments below, my source device connects over a very high latency link with low and limited throughput as these devices are located in rural/remote areas. My latency is up to 1 seconds (shocking yes). Using TCP will therefore degrade my service due to its two way coms, acks, resends and the likes.

The issue I'm trying to resolve is that on the receiver server (Linux Ubuntu 18.04), I am seeing all the packets but many times slightly out of sequence. I.e. packet 5 comes before packet 4.

Is there a way in Linux to buffer the incoming UDP stream such that consumers (FFMPEG in my case) is able to get stream in order (being a buffer, with an expected delay, that would be linked to buffer size).

Tcpdump example of issue (note: ids are out of sequence):

16:04:48.648448 IP (tos 0x0, ttl 24, id 25335, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80
16:04:48.648503 IP (tos 0x0, ttl 24, id 25334, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:48.884324 IP (tos 0x0, ttl 24, id 25336, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:48.884357 IP (tos 0x0, ttl 24, id 25337, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80
16:04:49.145213 IP (tos 0x0, ttl 24, id 25339, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80
16:04:49.145257 IP (tos 0x0, ttl 24, id 25338, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:49.406068 IP (tos 0x0, ttl 24, id 25340, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:49.406125 IP (tos 0x0, ttl 24, id 25341, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80
16:04:49.667095 IP (tos 0x0, ttl 24, id 25342, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:49.667175 IP (tos 0x0, ttl 24, id 25343, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80
16:04:49.929139 IP (tos 0x0, ttl 24, id 25344, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:49.929200 IP (tos 0x0, ttl 24, id 25345, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80
16:04:50.164307 IP (tos 0x0, ttl 24, id 25346, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:50.164385 IP (tos 0x0, ttl 24, id 25347, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80
16:04:50.425221 IP (tos 0x0, ttl 24, id 25348, offset 0, flags [DF], proto UDP (17), length 1228)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 1200
16:04:50.425285 IP (tos 0x0, ttl 24, id 25349, offset 0, flags [DF], proto UDP (17), length 108)
    xx.xx.xx.xx.2011 > 172.xx.xx.xx.2011: UDP, length 80

If you care about the sequencing of the traffic, you should consider TCP instead of UDP. — eftshift0, Apr 06 '21 at 14:57
For various reasons, lets assume I cannot use TCP in my present use-case. — QuickPrototype, Apr 06 '21 at 15:00
Does this answer your question? [Ensuring packet order in UDP](https://stackoverflow.com/questions/3745115/ensuring-packet-order-in-udp) — red0ct, Apr 06 '21 at 15:02
*"I cannot use TCP"* - Let's assume that you know the exact reason why you can't. And just tell us this reason. — red0ct, Apr 06 '21 at 15:03
*"Using TCP will therefore degrade my service due to its two way coms, acks, resends and the likes."* Do you think that implementing UDP sequencing at least at the receiver side is easy enough and will not degrade the overall performance? — red0ct, Apr 06 '21 at 16:50
@red0ct, 1st prize would have been to get the packets in order they were sent. However since this issue exists in my setup, the best place i see to correct it is at the receiving sever as it is not resource or network constraint. I.e cloud server that should easily be able to handle this relatively low bitrate udp stream. As shown, im not loosing packets or having large sequencing issues, 99% is just one or two packets swapped around. — QuickPrototype, Apr 06 '21 at 22:17
After you implement sequencing you can start on guaranteed delivery and congestion control. — stark, Apr 07 '21 at 11:33
For anyone who may stumble across this, I found a "udp_refelector" project that I modified to implement a packet buffer in order to allow basic sequence correction. been working well so far. https://github.com/StudentSA/udp-reflector-buffer — QuickPrototype, Jul 08 '21 at 12:17

red0ct · Answer 1 · 2021-04-06T16:49:58.953

UDP protocol does not care about the sequence issues exactly as well as guaranteed delivery.
So Linux UDP implementation even in theory can't have the ability to guarantee that the order of transmitted packets will be respected at the receiver side (while some packets in network can follow different ways causing shuffling at the receiver side). It's just protocol issue and UDP protocol does not have any resources to implement this as opposed to TCP.

By the way you can implement some L7 (application level) logic to handle sequence issues. But this is the common case that media traffic (VoIP - e.g. SIP-over-TCP/UDP + RTP-over-UDP and others) is transmitted over UDP because the sequence and losses are not so crucial in case of media transmission (voice, video, ...).

Can Linux buffer UDP packets to correct the sequence

1 Answers1