36

I had a strange idea. I heard of software which from my understanding uses UDP to transfer files decreasing the overhead found in TCP packets.

If my app requires TCP and my LAN has software set up to communicate with another datacenter on the other side of the coast with software setup on their end. Would it be possible to send the actual data via UDP but than simulating TCP on both ends?

Does anyone have any ideas or information about such projects?

Robert Houghton
  • 1,202
  • 16
  • 28
  • 3
    Why does the question get a downvote? -edit- maybe its bc i mentioned nonfree software? I dont know any nonfree software but i edited the question –  May 21 '12 at 07:03

9 Answers9

28

If you're asking if you can use UDP as a Layer 2, then the answer is yes, sort of. There are various protocols that allow you to create a tunnel to another network using a UDP transport, such as L2TP and even IPsec (with NAT traversal). You could also do it at the application layer.

If you're asking if TCP can be implemented in UDP, the answer is no. First, TCP packets and UDP packets have an incompatible format. Second, TCP and UDP have different protocol numbers (seen in the IP header) which means that TCP traffic destined for a UDP port would not be passed to the correct upper-layer protocol.

Community
  • 1
  • 1
mpontillo
  • 13,559
  • 7
  • 62
  • 90
  • 1
    You might be taking the word "simulate" a little too literally, or perhaps I did not read OP's use of "simulate" literally enough. In any case, as long as the sender and receiver applications are written to handle it, it is absolutely possible to stuff a TCP header into the UDP data field and use application logic to handle the TCP semantics. It seems inefficient to do it this way, but data is data. The only requirement is that sender and receiver applications agree on the format and handling semantics. – Chuck Wolber Apr 23 '22 at 22:49
  • @ChuckWolber, yes, this was a challenging question to answer because there were multiple ways to interpret the question. But I think I addressed that in my answer when I said "various protocols that allow you to create a tunnel to another network using a UDP transport". The approach you mentioned more or less aligns with with that. Based on the phrasing in OP's question that "my app requires TCP" I assumed it might not be possible for the sender and receiver to agree on a new protocol. – mpontillo Apr 24 '22 at 21:23
18

Both TCP and UDP are built on top of the IP, but the TCP uses different packet structure and at the layer-2 it is not possible to mimic the TCP using UDP packets.

Of course, if you have the control on both the source and destination, then it is possible to create a reliable UDP tunnel for the TCP packets. This would require some internal information (packet number, ack/nack flags) in the body of the UDP packet.

There is an interesting project http://udt.sourceforge.net/

It is a broadcast-capable reliable file transfer mechanism built on top the UDP.

Viktor Latypov
  • 14,289
  • 3
  • 40
  • 55
7

PseudoTCP is a protocol which implements TCP algorithms on top of the UDP. It was introduced since the NAT traversal for TCP is much more complicated than UDP. But some P2P applications do need a reliable data transfer among nodes.

So far as I know, there are two PseudoTCP variations: Libjingle and Libnice.Libjingle is an open source library from google which was initially for gtalk. You could take a look at file sharing example from libjingle: https://developers.google.com/talk/libjingle/file_share. Recently, Chrome desktop also use PseudoTCP implementation from libjingle for reliable connections.

Qinjin
  • 401
  • 4
  • 12
5

Yes, you can develop a protocol on UDP that simulates TCP. However, if you simulated TCP fully, it would technically have more overhead. Because TCP is implement as the packet and your simulated TCP is implemented in the body of the packet.

If you only need one or two features of TCP (such as basic ordering), then implementing it in UDP is useful.

Halo uses 2-3 (IIRC) UDP protocols that simulate different features of TCP, then full fledged TCP for initializing game-states. I Shot You First Networking, GDC publication

For example, in one case, they send 3 duplicate UDP packets to overcome packet loss.

If you control the software on both ends, and it is cost-effective to build your own protocol, then UDP can be versatile.

Nathan Goings
  • 1,145
  • 1
  • 15
  • 33
5

One way to do it now on Linux-3.18+ is to use Foo over UDP (FOU) which implements Generic UDP Encapsulation (GUE). Here's a good introduction to FOU, and the man page for ip-fou.

Or if you want an [open source] UDP based file transfer system there are things like UDT, UFTP, Tsunami-UDP, and even Google's QUIC (Now deprecated in favour of IETF QUIC).

Update: The QUIC protocol now has been standardised by the IETF which provides for secure reliable and unreliable transport over UDP as an alternative to TCP. There's a wide range of QUIC implementations available. There is also a growing set of protocol mappings on to QUIC such as HTTP/3, DNS over QUIC, etc

Pierz
  • 7,064
  • 52
  • 59
3

If my app requires TCP and my LAN has software setup to communicate with another datacenter on the other side of the coast with software setup on their end. Would it be possible to send the actual data via UDP but than simulating TCP on both ends?

No. A UDP socket is in a different namespace from a TCP socket. You will be unable to write UDP at one end and send or receive TCP at the other end. TCP and UDP are peer protocols; both exist at the layer above IP. You can't use one to spoof the other.

user207421
  • 305,947
  • 44
  • 307
  • 483
  • Thats not exactly what i am asking but it does answer one of my questions. If i cant spoof it than it means i cant use TCP apps directly. Could i perhaps have the TCP apps go through a proxy which transport data using UDP until it hits the other side of the proxy (allowing the software on the other end to use TCP as well)? +1 –  May 21 '12 at 14:08
  • @acidzombie24 You probably could do that but I fail to see the point. If you keep your socket send/receive buffers large the TCP overhead vanishes in the noise. I also don't understand why it isn't exactly what you're asking. You said your app has to use TCP. – user207421 May 22 '12 at 00:33
1

Hmm, I believe so. You'd need to use a proxy at both ends, but it should be possible.

The biggest problem you are going to run into is that UDP is designed with the idea that you don't care if some of the packets don't ever make it to the other end.

Here's a link with some more info:

http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/

IMHO, it's not a good idea to transmit files via UDP.

user978122
  • 5,531
  • 7
  • 33
  • 41
  • Why not? Filecatalyst seems to have gotten it right. I had this idea years ago and i was surprised when i saw a company was built with that idea –  May 21 '12 at 06:57
  • "Similar to TCP, the FileCatalyst implementation will break data into blocks (analogous to the TCP window) although there is no limit to the size of the blocks in FileCatalyst." -> Allow me to summarize: they re-implemented TCP, and modified a few features. It's not UDP, as you might be led to believe. – user978122 May 21 '12 at 06:59
  • How would you reimplemented TCP on TCP... Anyways "The UDP-based protocol used in FileCatalyst is proprietary" source http://www.filecatalyst.com/company/faq/#5 –  May 21 '12 at 07:02
  • Yes, but they've provided enough information in their description that if anyone wanted to create something similar, they could. – user978122 May 21 '12 at 07:04
  • yep. But your answer is clear that it can be possible to tunnel. Good. I thought so to but wasn't sure. (+1 before i wrote my first comment) –  May 21 '12 at 07:07
1

TCP's problems are in its algorithms, not its headers.

You certainly could implement the TCP algorithms on top of UDP. That would effectively be the same as tunneling TCP datagrams inside of UDP datagrams. But all this accomplishes is to add a few more bytes of overhead to each packet, and require another endpoint to unwrap the packets.

UDP itself is just thin shim on top of IP: its a convenient way to access IP packet switched networking without having to dive into kernels or receive special handling from routers. The main reason to implement reliable transport on top of UDP is to get away from TCP algorithms in favor of something more efficient. FileCatalyst was mentioned above as one company which does this, and my own company Data Expedition, Inc. does so as well.

So you could implement TCP algorithms on top of UDP, but you wouldn't want to.

Seth Noble
  • 3,233
  • 19
  • 31
  • 2days ago i figured out why the internet uses TCP rather than UDP. But why do you say i wouldnt want to? (a 1 or 3 sentence reason is fine) –  May 24 '12 at 14:48
  • 1
    You would not want to implement TCP on top of UDP because there is no advantage. It would still be the same TCP algorithms, just with less space for payload and a requirement to have proxies at both ends. The only reason to use UDP for reliable transport is to build your own (not TCP) algorithms. That's what my [company](http://www.dataexpedition.com/) does. – Seth Noble May 24 '12 at 15:12
  • oh i forgot to mention one suspected reason not to. Because if i tunnel tcp over udp i may still need to wait for ACK so there would be no speed benefit. –  May 24 '12 at 15:34
  • the advantage is your packets are no longer subject to tcp congestion control. – Skyler Aug 20 '15 at 13:11
0

You can simulate something like a connection over UDP, and you as well can add reliability checks and ordering and retransmission and so on. - but then, it still isn't TCP, it just acts the way.

Of course, one of the ends can be a kind of "hub" or "proxy" which does an adaption. Then you don't have a 2-end solution, but in fact a 4 end solution - one pair with "real" TCP and the other with the "self-knitted" "TCP" - which you put together with an appropriately crafted program.

glglgl
  • 89,107
  • 13
  • 149
  • 217
  • 1
    ... and by the time you get it to act that way, it *is* TCP, with the same overhead, or more because you are constrained by what you can do in user mode as opposed to being inside the OS kernel. – user207421 May 21 '12 at 07:24
  • But incompatible - both ends have to act that way. – glglgl May 21 '12 at 07:59
  • ok but this doesnt answer my question. "Like TCP" isnt reimplementing tcp. I explicitly said applications using TCP must work. –  May 21 '12 at 08:43
  • That won't be feasible, as TCP and UDP sit together above IP on the protocol layer stack and are mutually exclusive. – glglgl May 21 '12 at 10:38
  • Perhaps some kind of proxy solution on both ends would allow it? –  May 21 '12 at 14:05
  • @acidzombie24 Of course - with this you make applications using "real" TCP work with this solution. – glglgl May 21 '12 at 14:41
  • Solutions to tunnel TCP across a high-performance UDP-based transport do exist. Contact me if you want more information (its commercial and I work for the company that makes it). – Seth Noble May 23 '12 at 19:13