How can non-exclusive ports exist when tcp identify applications using 16-bit port number?

Question

I can't come up with the exact socket api,but I remember that there is a socket option that demonstrates the port exclusive/non-exclusive.

If it's not exclusive,how can TCP know which application it should forward to for a specific destination port?

score 6 · Accepted Answer · edited May 23 '17 at 11:45

I think you might be referring to the SO_REUSEPORT option available on some systems.

From the BSD man page:

SO_REUSEPORT allows completely duplicate bindings by multiple processes if they all set SO_REUSEPORT before binding the port. This option permits multiple instances of a program to each receive UDP/IP multicast or broadcast datagrams destined for the bound port.

Implementations vary a lot for this (from non-existant, to restricted to UDP, to allowing TCP also). In the cases where TCP is allowed, the connexions are distinguished by both source and target (ip,port) pairs. This is sufficient to allow the implementation to decide which app needs which packet. (see Trek - Socket options for instance.)

With multiple apps bind to the same TCP port, you could only have one socket accepting on the port. The others would use the port to initiate outbound connections. The TCP stack always knows where to send the packets to.

incoming packets that initiate (SYN) a connection go to the only accepting socket
incoming packets for a connected stream are routed to the socket they belong to

Note: the sockets themselves (including, I believe the accepting socket) can be shared across multiple processes. See Is there a way for multiple processes to share a listening socket? for example.

Here's how it could work. Voluntarily simplifying TCP (no three-way handshake). Let's note the socket information held by the TCP stack like this

(socketname)[owner app, (local IP, local port), (state, remote IP, remote port)]

With that, let's set up three apps A, B and C:

App A -> bind (localhost,12345,SO_REUSEPORT)
    TCP stack: create socket (s1)[belongs to A, (localhost,12345), (not connected)]
App A <- s1

App B -> bind (localhost,12345,SO_REUSEPORT)
    TCP stack: create socket (s2)[belongs to B, (localhost,12345), (not connected)]
App B <- s2

App C -> bind (localhost,12345,SO_REUSEPORT)
    TCP stack: create socket (s3)[belongs to C, (localhost,12345), (not connected)]
App C <- s3
App C -> s3.listen()
    TCP stack: update socket (s3)[belongs to C, (localhost,12345), (open for business)]
App C -> s3.accept()

At this point, no data has been sent, but the kernel knows exactly what socket belongs to what application. Let's have A actually try to do something with its socket:

App A -> s1.connect(otherhost,54321)
    TCP stack: update socket (s1)[belongs to A, (localhost,12345), (connecting, otherhost,54321)]
    TCP stack: send SYN

Here, three things can happen:

incoming ACK packet from (otherhost,54321) with correct sequence: this is fine, it's for s1, no other possibility

    TCP stack: update socket (s1)[belongs to A, (localhost,12345), (connected,otherhost,54321)]
    TCP stack: send SYN/ACK
    TCP stack: notify App A that socket is connected
App A <- connect succeeded, you can start doin' stuff

incoming SYN packet from (client,4343): can only be for s3, only socket ready for SYN

    TCP stack: create new socket (s4)[belongs to C, (localhost,12345), (connected,client,4343)]
    TCP stack: send ACK to client
    TCP stack: notify App C that (s4) has been accepted
App C <- s4 returned from accept()

incoming packet from somewhere else:

    TCP stack: drop or reject, there are no matching sessions

Let's imagine the two things above happened. The kernel information is now:

(s1)[belongs to A, (localhost,12345), (connected,otherhost,54321)]
(s2)[belongs to B, (localhost,12345), (not connected)]
(s3)[belongs to C, (localhost,12345), (open for business)]
(s4)[belongs to C, (localhost,12345), (connected,client,4343)]

Now four kinds of packets can come in:

incoming normal packet from (otherhost,54321): this matches s1, hand it over to App A

incoming normal packet from (client,4343): this matches s4, hand it over to App C

incoming SYN packet from (otherclient,2398): this matches s3, same as before

anything else: drop or reject, invalid state

The TCP stack always nows which socket a packet belongs to. So it knows where to deliver the data.

SO_REUSEADDR is different: it only allows to bind to a port in TIME_WAIT state - i.e. the port is already closing down, the socket that had opened it has been issued a close already (more or less, there are other conditions). With SO_REUSEADDR, only one socket at a time holds the socket open.

how can TCP know which application it should forward to for a specific destination port? — mysql_go, Apr 17 '11 at 13:54
it's not really a matter of selecting the application, it's a matter of selecting the right socket, which is doable with both source and destination (ip/port) pairs. What application uses which socket is not really a problem, that's dealt with like other I/O aspects (i.e. open files on Unix-like plateforms) — Mat, Apr 17 '11 at 14:14

score 0 · Answer 2 · answered Apr 17 '11 at 13:04

A socket is identified by the IP address and the port - both. Together, in the BSD socket API, they are called a "name". You can bind your socket to a name and, if that name comprises a specific address, you can bind another socket to another name with the same port.

Also: a connection is a pair of two sockets, so a single address/port pair can be used in several connections - meaning more than one connected socket can have the same name (but not the same fd).

Can you illustrate by some examples? – mysql_go Apr 17 '11 at 13:08 — mysql_go, Apr 17 '11 at 13:08

How can non-exclusive ports exist when tcp identify applications using 16-bit port number?

2 Answers2