0

Yesterday, while debugging in a hadoop cluster i noticed something strange

# netstat -taupen | grep 54310 tcp 0 0 10.0.12.209:54310 10.0.12.209:54310 TIME_WAIT

You can notice that source ip:port is same as destination ip:port. How is it possible. Can someone explain me how things are working in tcp layer for this connection to work ?

McITGuy
  • 218
  • 1
  • 4
  • 18
pradeepchhetri
  • 2,698
  • 6
  • 37
  • 47
  • I don't know much about Hadoop, but from a networking perspective it looks like its just sending data to itself either via its Loopback address (127.0.0.x) or to its own IP address. Do you know why it would be sending information to itself? – McITGuy Jun 08 '18 at 15:42

2 Answers2

1

It is possible because there were two processes (presumably from a fork). The left hand side is a client (who called bind(2) to set its source IP and port which isn't ordinarily done) and the right hand side is the server.

This is one method to achieve IPC (interprocess communication).

Mark Wagner
  • 18,019
  • 2
  • 32
  • 47
  • 1
    This answer would be correct for the case where port number on both sides is identical but the IP addresses are different. However in the question it's not only the port numbers which are identical, the IP addresses are identical as well. – kasperd Jun 08 '18 at 21:44
  • It is correct when the IP addresses are identical as well. Why wouldn't it be? – Mark Wagner Jun 08 '18 at 21:45
  • Your answer suggests there is a difference between the left and right side. There isn't. It's literally the same socket, which is connected to itself. Anything you write on the socket you can read back from the socket. Sure a socket constructed this way can (like any other socket) be shared between multiple processes after a fork call. But that doesn't make it useful for IPC. And netstat output doesn't even show you how many processes have handles for a specific socket. – kasperd Jun 08 '18 at 21:54
  • It is not literally the same socket. There are two sockets with the same values for sockaddr_in's sin_port and sin_addr. There is nothing that prevents this. – Mark Wagner Jun 08 '18 at 22:07
  • 2
    Can't downvote my own post but this is wrong. I thought the introduction of SO_REUSEADDR and SO_REUSEPORT facilitated this but this doesn't appear to be the case. https://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t goes into detail. I won't delete it in case this is helpful to other people. – Mark Wagner Jun 08 '18 at 23:00
1

The most common way to establish a TCP connection is using a 3-way handshake consisting of:

  • SYN
  • SYN-ACK
  • ACK

But that is not the only way to establish a TCP connection. In order to establish a TCP connection each side has to send a SYN which the other side has to ACK, but it is not required that one party combines SYN and ACK in a single packet. A 4-way handshake is equally possible in which each endpoint sends a SYN and then each endpoint sends an ACK.

One use case for this 4-way handshake is for establishing a connection between a pair of hosts which are both behind a firewall. This can be useful if you are running peer-to-peer applications on a network protected by a firewall. Each endpoint will send a SYN and re-transmit it until a response is received. This means the firewall at each end of the connection will see the local endpoint sending out SYN packets, and once it has seen such an outgoing packet it will allow SYN and ACK packets from the other end to come through. (Though this method works with both TCP and UDP in practice it is rarely used with TCP.)

What all of this means to your scenario is that if an application create a TCP socket and binds it to a local IP address and port number and then attempts to connect to the same IP address and port number, the TCP layer will first generate a SYN packet which is delivered to itself and then respond to that with an ACK which is also delivered to itself.

The outcome is that it is possible for a TCP socket to be connected to itself and to the TCP layer this looks like a 4-way handshake.

I don't know whether there is any useful purpose for such a TCP socket connected to itself, but that's what would produce a connection like the one in your case.

kasperd
  • 30,455
  • 17
  • 76
  • 124