Inconsistent behaviour while sending data on a TCP connection closed by the server

Question

There is a Unit Test program (C++ on Solaris) I came across in my company.

It tests whether an error is thrown when the client tries to send a msg to the server across a TCP connection that has been closed by the server.

Scenario:

Client                          Server
  |<----------Connected----------->|
  |                                |
  |          2 sec pause           |
  |                                |
  |<-----shutdown(SHUT_RDWR)-------|
  |<-----------close---------------|
  |                                |
  |          2 sec pause           |
  |                                |
  |-------------send-------------->|
  |-------------send-------------->|

General behaviour: The 1st send() returns the no. of bytes written to the socket. The 2nd send() fails, i.e. returns -1. This is the expected behaviour.

Inconsistent behaviour: Both the send() pass, making the test case to fail.

Note: There is no delay between the two send().

The behaviour is inconsistent on SunOS 5.10 (64-bit)

Question 1: Can anyone please explain the "General Behaviour" described above? Why did the 1st send() not fail? Why did the 2nd send() fail?

I know very little about TCP so did a bit of research and guessed that maybe after close() the TCP stack at the server side is in FIN-WAIT-2 state, and when it receives the 1st msg after close() (via the 1st send()) it responds with an ACK, and maybe in this ACK it informs the client to stop sending any more msgs?

Question 2: Regarding "Inconsistent behaviour" - the 2nd send() does not fail sometimes - why?

Is it because the 2nd msg is sent before the ACK from 1st msg is received? (this explanation depends of course on the assumption of 1st question)

Note: The Client and Server are on the same machine and snoop did not help.

Unfortunately, there is no "expected behavior" when a socket-level error occurs on an established connection. Hopefully the underlying network stack returns some kind of an error to the application on the top of the stack. However a robust application cannot rely on this. Robust applications employ tactics that involve heartbeat messages and timeouts, in order to attempt to reliably detect a broken/disconnected connection. The only thing the unit test you described is good for is testing the specific behavior of the OS they are executed on. Other OS may show different behavior. — Sam Varshavchik, May 30 '17 at 00:26
Why is that the expected behavior? What do you think changes after the first send? — David Schwartz, May 30 '17 at 00:27

user207421 · Answer 1 · 2017-05-30T00:47:29.213

There is a Unit Test program (C++ on Solaris) I came across in my company.

It tests whether an error is thrown when the client tries to send a msg to the server across a TCP connection that has been closed by the server.

There should be no such test. You don't test the platform, you test the application.

General behaviour: The 1st send() returns the no. of bytes written to the socket. The 2nd send() fails, i.e. returns -1. This is the expected behaviour.

No it isn't. It is likely to be the usual behaviour, but there is no specification anywhere that mandates it as the correct behaviour. So again the test is invalid.

Inconsistent behaviour: Both the send() pass, making the test case to fail.

This can happen any time and should not be treated as a failure.

Note: There is no delay between the two send().

That makes it even more likely that the second send will succeed, indeed that both sends will be coalesced into a single outbound TCP segment, or better still that both sends fit into the socket send buffer and haven't been sent yet at all.

The behaviour is inconsistent on SunOS 5.10 (64-bit)

This is normal.

Question 1: Can anyone please explain the "General Behaviour" described above? Why did the 1st send() not fail? Why did the 2nd send() fail?

Why anything? In the absence of any specification that mandates either of those behaviours, there is nothing here to explain.

I know very little about TCP so did a bit of research and guessed that maybe after close() the TCP stack at the server side is in FIN-WAIT-2 state, and when it receives the 1st msg after close() (via the 1st send()) it responds with an ACK, and maybe in this ACK it informs the client to stop sending any more msgs?

That's not correct, but it is pointless to go into detail on this, as the entire test is invalid and should be removed.

Question 2: Regarding "Inconsistent behaviour" - the 2nd send() does not fail sometimes - why?

See above.

Is it because the 2nd msg is sent before the ACK from 1st msg is received? (this explanation depends of course on the assumption of 1st question)

You need to understand that send() is asynchronous. All it does is buffer data in the socket send buffer. The data may never have been sent to the peer at all at the moment that either send() returned. This is another factor that alone is sufficient to completely invalidate the test.

Note: The Client and Server are on the same machine

This would only make the test even more pointless, if that were possible.

Inconsistent behaviour while sending data on a TCP connection closed by the server

1 Answers1