18

When I close the socket on one end of a connection, the other end gets an error the second time it sends data, but not the first time:

import socket

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("localhost", 12345))
server.listen(1)

client = socket.create_connection(("localhost",12345))
sock, addr = server.accept()
sock.close()

client.sendall("Hello World!")    # no error
client.sendall("Goodbye World!")  # error happens here

I've tried setting TCP_NODELAY, using send instead of sendall, checking the fileno(), I can't find any way to get the first send to throw an error or even to detect afterwards that it failed. EDIT: calling sock.shutdown before sock.close doesn't help. EDIT #2: even adding a time.sleep after closing and before writing doesn't matter. EDIT #3: checking the byte count returned by send doesn't help, since it always returns the number of bytes in the message.

So the only solution I can come up with if I want to detect errors is to follow each sendall with a client.sendall("") which will raise an error. But this seems hackish. I'm on a Linux 2.6.x so even if a solution only worked for that OS I'd be happy.

Eli Courtwright
  • 186,300
  • 67
  • 213
  • 256
  • I am having the same issue. How did you end up resolving the problem in your code? – HighCommander4 Feb 21 '11 at 03:08
  • @HighCommander: I ended up adding a `client.sendall("")` after each `send`, which is not technically guaranteed to error out, but works for my program giving the timing and specifics of what I'm doing. This is inelegant and I really wish that there was a way to access the underlying TCP socket state, but alas there seems not to be, either in Python or C. – Eli Courtwright Feb 21 '11 at 19:10
  • does that work for you when the client and server are on different machines? It doesn't for me, the sendall() succeeds in that case. – HighCommander4 Feb 22 '11 at 19:06
  • @HighCommander: This works for me when talking over `localhost` but I haven't tried it across multiple machines, and indeed I wouldn't expect it to be reliable in these cases, since it would take longer to detect a failure. If you were running over a local intranet with fairly consistent round trip times and performance wasn't a huge issue, you could probably add a short sleep before the second send. – Eli Courtwright Feb 22 '11 at 19:22

2 Answers2

15

This is expected, and how the TCP/IP APIs are implemented (so it's similar in pretty much all languages and on all operating systems)

The short story is, you cannot do anything to guarantee that a send() call returns an error directly if that send() call somehow cannot deliver data to the other end. send/write calls just delivers the data to the TCP stack, and it's up to the TCP stack to deliver it when it can.

TCP is also just a transport protocol, if you need to know if your application "messages" have reached the other end, you need to implement that yourself(some form of ACK), as part of your application protocol - there's no other free lunch.

However - if you read() from a socket, you can get notified immediatly when an error occurs, or when the other end closed the socket - you usually need to do this in some form of multiplexing event loop (that is, using select/poll or some other IO multiplexing facility).

Just note that you cannot read() from a socket to learn whether the most recent send/write succeded, Here's a few cases as of why (but it's the cases one doesn't think about that always get you)

  • several write() calls got buffered up due to network congestion, or because the tcp window was closed (perhaps a slow reader) and then the other end closes the socket or a hard network error occurs, thus you can't tell if if was the last write that didn't get through, or a write you did 30 seconds ago.
  • Network error, or firewall silently drops your packets (no ICMP replys are generated), You will have to wait until TCP times out the connection to get an error which can be many seconds, usually several minutes.
  • TCP is busy doing retransmission as you call send - maybe those retransmissions generate an error.(really the same as the first case)
nos
  • 223,662
  • 58
  • 417
  • 506
  • Thanks for the detailed answer, but I'm still a bit confused about this. Doesn't the other side of a connection send ACK packets to confirm that it received the data? I assume from your answer that `send` doesn't wait for this acknowledgement, but is there any other way to block (at the transport level) until we know that our data got through? I understand that in general this should be happening at the application level, but I'm currently speaking a protocol which has no such acknowledgement, so I'm wondering what my options are at the transport level. – Eli Courtwright Feb 04 '11 at 15:47
  • Also, doesn't TCP send a FIN packet when it closes? – Eli Courtwright Feb 04 '11 at 16:35
  • 3
    @Eli Courtwright. Yes. TCP sends ACKs. That's however ACKs at the TCP(transport layer). TCP does not know about your message, it's just a stream of bytes as far as TCP is concerned. A TCP ACK does NOT ack one send() call. It can ack the accumulation of 100 of *your* messages. Or 3 bytes of your message. – nos Feb 04 '11 at 19:05
  • 2
    No - no good way to wait for an ACK., Yes - a FIN (or RST depending on the state) is sent when the socket closes. Imagine what happens if that FIN packet disappears(routers/switches dropping packets is more common than most people think). Also - how'd you you know that the FIN packet was received as you call send() ? Maybe it got received 3 nanoseconds after you called send(). If you also read() from that socket, you can detect that close() , and you'll get notified of send errors too - but only in the perfect cases. However, no close/error "events" from read(or select/poll) is non-conclusive. – nos Feb 04 '11 at 19:16
  • Thanks for the details; I've marked your answer as accepted, though I resent the TCP stack (or possibly the Python socket library) for this. I know that in general receipt verification should be at the application level, rather than the TCP level. But while seeing TCP packets arrive is not sufficient proof that the data got where it was going (e.g. the application might have crashed before saving it afterwards), seeing that TCP packets did NOT arrive actually IS sufficient to know that the data did NOT get where it's going. So as a programmer I'd like access to this failure data. – Eli Courtwright Feb 07 '11 at 14:23
1

As per the docs, try calling sock.shutdown() before the call to sock.close().

ncoghlan
  • 40,168
  • 10
  • 71
  • 80
  • Upvoted for suggesting something I hadn't tried (they don't mention shutdown in the socket.close docstring), but unfortunately this also has no effect. – Eli Courtwright Feb 04 '11 at 15:41
  • 2
    shutdown() will accomplish nothing over a close() in this case. And even if it did - if there's a slight network delay, the other end won't receive the shutdown request immediatly, the first send() call will still succeed, and you have the same problem again. – nos Feb 04 '11 at 15:44