0

I'm using OpenSSL 1.1.1 to add TLS 1.3 support for a client written in C that has previously communicated with the server over plain TCP. The setup, handshake, cert authentication and communication with the server happens correctly.

The problem comes when I try to close the file descriptor. The TCP code was simply calling close(fd). I added SSL_shutdown(ssl) and SSL_free(ssl) (where ssl is the SSL object). Now, if I do a close(fd), I see 3 TCP RSTs being sent from the client to the server. On the other hand, if I do a shutdown(fd, SHUT_RDWR), these RSTs don't occur, probably because the FD is still around. I don't believe these RSTs were happening before the TLS support was added. I understand that shutdown() only prevents data being sent and doesn't actually destroy the fd. We'd like to prevent that fd leak. I've tried calling shutdown(fd, SHUT_RDWR) and then calling close(fd), but the same thing happens. I've also tried:

 1. shutdown(fd, SHUT_RDWR)
 2. int res=read(fd, buffer, 4000)
 3. // check to make sure res is 0
 4. close(fd)

What would be the right way to do this?

ShivanKaul
  • 687
  • 1
  • 8
  • 21
  • 1
    If the original application made no attempt to cleanly shut the connection down before you added TLS support and just called `close`, why do you care whether the connection is cleanly shut down? (And, if so, start by adding clean shutdown support to the code in the TCP case and it will easily port to TLS.) – David Schwartz Mar 04 '21 at 22:50
  • I don't believe the TCP RSTs were happening before we added TLS support. – ShivanKaul Mar 04 '21 at 22:52
  • Probably not, but by luck not by design. Nothing prevented them. If the other side were to send data just before the call to `close`, the connection would have aborted. If clean shutdown was a requirement, the code would have enforced/ensured it. If it turns out it was, then fix the code for TCP first! – David Schwartz Mar 04 '21 at 23:00
  • I see, you're saying that the TCP RSTs don't have anything to do with TLS - it's just that the TLS exchange manifests the underlying lack of TCP cleanup because... why? I'd love to understand why adding TLS brought this up. – ShivanKaul Mar 04 '21 at 23:05
  • Just to be clear, adding shutdown(fd, SHUT_RDWR) and then calling close(fd) doesn't fix the issue either. – ShivanKaul Mar 04 '21 at 23:07
  • 1
    You can't close the descriptor until the TCP connection is actually shutdown by the other side. Otherwise, how can you know that the connection was shut down cleanly? You have to wait for the other side to do its part before closing the descriptor. After shutting down the write side, you need to keep reading until `read` returns zero indicating the shutdown is complete. – David Schwartz Mar 04 '21 at 23:12
  • 1
    (And then when you convert all those operations to their TLS equivalents, you will get a clean TLS shutdown too.) – David Schwartz Mar 04 '21 at 23:13
  • This is what I have now. shutdown(fd, SHUT_RDWR) followed by read until 0 followed by close(fd). I still get the RSTs. – ShivanKaul Mar 04 '21 at 23:33
  • Well that should fix the TCP case. Now you need to port that to TLS. Call the TLS versions of shutdown, read, and close in the TLS version code and you should now get a clean shutdown of the TLS connection, allowing the TLS code to do a clean shutdown of the TCP connection. – David Schwartz Mar 05 '21 at 00:49

0 Answers0