2

Usually Win32 API can tell what is the length of output buffer required. One need just pass 0 as buffer length and API returns error BUFFER_TOO_SMALL and number of bytes required.

But it is not the same with SslEncryptPacket. It just returns error about small buffer and that's all.

There is also SslLookupCipherLengths which I suppose should be used for that, but documentation gives no clue about how to calculate output buffer having that info.

Maybe you can tell ? Usually I would reserve + kilobyte , but in my situation I need to know exactly.

IInspectable
  • 46,945
  • 8
  • 85
  • 181
user2708351
  • 121
  • 7
  • The maximum size of an SSL message is in the vicinity of 16k, depending slightly on the cipher suite. You may be able to get that exact information out of the MS SSL API, I'm not an expert on it. – user207421 Jun 16 '17 at 10:04
  • To get an idea, you might start with a buffer of length 1 (of course this is ridiculously small, you could start with smth like 512), and every tine the func call returns `BUFFER_TOO_SMALL`, double the buffer size, until you no longer get the error. After that to check if this isn't too large, you could decrease its size till you get the error again, and thus get the exact needed buffer size. – CristiFati Jun 16 '17 at 14:40

1 Answers1

1

You probably already know that in order to go through the TLS/SSL handshake, you repeatedly call SSPI->InitializeSecurityContext (on the client side) or SSPI->AcceptSecurityContext (on the server side).

Once the function returns SEC_E_OK, you should call SSPI->QueryContextAttributes with SECPKG_ATTR_STREAM_SIZES to determine the sizes of the header and trailer. It also tells you the number of SecBuffers to use for the SSPI->EncryptMessage function, and it tells you the maximum size of the message that you can pass to EncryptMessage.

As I understand, the values that are returned may vary depending on the type of encryption that the OS chooses for the connection. I'm not intimately familiar with TLS/SSL but I think it uses 5 bytes for the header, 36 for the footer and 16384 for the maximum message length. You mileage may vary, so that's why you should call QueryContextAttribute(... SECPKG_ATTR_STREAM_SIZES ...).

Jac Goudsmit
  • 91
  • 2
  • 5