DecryptMessage (Schannel). Dealing with empty output buffers

Question

I'm trying to use Schannel SSPI to send/receive data over SSL connection, using sockets. I have some questions on DecryptMessage()

1) MSDN says that sometimes the application will receive data from the remote party, then successfully decrypt it using DecryptMessage() but the output data buffer will be empty. This is normal and the application must be able to deal with it. (As I understand, "empty" means SecBuffer::cbBuffer==0)

How should I deal with it? I'm trying to create a (secure) srecv() function, a replacement for the winsock recv() function. Therefore I cannot just return 0. Because the calling application will think that the remote party has closed the connection. Should I try to receive another encrypted block from the connection and try to decrypt it?

2) And another question. After successfully decrypting data with DecryptMessage (return value = SEC_E_OK), I'm trying to find a SECBUFFER_DATA type buffer in the output buffers.

  PSecBuffer pDataBuf=NULL;
  for(int i = 1; i < 4; ++i) { // should I always start with 1?
    if(NULL == pDataBuf && SECBUFFER_DATA == buffers[i].BufferType) {
      pDataBuf  = buffers+i;
    }
  }

What if I don't find a data buffer? Should I consider it as an error? Or should I again try to receive an encrypted block to decrypt it? (I saw several examples. In one of them they were retrying to receive data, in another one they were reporting an error)

if *DecryptMessage* return *SEC_E_OK* buffers have next order. `[SECBUFFER_STREAM_HEADER]-[SECBUFFER_DATA]-[SECBUFFER_STREAM_TRAILER]-[SECBUFFER_EMPTY or SECBUFFER_EXTRA]` if you can not find *SECBUFFER_DATA* - so some error in your code. in your code snippet *pDataBuf=NULL* always. if *DecryptMessage* succeeded but the output buffers are empty - nothing to do. — RbMm, May 03 '17 at 20:14
*Should I try to receive another encrypted block from the connection and try to decrypt it?* - yes, what else could you possibly do? — Harry Johnston, May 04 '17 at 00:41
@HarryJohnston and in the case when DecryptMessage returns SEC_E_OK but SECBUFFER_DATA is not found? Is it an error? Or again I should try to receive another encrypted block? — CITBL, May 04 '17 at 04:49
I think RbMm is correct in saying that that should never happen. My recommendation is to treat it as an error. — Harry Johnston, May 04 '17 at 05:03
if library code for ssl well design this(empty buffers on ok) is not problem. - if data exist - you call next layer with decrypted data, if last buffer is *SECBUFFER_EXTRA* - you need call *DecryptMessage* again with this data (this mean you receive by net more than single encrypted message data). if some error happens (except *SEC_E_INCOMPLETE_MESSAGE* ) - you notify about this next layer. - no data - nothing to do. wait for next packet - this is task of another layer. and this is permanently doing until disconnect. on disconnect you usually get *SEC_I_CONTEXT_EXPIRED* if using ssl — RbMm, May 04 '17 at 06:22

score 0 · Answer 1 · answered Jan 28 '21 at 11:07

It appears that you are attempting to replicate blocking recv function for your Schannel secure socket implementation. In that case, you have no choice but to return 0. An alternative would be to pass a callback to your implementation and only call it when SecBuffer.cbBuffer > 0.
Yes, always start at index 1 for checking the remaining buffers. Your for loop is missing a check for SECBUFFER_EXTRA. If there is extra data and the cbBuffer size > 0, then you need to call DecryptMessage again with the extra data placed into index 0. If your srecv is blocking and you don't implement a callback function (for decrypted data sent to application layer), then you will have to append the results of DecryptMessage for each SECBUFFER_DATA received in the loop before returning the aggregate to the calling application.

DecryptMessage (Schannel). Dealing with empty output buffers

1 Answers1