2

So I have this code:

BlueSOD::Encryption::DecryptionData BlueSOD::Encryption::EncryptionFactory::OpenEnvelopeDecrypt(DecryptionWork && work)
{
    EVP_PKEY* privateKey = work.privateKey.get();
    auto eKey = (unsigned char*)work.aes_info.key.c_str();
    auto eIV = (unsigned char*)work.aes_info.iv.c_str();
    EVP_CIPHER_CTX_ptr cipherCtxPtr{ AcquireCipherCtx() };
    EVP_CIPHER_CTX* cipher = cipherCtxPtr.get();
    int status;

    status = EVP_OpenInit(cipher, m_Cipher, eKey, work.aes_info.key.size(), eIV, privateKey);
    CheckForError(status);

    auto decrypted = make_unique<unsigned char[]>(work.cipherText.size() + EVP_MAX_BLOCK_LENGTH);
    auto cipherTemp = (unsigned char*)work.cipherText.c_str();
    int amtDecrypted = 0;
    int bufferLength = 0;

    status = EVP_OpenUpdate(cipher, decrypted.get(), &amtDecrypted, cipherTemp, work.cipherText.size());
    CheckForError(status);
    bufferLength += amtDecrypted;

    //error occurs in both versions of the following code
    //status = EVP_OpenFinal(cipher, decrypted.get() + amtDecrypted, &amtDecrypted);
    status = EVP_OpenFinal(cipher, decrypted.get(), &amtDecrypted);
    CheckForError(status);
    bufferLength += amtDecrypted;

    DecryptionData data;
    data.plainText = CreateSecureString(decrypted.get(), bufferLength);
    return move(data);
}

And when I step through it in the debugger decrypted holds the correct plain text before the call, yet EVP_OpenFinal returns the error in the title. The code above it is what I initially had (and it returns the same error).

What would be the reason for EVP_OpenFinal returning an error when decrypted holds the correct plain text before the call?

Ricky L.
  • 246
  • 5
  • 13
  • Sorry about that. I don't use Envelope and Seal, and it did not occur to me they were OpenSSL functions instead of your functions. – jww Jan 06 '17 at 19:29
  • No problem! Do you just use symmetric encryption? Or do you use asymmetric as well like `rsa_public_encrypt`? – Ricky L. Jan 06 '17 at 19:38
  • A quick look leads me to something like [Unable to encrypt/decrypt messages longer than 16 bytes](http://stackoverflow.com/a/10733759/608639). I think the call to `status = EVP_OpenFinal(cipher, decrypted.get(), &amtDecrypted);` needs to use `decrypted.get()+bufferLength` to skip the previously decrypted bytes already in the buffer (if I am reading it properly). Did you do the same thing during encryption? If so, that could explain why the ciphertext looks malformed to OpenSSL. – jww Jan 06 '17 at 19:38
  • I'm not using OpenSSL to encrypt. I have a feeling my error lies somewhere else. I just found it odd I would have the exact original text in the buffer, so I thought I was on the right track. – Ricky L. Jan 06 '17 at 19:42
  • Also, I originally tried it with `decrypted.get()+bufferLength` and got the error there as well. – Ricky L. Jan 06 '17 at 19:44

0 Answers0