OpenSSL Decryption - EVP_DecryptFinal_ex fails

Question

I'm using this decryption function to get the plain text value of a cipher which was encrypted using EVP AES 265 GCM; I can see data in rawOut but ret = EVP_DecryptFinal_ex(ctx, rawOut, &len); returns 0; can you provide any insight as to why? I've also seen sources which do rawOut + len in the EVP_DecryptFinal_ex code, I'm not sure why this would be needed as it would move the pointer to the end of the buffer.

unsigned char* keyDecrypter(unsigned char* pszMasterKey)
{
    ERR_load_crypto_strings();

    int ret, len;
    EVP_CIPHER_CTX* ctx;
    unsigned char* rawOut = new unsigned char[48]; // ToDo Remove Hardcoded Value

    Info info = m_header.processKeyInfo();
    if (NULL == info.nonce)
        return NULL;

    if (!(ctx = EVP_CIPHER_CTX_new()))
        return NULL;

    if (!EVP_DecryptInit_ex(ctx, EVP_aes_256_gcm(), NULL, pszMasterKey, info.nonce))
        return NULL;

    if (!EVP_DecryptUpdate(ctx, NULL, &len, m_header.aad, m_header.aad_len))
        return NULL;

    if (!EVP_DecryptUpdate(ctx, rawOut, &len, m_header.encryptedValue, m_header.encryptedValueLen))
        return NULL;

    // Finalise the decryption. A positive return value indicates success,
    // anything else is a failure - the plain text is not trustworthy.
    ret = EVP_DecryptFinal_ex(ctx, rawOut, &len);

    ERR_print_errors_fp(stderr);

    EVP_CIPHER_CTX_free(ctx);

    if (ret > 0)
    {
        return rawOut;
    }
    else
    {
        return NULL;
    }
}

score 0 · Answer 1 · answered Jun 29 '21 at 21:56

From the OpenSSL doc:

"EVP_DecryptFinal() will return an error code if padding is enabled and the final block is not correctly formatted."

Apparently, the padding scheme between encryption and decryption do not match, or perhaps the size of ciphertext fed into the decryption engine did not exactly match the size of the ciphertext that was output from the encryption engine. Note that the ciphertext must include the result of a corresponding call to EVP_EncryptFinal_ex.

It is unfortunate that the original poster did not provide sufficient information to make an exact determination.

score -1 · Answer 2 · answered Nov 20 '18 at 14:25

-1

You need to pass rawOut + len to EVP_DecryptFinal_ex. See in the example at the end of the documentation:

    /* Buffer passed to EVP_EncryptFinal() must be after data just
     * encrypted to avoid overwriting it.
     */
    if(!EVP_EncryptFinal_ex(ctx, outbuf + outlen, &tmplen))
    {
         /* Error */
         return 0;
    }
    outlen += tmplen;

Also note that rawOut must have enough room for (m_header.aad_len + cipher_block_size) bytes. You can get the block size with EVP_CIPHER_block_size().

answered Nov 20 '18 at 14:25

sergiopm

194
5

I have read the documentation and as I mentioned I'm unsure why the outbuf + outlen is needed, as for the cipher_block_size, thank you I've missed that I'll test that out. thanks for your answer – Ol1v3r Nov 20 '18 at 14:44
So I've tried both suggestions and still not getting a positive value from the function. Thanks tho. – Ol1v3r Nov 20 '18 at 15:07

OpenSSL Decryption - EVP_DecryptFinal_ex fails

2 Answers2