0

I have an issue where the encryption process using EVP_SealInit, EVP_SealUpdate, and EVP_SealFinal are working and not returning any failure codes.

Attempting to decipher the message works through EVP_OpenInit and EVP_OpenUpdate, however EVP_OpenFinal fails returning 0. Despite the function returning 0, the fully deciphered text is stored in the output buffer, and the total output length returning from EVP_OpenFinal is identical to the total length returning from EVP_SealFinal.

I was referencing this SO post: OpenSSL RSA: Unable to encrypt/decrypt messages longer than 16 bytes

I believe the OP modified his code with fixes, so I wasn't able to get much help there. Worth noting that my problem is present regardless of message length. 10, 15, and 140 char messages all fail during the EVP_OpenFinal call, but each message was completely stored in the output buffer.

Encryption:

    int envelope_seal(EVP_PKEY *pub_key, uint8_t const *plaintext, 
                      int plaintext_len, uint8_t *encrypted_key, 
                      int *encrypted_key_len, uint8_t *iv,
                      uint8_t *ciphertext)
    {
        EVP_CIPHER_CTX *ctx = NULL;

        int ciphertext_len = 0;
        int len = 0;

        if (!(ctx = EVP_CIPHER_CTX_new()))
        {
            handleErrors();

            return -1;
        }

        if (!EVP_SealInit(ctx, EVP_aes_256_gcm(), &encrypted_key,
                  encrypted_key_len, iv, &pub_key, 1))
        {
            handleErrors();

            return -1;
        }


        if (!EVP_SealUpdate(ctx, ciphertext + len, &len, plaintext, plaintext_len))
        {
            handleErrors();

            return -1;
        }

        ciphertext_len += len;

        if (!EVP_SealFinal(ctx, ciphertext + ciphertext_len, &len))
        {
            handleErrors();

            return -1;
        }

        ciphertext_len += len;

        EVP_CIPHER_CTX_free(ctx);

        return ciphertext_len;
    }

Decryption:

    int envelope_open(EVP_PKEY *priv_key, uint8_t const *ciphertext, 
                     int ciphertext_len, uint8_t const *encrypted_key, int encrypted_key_len, uint8_t const *iv, uint8_t *plaintext)
    {
        EVP_CIPHER_CTX *ctx = NULL;

        int plaintext_len = 0;
        int len = 0;

        if (!(ctx = EVP_CIPHER_CTX_new()))
        {
            handleErrors();

            return -1;
        }

        if (!EVP_OpenInit(ctx, EVP_aes_256_gcm(), encrypted_key,
                  encrypted_key_len, iv, priv_key))
        {
            handleErrors();

            return -1;
        }

        if (!EVP_OpenUpdate(ctx, plaintext + plaintext_len, &len, ciphertext, ciphertext_len))
        {
            handleErrors();

            return -1;
        }

        plaintext_len += len;

        if (!EVP_OpenFinal(ctx, plaintext + plaintext_len, &len))
        {
            handleErrors();

            return -1;

        }

        plaintext_len += len;

        EVP_CIPHER_CTX_free(ctx);

        return plaintext_len;
    }

Error handling:

    void handleErrors(void)
    {
        ERR_print_errors_fp(stderr);
    }

Any help to highlight what I may be overlooking would be great, if anyone can highlight the internal differences between OpenUpdate and OpenFinal, that may help as well.

Thanks!

Community
  • 1
  • 1
iterator
  • 53
  • 8
  • Update: It occurs to me that in all examples I've seen they similarly had a handleErrors function that was called with no explicit check on the return. If ERR_print_errors_fp (what's inside handleErrors) does not print any errors, is there actually is an error. – iterator Mar 07 '16 at 21:14
  • Looking at the source code for `EVP_OpenFinal()`, it is a little strange. Internally it does two things, but only returns one code. My guess is that the first thing it does (`EVP_DecryptFinal_ex`) succeeds, but the second thing it does (`EVP_DecryptInit_ex`) fails. – President James K. Polk Mar 07 '16 at 22:42

1 Answers1

1

AES_GCM and AES_CCM supports the ability to perform authenticated encryption and decryption of data, which is achieved by creating a MAC tag over the encrypted data. The MAC tag will ensure the data is not accidentally altered or maliciously tampered during transmission and storage.

The output from the encryption operation will be the ciphertext, and a tag. The tag should be used during the decryption operation to ensure that the ciphertext has not been tampered.

Refer https://wiki.openssl.org/index.php/EVP_Authenticated_Encryption_and_Decryption for more info.

using EVP_aes_128_gcm in openssl for aad with size of not multiple of 16 is with tag and aad implementation, which is missed here and is causing the decryption to fail.

We need to retrieve the tag used during encryption after EVP_SealFinal() is called.

unsigned char tag[32]={0};
if(1 != EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, 16, tag))
{
        fprintf(stderr, "EVP_CTRL_GCM_GET_TAG failed\n");
        exit(1);
}

And we need to pass the same tag obtained during encryption, before calling EVP_OpenFinal

if(!EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_TAG, 16, tag))
{
        fprintf(stderr, "EVP_CTRL_GCM_SET_TAG failed\n");
        exit(1);
}
Bharath
  • 21
  • 5