AES decryption failing

Question

I'm trying to perform AES encryption/decryption operation on OpenSSL private.pem file. I'm using polarssl library from here for this operation.

I read private.pem using the following code,

unsigned char * buffer = 0;
long length;
FILE * fp = fopen ("private.pem", "rb");

if (fp)
{
    fseek (fp, 0, SEEK_END);
    length = ftell (fp);
    fseek (fp, 0, SEEK_SET);
    buffer = malloc (length);
    if (buffer)
    {
        fread (buffer, 1, length, fp);
    }
    fclose (fp);
}
printf("buffer is \n %s\n", buffer);

Output:

buffer is 
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,440969729B4F3D6AFEF6D2A0ABD1BE50

zRCM0nSfz6hqi/viRhXNT/5X3/i5lZUgePoM+JYzpXSjOrY1DWRKzFfKfOrHUZNx
DqJzcRMll6E/DfFfoQacu+TDyEsQe2kXkByHJKqDEDoG3FZoIMJOssAzjoQ1qseM
cd61c4imktl7MO7wFVb0z+HhqSdc5zOLd9aN25FbQRVXJdROGajQiuLothEfcbrS
OWMNhNKo3FsN1mKhhaJGha5EQnwmcFvc7aslgda0bAMnnvg5o2g3RPXdLlhi7XE6
EdOpqJYVkfUv4zj2/DcMcSqRNixguRez0macMWsYHV00MQrNS7HFILHjb4bhYKUt
ZHeY36dh48/fyIrQAheB+2Tyq4qxWnQTF4XLj6Y4B+NYqvCOP4+s1ERp574ZMvkZ
oK6LmAgcneYobuuzDYsaZczYxQcM9HYIodVO7Si8RmM/XfwS6Xftt3Rg1TFPWXu8
P4LRj8/AHGd6+Tniky9McGpr/7a79+mr97xbG3hjhayzhQc1uq212jgACDy1m7QF
3evChUQINH0uN8URmhlkXs7ORz1nK0EyVGWYQ0+3GVYyQ7AarqgZIm1xemUb6z33
knwpmjLbdglKg2qQLi/yJdRCCaQr8gJ1QE5GrY7GtE+g09RqxmsT7khIdK8TDbqY
+CRQVyHODZaYf21gNUsVgnyzkAqVndNCU14A7DhzXZBMtROjJGeRCsNwHt4aUSW8
ANdaMBNlDRYRXYrxPpqGlK3T4A0xpi/o3Heu/p4K7lCOgX+v7TVvP3dS+dmBSb0P
9XkuMYfmyuEvX63DgaI1K/QLEMyJ0CVzICKaX8vKZ0ervKW+OLfz4VGITp0fsnd3
-----END RSA PRIVATE KEY-----

I'm trying to perform AES encryption operation on buffer. For that, I did the following,

#define AES_BLOCK_SIZE 16
unsigned char private_encrypt[strlen(buffer)];
int j;


for(j=0;j<length/AES_BLOCK_SIZE;++j){
    aes_crypt_ecb(&aes,AES_ENCRYPT, buffer+AES_BLOCK_SIZE*j,private_encrypt+AES_BLOCK_SIZE*j);        
}

the output of the encryption is:

encrypted private key is ��xvI��8��˷�%�C�Քb��s��&�y�k;.��f��H֕

I don't know any other way to print it. Then I try to decrypt private_encrypt using the following,

unsigned char buf_test[strlen(buffer)];
for(j=0;j<length/AES_BLOCK_SIZE;++j){
    aes_crypt_ecb(&aes,AES_DECRYPT, private_encrypt+AES_BLOCK_SIZE*j,buf_test+AES_BLOCK_SIZE*j);
     buf_test+AES_BLOCK_SIZE*j);
}
printf("output buffer is  %s\n",buf_test);

I get the following output,

output buffer is �Nxړ��f

But shouldn't the output matches with the original private.pem? What I'm doing wrong here?

The same code works if I use plaintext instead of private.pem for encryption/decryption operation using AES.

The text clearly shows `AES-128-CBC` as the cipher type and block chain method. So... why are you doing anything with ECB *at all* ? — WhozCraig, Aug 31 '20 at 19:52
I did not notice that. I generate this **private.pem** using OpenSSL. But regardless of the generated key is **AES-128-CBC** or some other format, I'm using an entirely different AES operation on the **private.pem**. Does it matter in what format my input file is? pleas ecorrect me if I'm wrong. — perplex, Aug 31 '20 at 19:59
It absolutely matters. The format of the file you're presenting appears to be a "legacy" or "nominal" OpenSSL encrypted RSA private key. You can find out about the file format by [looking here](https://www.openssl.org/docs/manmaster/man3/PEM_read_PrivateKey.html#PEM-ENCRYPTION-FORMAT). It follows a very specific format with very specific purpose. You can't just turn some arbitrary encryption/decryption algorithm loose on the PEM encoding, especially in this case where the IV/salt and passphrase (which you're not even specifying) are critical. That said.... — WhozCraig, Aug 31 '20 at 20:03
I'm not familiar with PolarSSL, but I wouldn't be remotely surprised if someone from that community has taken up the cause to the steps necessary to decrypt an OpenSSL encrypted private key via AES-128-CBC (or any other encipherment+chaining) PEM using their toolchain. — WhozCraig, Aug 31 '20 at 20:04
What are you going to achieve when encrypting a textfile with an encrypted private key? Please do not use the UNSECURE ECB mode for this. @WhozCraig: as far as I can read the code the poster is trying to encrypt a pure text file ('private.pem'). You are right that we can read 'AES-128-CBC' but that is the algorithm and mode that was used to encrypt the private key "itself" and after the the encryption the key was saved in the 'PEM-format' for convenience (so it is easier to see what kind of it is ('RSA PRIVATE KEY') instead of a bunch of bytes in a file. — Michael Fehr, Aug 31 '20 at 20:50
There are obvious problems with the code you posted. The input to the encryption does not have a size that is a multiple of 16, so the last block will overwrite a few bytes of memory after the output buffer. After memory corruption like this, anything can happen. It's not clear what you put in `buf_test`, but it is not a null-terminated string: it's binary data. So the buffer is smaller than the data, and again there's memory corruption. It's likely that there are other problems in the parts of the code that you aren't showing. **You must post complete code to reproduce the problem**. — Gilles 'SO- stop being evil', Aug 31 '20 at 20:51
In addition, even as a learning exercise, why on earth are you using an 8-year-old version of the PolarSSL library from some random GitHub fork? — Gilles 'SO- stop being evil', Aug 31 '20 at 20:52
Apart from the padding (you have to pad the file to the next integer multiple of 16 bytes) and the inappropriate output (a hex encoded or a Base64 encoded output would make sense) I see another bug. Apparently you use the same context for encryption and decryption. You should use different contexts and set the key for encryption with `aes_setkey_enc()` and the key for decryption with `aes_setkey_dec()`. This will allow you to perform the AES-ECB decryption and you will get the original AES-128-CBC encrypted key, which is more complex to decrypt. — Topaco, Sep 01 '20 at 13:26

AES decryption failing

0 Answers0