decrypting files using openssl

Question

I'm trying to decrypt a file, which I had originally encrypted using openssl:

#include <stdio.h>
#include <openssl/aes.h>
#include <string.h>
#include <stdlib.h>

const static unsigned char aes_key[]={"passwordpasswor"}; //15 characters + \0

int main(int argc, char *argv[]) {
if (argc>1) {
  if (strcmp("encrypt",argv[1])==0) {
    FILE *file;
    file=fopen("file.txt","w+b");
    unsigned char aes_input[]="#!/bin/bash\necho hello world\0";
    unsigned char iv[AES_BLOCK_SIZE];
    memset (iv,0x00,AES_BLOCK_SIZE);
    unsigned char enc_out[sizeof(aes_input)];
    unsigned char dec_out[sizeof(aes_input)];
    AES_KEY enc_key,dec_key;
    AES_set_encrypt_key(aes_key,sizeof(aes_key)*8,&enc_key);
    AES_cbc_encrypt(aes_input,enc_out,sizeof(aes_input),&enc_key,iv,AES_ENCRYPT);
    //decryption
    memset(iv,0x00,AES_BLOCK_SIZE);
    AES_set_decrypt_key(aes_key,sizeof(aes_key)*8,&dec_key);
    AES_cbc_encrypt(enc_out,dec_out,sizeof(aes_input),&dec_key,iv,AES_DECRYPT);
    //verify
    printf("original %s\n",aes_input);
    hex_print(enc_out, sizeof enc_out);
    printf("sizeof enc_out is %i\n",sizeof(enc_out));
    fwrite(enc_out,1,sizeof(enc_out),file);
    printf("decrypted %s\n",dec_out);
    }

  if (strcmp("decrypt",argv[1])==0) {
    printf("decrypt\n");
    FILE *file;
    char * ciphertext=0;
    int file_length;
    file=fopen("file.txt","r+b");
    if (file) {
      fseek (file,0,SEEK_END);
      file_length= ftell(file);
      fseek(file,0,SEEK_SET);
      ciphertext=malloc(file_length);
      if (ciphertext)
        fread(ciphertext,1,file_length,file);
      fclose(file);
      }
    //printf("ciphertext is %s\n",ciphertext);

    //begin decryption
    AES_KEY dec_key;
    unsigned char enc_out[sizeof(ciphertext)];
    unsigned char iv[AES_BLOCK_SIZE];
    memset (iv,0x00,AES_BLOCK_SIZE);
    AES_set_decrypt_key(aes_key,sizeof(aes_key)*8,&dec_key);
    AES_cbc_encrypt(ciphertext,enc_out,sizeof(ciphertext),&dec_key,iv,AES_DECRYPT);
    printf("original string is %s\n",enc_out);
    }

  }
    return 0;
}

I'm testing decryption in the encryption function as well(to decrypt directly before writing to or reading from the file) which is working fine the standalone decrypt function however is not decrypting properly:

decrypt
original string is #!/bin/b�:�j�

Since decryption is working properly, I'm assuming it's either writing to the file or reading from the file into string that causes this gibberish output.

What have I done wrong?

Thanks.

Answer 1

I have some suggestions that may help. The most important is buffer management.

1. Close your encryption file in the first section. Also, you do not need to null terminate a string declared in quotes. For example, "hello world" not "hello world\0" That will equate to "hello world\0\0" in your cstring (which will just be treated as "hello world\0"). It is more of a convention thing.

2. Make the conditionals if and else if otherwise you are always performing both checks.

3. My understanding is that the encrypt/decrypt functions of openssl deal with buffers and using sizeof(buffer) is unreliable. I believe those functions return the length of the output so you should save that in a variable and use it.

Key take away, use the fwrite function with the correct buffer size without using sizeof. Since encryption will give you pseudorandom characters, you may have null terminators in your buffer before the end as a result.

Here is a link to the openssl manual that uses AES256 CBC. I just recently used it to encrypt and decrypt files and it worked for me. I would provide a code snippet but it is kind of long. Unfortunately, it uses different functions than what you are currently using. https://wiki.openssl.org/index.php/EVP_Symmetric_Encryption_and_Decryption