How do I use AES_CBC_Encrypt 128 openssl properly in Ubuntu?

Question

Still finding it hard to work with memory in C. I'm reading a text file, where I store each line of text containing a name and random string of 40 characters exactly, into 2 buffers, char buffer1[128] and char buffer2[128] using fscanf() which people seem to dislike, and later I need to take JUST the string (without any padding or empty space in the rest of the buffer) and encrypt it using AES-CBC-128 and then later on decrypt it. I use md5 on the names to generate a key of 16 bytes which I use in the encryption process and a random initialization vector 16 bytes again. Now, the encryption/decryption works fine almost if I type in the string of 40 chars myself into the AES encrypt function (i.e. the string is decrypted later correctly but there's a few extra gibberish characters appearing beforehand), but if I pass the buffer containing the string into the function, the decryption comes out ALL with gibberish characters (in other words it does not work). If I try to make the buffer small enough to contain the string exactly (char buffer[40]), the encryption/decryption doesn't work. If I try to make it char buffer[41] for the terminator in strings in C, I get a Core Dump error. I'm still a C noob and can't figure out what I am doing wrong! N.B. If I use 48 bytes for the encryption length (multiple of AES BLOCK SIZE which is 16 bytes), it DOES NOT WORK. When I use 40 bytes for encryption length (same as the data to be encrypted) it almost works like I said above. Please explain how I can use AES_CBC_Encrypt PROPERLY ONCE AND FOR ALL! I am using openssl and the test code is:

void alice() {
    FILE *fp=fopen("alice.txt","r"); //read mode
    int j;
    char buffer1[128], buffer2[128]; //buffer1->names, buffer2->data
    unsigned char h_j[SHA_DIGEST_LENGTH];
    unsigned char k_j[MD5_DIGEST_LENGTH];

    //loops over the 25 lines in alice.txt file
    for (j=0; j<n_alice; j++) {
        //read a_j data_j into addresses of buffer1 and buffer2
        fscanf(fp,"%s %s\n",(char*)&buffer1, (char*)&buffer2); //buffer1=names, buffer2=data
        if (dbg) printf("Reading %s %40s\n", buffer1, buffer2);

        //Calculate h_j=SHA1(a_j)
        SHA1((unsigned char*)&buffer1, strlen(buffer1),(unsigned char*)h_j);

        //Calculate k_j=MD5(a_j)
        MD5((unsigned char*)&buffer1, strlen(buffer1), (unsigned char*)k_j);

        //Encrypt c_j=AES-CBC-ENC(k_j,data_j)
        //using MD5 digest as the key for the AES CBC
        //initialization vector
        unsigned char init_vector[AES_BLOCK_SIZE];
        RAND_bytes(init_vector, AES_BLOCK_SIZE); //16 bytes

        const size_t encslength = 48;//((inputslength + AES_BLOCK_SIZE) / AES_BLOCK_SIZE) * AES_BLOCK_SIZE;
        unsigned char enc_out[encslength];
        unsigned char dec_out[40];
        memset(enc_out, 0, sizeof(enc_out));
        memset(dec_out, 0, sizeof(dec_out));

        AES_KEY enc_key, dec_key;
        AES_set_encrypt_key(k_j, sizeof(k_j), &enc_key);
        AES_cbc_encrypt(buffer2, enc_out, strlen(buffer2), &enc_key, init_vector, AES_ENCRYPT);

        AES_set_decrypt_key(k_j, sizeof(k_j), &dec_key);
        AES_cbc_encrypt(enc_out, dec_out, encslength, &dec_key, init_vector, AES_DECRYPT);

        printf("original:\t %s\n", buffer2);
        printf("encrypt:\t %s\n", enc_out);
        printf("decrypt:\t %s\n", dec_out);
    }
    fclose(fp);
}

Alice.txt contains:

yves E9D0EEFC4C6DD81F2A7BDE99CCE833FB7B2BDD6D adam D6C33F73EC7175AEA67DEC98427B42AAF6E5E04C david F58EEB1DDB9511A3F220A4B10F01F88260BC99AB umbert F6A6127E6562C5816433FB3B7C341C45A8805DCB victoria CCF10AA4A4725C964FFF78BA3694393ADE1D7B6C nick C8030118EB5E037F6131D9BB5B1BEF0A3F2AC458 xiaofeng E9B45ECAE2F42D59B90EACB2CA8E75A969A7EFAB eve D8DA8BC812D9CD72B91EB7AF49D099A9BE85AB43 bob CC039D2746A3C55E4BA1DCEE46F329E4CA7E0A1A florence C2DFD6650343659BFB530FE719139AB4D3F2BEDC herbert CDCB4196E8CA9002219EC7A8F372911501BD1BBD ivy E53BC8DF9A81BDD881E5352BCE11A2BF0F33236B jack C5213B5F8DC334010586FB094A63D50A7572470E osvald D0FBD9B2102DF6C41C8B26F25AF3E18ACF2BD27C steven C9A3AB53FAA9E8243A63EBDC3257D1C8CCEC7D1C kevin D7C6774E65DBB8F312F50183C87D67BC6FBF7BED gavin F4C1945162294DD902C6BC11EE23BF8B682AC6C9 luke DF2FD780E13F9511411EC92B476D167A6D9F334A martin FE6D5175D3B6E49B6649DBD6F21559F15847CA31 pippo DA908468DBE291E4DDEB082E36E9F5BB316A3C3C richard EC26FF6B364C51DCA5A7CB5D711BCC85946D2517 troy F1165BE81E08B38A42C582A8F25C2CC382233F3D zach EF19BED4FD6732C92437A3F65C4BE69A5010994D william EC74AA8C93AA32EFF85EC4437F50F1F86AECAC29 charlie C86D2F8A3EF1F03127628C7CF9C6D9FB730DACBF

I just call alice() from main to test this.

Answer 1

I managed to figure out my grave mistake. The line:

AES_set_encrypt_key(k_j, sizeof(k_j), &enc_key);

should be:

AES_set_encrypt_key(k_j, sizeof(k_j)*8, &enc_key);

Also the enclength and declength in the AES_CBC_Encrypt() function should be the same apparently. So I used 48bytes for both encryption and decryption, otherwise it didn't work for me. I could be wrong but when I used strlen(buffer2) for encryption (which is 40bytes) the decryption only managed to decrypt part of the string correctly, the rest of it was still gibberish characters. – val37

Answer 2

I struggled with that bad documented API of OpenSSL myself. It really doesn't help that the functions itself are implemented in Assembler, so even the sources are not helping. But I was able to identify some mistakes you've made:

1. As you've stated, AES_set_encrypt_key takes bit-size as second parameter, not bytes.
2. The key must be exactly of sizes 16, 24, or 32 bytes.
3. The given init_vector must be the same at start of encryption and decryption.

Because of 2. your whole example won't work with sha1. MD5_DIGEST_LENGTH and has 16 bytes (128 bit) just by coincidence. The initialization vector also gets changed by the encryption operation, so you could add additional data to the block chain again and again. But since you were not following 3. and used the changed vector for decryption without resetting it first, that's why your decryption returns "gibberish characters".

As an example I took your code, threw out anything unnecessary, and extracted the encrypt/decrypt operations for clearing up the important parts:

void encrypt(unsigned char* buf, size_t length, const AES_KEY* const enc_key, const unsigned char* iv)
{
  unsigned char local_vector[AES_BLOCK_SIZE];
  memcpy(local_vector, iv, AES_BLOCK_SIZE);

  AES_cbc_encrypt(buf, buf, length, enc_key, local_vector, AES_ENCRYPT);
}

I'm working on buf for reading and writing. As you can see the init_vector gets copied locally to keep the original constant. I did the similar for decryption.

void decrypt(unsigned char* buf, size_t length, const AES_KEY* const dec_key, const unsigned char* iv)
{
  unsigned char local_vector[AES_BLOCK_SIZE];
  memcpy(local_vector, iv, AES_BLOCK_SIZE);

  AES_cbc_encrypt(buf, buf, length, dec_key, local_vector, AES_DECRYPT);
}

And here is what I did with your code. I haven't changed fscanf, because we should focus on the asked problem instead of confusing people:

void tst_function()
{
  FILE *fp=fopen("alice.txt","r"); //read mode
  char buffer1[128], buffer2[128]; //buffer1->names, buffer2->data
  memset(buffer1, 0, sizeof(buffer1));
  memset(buffer2, 0, sizeof(buffer2));
  // remember, this won't work with any other size
  unsigned char k_j[AES_BLOCK_SIZE];

  //until end of file
  while(fscanf(fp,"%s %s\n",(char*)&buffer1, (char*)&buffer2) > 0)
  {
     printf("Reading %s %40s\n", buffer1, buffer2);
     printf("original:\t%s\n", buffer2);

     unsigned char init_vector[AES_BLOCK_SIZE];
     RAND_bytes(init_vector, AES_BLOCK_SIZE);

     // PKCS 5 Padding
     // ((inputs_length + AES_BLOCK_SIZE) / AES_BLOCK_SIZE) * AES_BLOCK_SIZE;
     const size_t encslength = 48;
     unsigned char enc_out[encslength];
     memset(enc_out, 0, sizeof(enc_out));
     strncpy(enc_out, buffer2, sizeof(enc_out));

     AES_KEY enc_key, dec_key;
     AES_set_encrypt_key(k_j, sizeof(k_j)*8, &enc_key);
     AES_set_decrypt_key(k_j, sizeof(k_j)*8, &dec_key);

     encrypt(enc_out, encslength, &enc_key, init_vector); //see above
     printf("encrypted:\t%s\n", (char*)enc_out);

     decrypt(enc_out, encslength, &dec_key, init_vector); //see above
     printf("decrpyted:\t%s\n", (char*)enc_out);
   }
   fclose(fp);
}

It now does what one would expect it to do. You have to be extremely careful about the key size. I crashed a lot because of dangling zeros (from strings) I wasn't aware of. Additionally you could take a look into EVP Symmetric Encryption and Decryption and XTS for keys up to 512 bit.