1

To the best of my knowledge OpenSSL's function FIPS_mode_set should not affect encryption. All it does is terminating the program if a weak cipher is used.

I have a piece of code that uses EVP_aes_128 encryption:

EVP_CIPHER_CTX ctx;// = EVP_CIPHER_CTX_new();
EVP_CIPHER_CTX_init(&ctx);
const EVP_CIPHER *cipher = EVP_aes_128_cbc();
EVP_EncryptInit(&ctx, cipher, key, IV);
EVP_CIPHER_CTX_set_padding (&ctx, 0);
EVP_EncryptUpdate(&ctx, encrypted.get(), &encrypted_size, paddedPlain.get(), encrypted_size);
return encrypted;

This code is consistent (I get the same output on every run) and working always as expected (decryption function is decrypting it back with no problems). But when I call FIPS_mode_set(1) on the beginning of the run, I get an inconsistent (different output on every run) output in the output buffer.

the input IV: IV file contents in HexEditor view

the key file: key file contents in HexEditor view

the input text: the input text

the encryption output without 'FIPS_mode_set'(1): the encryption output without FIPS_mode_set

the encryption output with 'FIPS_mode_set'(1): the encryption output with FIPS_mode_set

I'm using OpenSSL version 1.0.2k.

What could possibly cause such behavior?

Itzhak Hasson
  • 67
  • 9
  • What do you mean with "inconsistent output"? Do you have an expected value (+ reasoning why you are expecting it) and the actual result? Please include it *into the question* if you do. – Maarten Bodewes May 24 '17 at 21:44
  • @MaartenBodewes - I added some more information answering all your questions. – Itzhak Hasson May 25 '17 at 08:53

1 Answers1

1

You're not using the API correctly, as you are forgetting to call EVP_EncryptFinal_ex(). FIPS mode has more stringent requirements with regards to clearing buffers, so maybe that's why you don't get any ciphertext back before the call to EVP_EncryptFinal_ex() - which you don't seem to use.

Furthermore, you are using obsolete functions:

The functions EVP_EncryptInit(), EVP_EncryptFinal(), EVP_DecryptInit(), EVP_CipherInit() and EVP_CipherFinal() are obsolete but are retained for compatibility with existing code. New code should use EVP_EncryptInit_ex(), EVP_EncryptFinal_ex(), EVP_DecryptInit_ex(), EVP_DecryptFinal_ex(), EVP_CipherInit_ex() and EVP_CipherFinal_ex() because they can reuse an existing context without allocating and freeing it up on each call.

Please make sure you keep as much as possible to the examples in the OpenSSL (EVP) Wiki.

Maarten Bodewes
  • 90,524
  • 13
  • 150
  • 263
  • I copied the code from the link, and only changed the cipher used (EVP_aes_128_cbc instead of EVP_aes_256_cbc). I figured out two things: 1. EVP_EncryptInit_ex fails which is why I'm getting garbage in the output 2. This is the output I'm getting: '139693591844832:error:2D078072:lib(45):func(120):reason(114):fips.c:311: 139693591844832:error:2D06D073:lib(45):func(109):reason(115):fips_enc.c:124:' – Itzhak Hasson May 25 '17 at 16:18