Hashing a string containing the NUL character in C++ using crypto++

Question

In C++ (C++11 to be precise), I would like to get the HMAC SHA512 hash of a string containing the NUL character (the ASCII character that has all eight of its bits set to 0).

Using crypto++, so far my approach has been as follows:

std::string message("My\0Message",2+1+7);

std::string key_base64 = "MYSECRETKEY";
std::string key;
StringSource ss(key_base64, true, new Base64Decoder(new StringSink(key)));

std::string mac, encoded;

HMAC< SHA512 > hmac((byte*)key.c_str(), key.length());

StringSource(message,
             true,
             new HashFilter(hmac, new StringSink(mac))                                                                                                           
             ); // StringSource                                                                                                                                                

encoded.clear();
StringSource(mac,
           true,
           new Base64Encoder(new StringSink(encoded), false) // Base64Encoder. Pass argument 'false' to prevent insertion of line breaks                                       
           ); // StringSource                                                                                                                                                  

std::cout << "Hashed message (Base64): " << encoded << std::endl;

This doesn't work properly when a NUL character is included as in the message string above.

The base64 encoded version of the hash (variable mac) I get is

bXmQCkzhzq3J81vInF6IRtxXyd+mN8TRh8e3qHThJ+/RYVcgRkFZ9iStKnNaVsGgco/nisUeRpT3m388UR/BMg==

instead of the expected

hUk4PX3mnqs943JnMR+ptW6f8+enIUGBd4x7sUA+Ug2squOFVF6ZdiCewSBDlGAwNuWf+9Uh0AqUkQV1vMNHxg==

EDIT

The expected output can be obtained from the Bash command line as follows:

hex_encoded_secret=$(echo -n "MYSECRETKEY" | base64 --decode | xxd -p | tr '\n' ' ' | tr -d '[:space:]')
echo -ne "My\0Message" | openssl dgst -sha512 -mac HMAC -macopt hexkey:"${hex_encoded_secret}" -binary | base64 | tr -d '\n'

This generates the expected output as given above.

How do you determine the expected value? Do strings without embedded nulls work? — n. m. could be an AI, Mar 25 '18 at 15:55
Strings without embedded NULs work. The expected output can be obtained from the command line as shown in the edit of my original question above. — John S., Mar 25 '18 at 19:46
Please post a MCVE, including complete C++ code and your real bash comparison script.`echo -n "MY_SECRET_KEY" | base64 --decode` cannot possibly work. — n. m. could be an AI, Mar 25 '18 at 19:58
You should probably remove most of the commands in the OpenSSL contraption. Use `echo -e` to send the same exact binary string into OpenSSL. Start troubleshooting your OpenSSL related commands. — jww, Mar 25 '18 at 23:40
@jww: There was indeed a problem with the encoding as you suggested below. The secret key base64 string needs an extra = at the end in order to make it a multiple of 4 characters. Without, different tools seem to decode this differently. I managed to get it to work now, thanks for your help and the link to the online HMAC generator — John S., Mar 26 '18 at 00:01

score 0 · Answer 1 · answered Mar 25 '18 at 17:22

0

This doesn't work properly when a NUL character is included as in the message string above.

The constructor you used should be OK because the machinery uses std::string::size(), and not std::string::c_str() to determine length. I suspect something else is not quite correct, like a character encoding issue.

You are using the std::string constructor of StringSource. The second constructor of StringSource takes a binary string. You might have better luck with it in your production code:

StringSource(reinterpret_cast<const byte*>(&message[0]),
         message.size(),
         true,
         new HashFilter(hmac, new StringSink(mac))              
         ); // StringSource

Also see StringSource on the Crypto++ wiki.

answered Mar 25 '18 at 17:22

jww

97,681
90
411
885

Thanks for pointing out the StringSource page on the wiki. I had seen the overloaded versions, but your suggestion generates exactly the same output as my code. I have also verified that the message string does indeed contain the NUL character: message.size() is 10. I also checked that the Base64 encoding is not causing the problem; the binary outputs already differ. Any other suggestions or ideas? – John S. Mar 25 '18 at 19:50
@JohnS. - Use a NIST test vector and run it though the OpenSSL contraption. I believe you will find it does not arrive at a correct result. You might also verify your string with an online calculator. This one looks pretty good because it takes hex: [Online HMAC Generator](https://www.liavaag.org/English/SHA-Generator/HMAC/). – jww Mar 25 '18 at 20:30

Hashing a string containing the NUL character in C++ using crypto++

1 Answers1