Should right padding an HMAC SHA256 secret key with \0 return the same hash?

Question

When using the JavaX HMAC/SHA256 hashing libraries, if I right pad my secret key with non-zero bytes, the hash for the same message is different; as expected.

hmacSHA256digest(  "secret".getBytes("UTF-8"), msg) = "244d9c89069406d40803722ec6a793e5e04c55234d9ca03039a7b505cb3f8f00"
hmacSHA256digest("secret\1".getBytes("UTF-8"), msg) = "4f94305c91ca9d8dec13ffcff7e455d6f0c49373e1bbc4035da2b500b11063fb"

However, if I right-pad the secret key with an arbitrary number of \0 bytes, the hash comes back as the same for different byte arrays like:

"secret"
"secret\0"
"secret\0\0"

So, JavaX HMAC SHA256 is returning the same hash, even though the byte[] array returned from getBytes("UTF-8") for the secret just has a few additional zeros at the end (so it's not a UTF-8 issue):

hmacSHA256digest(   "secret".getBytes("UTF-8"), msg) 
= "244d9c89069406d40803722ec6a793e5e04c55234d9ca03039a7b505cb3f8f00"

hmacSHA256digest(   "secret\0".getBytes("UTF-8"), msg) 
= "244d9c89069406d40803722ec6a793e5e04c55234d9ca03039a7b505cb3f8f00"

hmacSHA256digest(   "secret\0\0".getBytes("UTF-8"), msg) 
= "244d9c89069406d40803722ec6a793e5e04c55234d9ca03039a7b505cb3f8f00"

Calls to other JavaX methods for MD5 and plain SHA256 do not return the same hash when extra \0 s are appended to the secret, so they pass our security test case for hash uniqueness across different secrets. Is the failure of this zero-padded-secrets case with MAC/SHA256 a possible attack vector?

This is the example code:

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.security.InvalidKeyException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;

static void testRightZeroPaddedSecretsHaveDifferentHashes() {
    try {
        byte[] msg = "msg".getBytes("UTF-8");

        // HMAC SHA256
        byte[] b3 = hmacSHA256digest(msg, "secret".getBytes("UTF-8"));
        byte[] b4 = hmacSHA256digest(msg, "secret\0".getBytes("UTF-8"));

        // Plain SHA256
        byte[] b5 = SHA256digest(msg, "secret".getBytes("UTF-8"));
        byte[] b6 = SHA256digest(msg, "secret\0".getBytes("UTF-8"));

        boolean same34 = Arrays.equals(b3, b4);
        boolean same56 = Arrays.equals(b5, b6);
        System.out.println(
                "\n" + Arrays.toString(b3) +
                "\n" + Arrays.toString(b4) +
                "\nHMAC SHA256 - identical hash results? = " + same34 +
                "\n" +
                "\n" + Arrays.toString(b5) +
                "\n" + Arrays.toString(b6) +
                "\nPlain SHA256 - identical hash results? = " + same56
        );
    } catch (Throwable e) {
        e.printStackTrace();
    }
}

static byte[] hmacSHA256digest(byte[] msg, byte[] secret) {
    try {
        SecretKeySpec keySpec = new SecretKeySpec(secret, "HmacSHA256");
        Mac mac = Mac.getInstance("HmacSHA256");
        mac.init(keySpec);
        byte[] hmac = mac.doFinal(msg);
        return hmac;
    } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
    } catch (InvalidKeyException e) {
        e.printStackTrace();
    }
    return null;
}

static byte[] SHA256digest(byte[] msg, byte[] secret) {
    try {
        MessageDigest digest = MessageDigest.getInstance("SHA-256");
        digest.update(msg);
        byte[] hash = digest.digest(secret);
        return hash;
    } catch (NoSuchAlgorithmException e) {
        e.printStackTrace();
    }
    return null;
}

And sample output:

[-2, 79, -100, 65, -113, 104, 63, 3, 79, 106, -7, 13, 29, -43, -72, 106, -64, 53, 93, -39, 99, 50, -59, -100, -57, 69, -104, -48, 115, 97, 7, -10] 
[-2, 79, -100, 65, -113, 104, 63, 3, 79, 106, -7, 13, 29, -43, -72, 106, -64, 53, 93, -39, 99, 50, -59, -100, -57, 69, -104, -48, 115, 97, 7, -10] 
HMAC SHA256 - identical hash results? = true

[-88, 92, 89, -29, -65, -48, -127, 51, 125, -120, 78, -38, 25, 57, -91, 91, -50, 111, -33, 40, -3, 0, -95, 89, -50, -88, 39, 118, 101, -56, 91, 126] 
[-40, 39, 49, -64, 58, 40, 124, 64, 110, -100, 50, 115, -32, 114, -107, 24, -73, -17, -37, 11, 67, -26, -48, -65, 109, -24, 119, 45, 74, -31, -81, 119]
Plain SHA256 - identical hash results? = false

Since JavaX HMAC SHA256 failed this zero-padded-secrets test case that passed for the plain SHA256/MD5 algorithms mentioned above, can anyone explain the difference in the behavior and if this can be exploited?

Regarding `"secret\0\0".getBytes("UTF-8")`, I *thought* UTF-8 requires a minimal encoding. Maybe JavaX is trimming the trailing null bytes for the string in that case. Perhaps you should try with `ASCII` character set to get the raw bytes. Also see [Convert String to/from byte array without encoding](https://stackoverflow.com/q/37963917/608639). — jww, Aug 17 '18 at 06:22
Hi Jeff, I considered that possibility and checked, but after converting the string constant with the padded zeros to a byte array via secret.getBytes("UTF-8")... the byte array itself does indeed have the additional zeros. And as far as I can tell, the HMAC SHA256 algorithm is defined for byte[] array inputs (secret and message) agnostic of charset, so the extra zeros in the byte array passed in should be considered in the input. — Roberto Olivares, Aug 17 '18 at 17:38

score 1 · Accepted Answer · edited Oct 07 '21 at 11:39

This is the correct behavior of the HMAC construct, by design.

The secret key should ideally be of the size of the block size of the underlying hash algorithm. For SHA-256, the block size is 512 bits, so your key should be 64 bytes.

From the RFC 2104, if a key is longer than the block size, it will be shortened by way of passing it through the hash function and using the hash as the key. If the key is shorter than the block size, it will be extended by appending zeros.

This is the first step of the HMAC algorithm:

(1) append zeros to the end of K to create a B byte string (e.g., if K is of length 20 bytes and B=64, then K will be appended with 44 zero bytes 0x00)

The recommendation from the RFC is to use keys that are at least the size of the output of the hash function, 32 bytes in your case. Even though this will still fail your test case that the key can be padded with zeroes and produce the same HMAC.

Thanks! That does indeed match the behavior we're seeing. – Roberto Olivares Sep 05 '18 at 19:37 — Roberto Olivares, Sep 05 '18 at 19:37

Should right padding an HMAC SHA256 secret key with \0 return the same hash?

1 Answers1