11

Based on recent feedback and findings on this problem, I've rewritten the question to get rid of noise.

I have 2 separate code paths, one in Java (Android), one and Python which accomplish the following for the purposes of negotiating a pairing between an Android device and a Python/Django.

Java:

  • Generate a syncKey
  • Hash a concatenated string of various values using the presharedKey (including the syncKey)
  • Encrypt the syncKey using a presharedKey
  • Send the Hash, encrypted syncKey, DeviceId and arbitrary variables to web server

Python

  • Get the presharedKey from the deviceId
  • Decrypt the encrypted syncKey
  • Hash a concatenated string of various values using the presharedKey (including the decrypted syncKey)
  • Make sure the hash matches, which confirms that the syncKey was decrypted successfully, and that the deviceId holds the correct presharedKey.

Now this process works if I send the syncKey unencrypted. The final hash matches, which proves the deviceId has the correct preshared-key, however as soon as I add the en/decryption into the process, the hash no longer matches, despite the fact that both the syncKey and concatenated string appear to match perfectly character for character from the debug output of both Java/Python.

One quirk of the process is that a 256bit key is necessary for the AES256 encryption algorithm, so I'm chopping the 512bit presharedKey in half. The alternative of using only a 256bit key across the board was requiring that I pass the key through encode('ascii') on the python side, or else it was throwing up errors during hashing with the shorter key.

Here is the relevant code:

Java:

String presharedKey = getKey();
// f8250b0d5960444e4de6ecc3a78900bb941246a1dece7848fc72b90092ab3ecd0c1c8e36fddba501ef92e72c95b47e07f98f7fd9cb63da75c008a3201124ea5d

String deviceId = getDeviceId();
// 1605788742789230

SyncKey syncKey = generateSyncKey();
// 824C1EE9EF507B52EA28362C71BD4AD512A5F82ACFAE80DEF531F73AC124CA814BA30CE805A157D6ADB9EC04FC99AAE6FDC4238FCD76B87CE22BC2FE33B2E5C9

String concat = syncKey.hexString();
// 824C1EE9EF507B52EA28362C71BD4AD512A5F82ACFAE80DEF531F73AC124CA814BA30CE805A157D6ADB9EC04FC99AAE6FDC4238FCD76B87CE22BC2FE33B2E5C9

String ALGORITHM = "HmacSHA256";
String hash = null;
try {
    SecretKeySpec keySpec = new SecretKeySpec(
        presharedKey.getBytes(),
        ALGORITHM);
    Mac mac = Mac.getInstance(ALGORITHM);
    mac.init(keySpec);
    byte[] result = mac.doFinal(concat.getBytes());
    hash = Base64.encodeToString(result, Base64.DEFAULT);
    // FpDE2JLmCBr+/rW+n/jBHH13F8AV80sUM2fQAY2IpRs=
} catch (NoSuchAlgorithmException x) {
} catch (InvalidKeyException x) {
}

String encKey = presharedKey.substring(0, presharedKey.length() / 2);
// f8250b0d5960444e4de6ecc3a78900bb941246a1dece7848fc72b90092ab3ecd

int len = encKey.length();
byte[] encKeyBytes = new byte[len / 2];
for (int i = 0; i < len; i += 2) {
    encKeyBytes[i / 2] = (byte) ((Character.digit(encKey.charAt(i), 16) << 4)
            + Character.digit(encKey.charAt(i+1), 16));
}

String encryptedSyncKey = null;
try {
    byte[] iv = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
    AlgorithmParameterSpec ivSpec = new IvParameterSpec(iv);
    SecretKeySpec encKeySpec = new SecretKeySpec(encKeyBytes, "AES");
    Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
    cipher.init(Cipher.ENCRYPT_MODE, encKeySpec, ivSpec);
    byte[] encryptedSyncKeyBytes = cipher.doFinal(syncKey.hexString().getBytes());
    encryptedSyncKey = Base64.encodeToString(encryptedSyncKeyBytes, Base64.DEFAULT);
    /*
        Yrl0/SuTUUTC6oJ8o4TCOy65EwO0JzoXfEi9kLq0AOlf6rH+nN7+BEc0s5uE7TIo1UlJb/DvR2Ca
        ACmQVXXhgpZUTB4sQ0eSo+t32lg0EEb9xKI5CZ4l9QO5raw0xBn7r/tfIdVm8AIFkN9QCcthS0DF
        KH3oWhpwNS+tfEuibLPgGqP/zGTozmido9U9lb4n
    */
} catch (InvalidAlgorithmParameterException e) {
} catch (NoSuchAlgorithmException e) {
} catch (NoSuchPaddingException e) {
} catch (InvalidKeyException e) {
} catch (IllegalBlockSizeException e) {
} catch (BadPaddingException e) {
}

sendStuffToWeb(encryptedSyncKey, deviceId, hash);

Python:

hash = getHash(request)
# hash from Java: FpDE2JLmCBr+/rW+n/jBHH13F8AV80sUM2fQAY2IpRs=

encrypted_sync_key = getEncSyncKey(request)
# encryptedSyncKey from Java:
# Yrl0/SuTUUTC6oJ8o4TCOy65EwO0JzoXfEi9kLq0AOlf6rH+nN7+BEc0s5uE7TIo1UlJb/DvR2Ca
# ACmQVXXhgpZUTB4sQ0eSo+t32lg0EEb9xKI5CZ4l9QO5raw0xBn7r/tfIdVm8AIFkN9QCcthS0DF
# KH3oWhpwNS+tfEuibLPgGqP/zGTozmido9U9lb4n

device_id = getDeviceId(request)
# 1605788742789230

preshared_key = getPresharedKeyFromDevice(deviceId)
# f8250b0d5960444e4de6ecc3a78900bb941246a1dece7848fc72b90092ab3ecd0c1c8e36fddba501ef92e72c95b47e07f98f7fd9cb63da75c008a3201124ea5d

enc_key = preshared_key[:len(preshared_key)/2]
# f8250b0d5960444e4de6ecc3a78900bb941246a1dece7848fc72b90092ab3ecd

aes = AES.new(enc_key.decode('hex'), AES.MODE_CBC, IV="\x00"*16)
sync_key = aes.decrypt(base64.b64decode(encrypted_sync_key))
# 824C1EE9EF507B52EA28362C71BD4AD512A5F82ACFAE80DEF531F73AC124CA814BA30CE805A157D6ADB9EC04FC99AAE6FDC4238FCD76B87CE22BC2FE33B2E5C9

concat = sync_key
# 824C1EE9EF507B52EA28362C71BD4AD512A5F82ACFAE80DEF531F73AC124CA814BA30CE805A157D6ADB9EC04FC99AAE6FDC4238FCD76B87CE22BC2FE33B2E5C9

import hashlib
from hmac import new as hmac

verify_hash = hmac(preshared_key, concat, hashlib.sha256).digest().encode('base64')
# IoSc2w2sQ4/fwhJTdUQHw/Hdyjy+ranzQ1z3J5LfYbA=

From the debug output below you can see the syncKey is encrypted and decrypted successfully, and the concat is identical. However the resulting hash ends up being different.

DanH
  • 5,498
  • 4
  • 49
  • 72
  • 1
    Try changing `key.getBytes()` to `key.getBytes("US-ASCII")`; if that doesn't work, then try `key.getBytes("ISO-8859-1")` – Zim-Zam O'Pootertoot May 10 '13 at 13:46
  • Thanks, I experimented with a number of different encodings for the key as per your advice, however all options seem to provide the same hash string. I also tried the same thing on the python side, however this also provides the same mismatching hash. – DanH May 14 '13 at 02:46
  • This probably isn't relevant, but it looks like you're using a padding scheme in the Java code, and not accounting for it on the Python side? PyCrypto does _not_ handle that padding by itself. Also, the Python code as shown isn't valid: params doesn't contain a key named 'serial_number', but rather has 'device_id'. – Colin Valliant May 14 '13 at 09:48
  • @AlcariTheMad: Thanks fixed the serial_number ref. I'll look into the padding you mention. – DanH May 14 '13 at 10:11
  • @AlcariTheMad: I guess you're referring to `AES/CBC/PKCS5Padding`, however doesn't the fact that the syncKey decrypts fine on Python and matches the original signify this padding is not significant to the problem? I'll pursue this train of thought any way, but the fact that it seems to match already leaves me skeptical. Thanks though. – DanH May 14 '13 at 10:28
  • 1
    @DanH Yes; I said it's probably not relevant because it's decrypting correctly, but if you're going to be using that type of encryption on other data (that's not a multiple of the block size) later, you'll need to handle it. I don't really know Java, but everything on the Python side looks fine. – Colin Valliant May 14 '13 at 10:52
  • Can you come up with a more minimal example of this? Pare each code sample to the smallest version that shows the behavior? Also, show each intermediate value so we can verify each step. – agf May 14 '13 at 14:54
  • Please share the code for the getHash function. The rest of the Python code is extraneous to the question, isn't it? – Chris Johnson May 14 '13 at 21:47
  • @ChrisJohnson `getHash` simply pulls the hash value from the request which is sent from Java. So later on Python can compare `getHash` to the hash it builds internally to verify that it is using the same preshared_key as Java. – DanH May 15 '13 at 02:05
  • @agf Good idea, I've reduced the complexity by removing all the values from `concat` except the decrypted `syncKey` so I'm only trying to hash that now. Also added intermediate values. – DanH May 15 '13 at 02:40

1 Answers1

9

Your Python code is wrong. I can reproduce, in Python, the answer you got in Java.

If I use your inputs:

>>> preshared_key_hex
b'f8250b0d5960444e4de6ecc3a78900bb941246a1dece7848fc72b90092ab3ecd0c1c8e36fddba501ef92e72c95b47e07f98f7fd9cb63da75c008a3201124ea5d'
>>> concat_hex
b'824C1EE9EF507B52EA28362C71BD4AD512A5F82ACFAE80DEF531F73AC124CA814BA30CE805A157D6ADB9EC04FC99AAE6FDC4238FCD76B87CE22BC2FE33B2E5C9'

I get the same value you get in Java:

>>> base64.b64encode(hmac.new(preshared_key_hex, concat_hex, hashlib.sha256).digest())
b'FpDE2JLmCBr+/rW+n/jBHH13F8AV80sUM2fQAY2IpRs='

However, that value is probably also wrong. You should almost certainly be hex decoding the input values.

I'm unable to reproduce what you got in Python; one of the values you're passing to hmac.new isn't what you think it is. print them immediately before calling hmac.new and you should see what doesn't match.

agf
  • 171,228
  • 44
  • 289
  • 238
  • Sorry I don't see where in my Python I'm decoding `preshared_key` or `concat`? I'm only decoding for the decryption. For the record I'm using Python 2.7, I guess you're using 3+ given the `b''` notation? Does that matter? Sorry if I'm being a noob, I just can't see it :) – DanH May 15 '13 at 07:30
  • 3
    @DanH I can't reproduce what you got in Python at all. I was just mentioning those values should probably be hex decoded. There aren't any Python 2/3 differences here. Regardless of encoding, you're making a mistake in Python somewhere. If you print out the values before calling `hmac.new` you will see that they don't match what you're passing it in Java -- since if you pass those values, as I did, you get the same answer as in Java. So your immediate problem is in Python. – agf May 15 '13 at 15:29
  • OK I'll have another look, the code doesn't even make sense to me anymore! It's just pixels D: – DanH May 16 '13 at 07:38
  • Unfortunately not yet, however I've been removed from the project frustratingly so I'm not sure I'll get around to a solution before the bounty lands, thanks for your help up til now :) – DanH May 20 '13 at 08:51