Passing gzencoded data to mcrypt_encrypt, can't unzip after decryption

Question

To keep this short, are there known issues when passing results of gzencode (or other non-text data) to mcrypt_encrypt functions?

Details:

Basically I have an issue where encryption/decryption is working for plain text but I get errors unzipping if I pass compressed data to the encryption function, then decrypt and unzip.

So in PHP, I'm passing results of gzencode() to the encrypt function. Then I base64 encode for showing results on a web service web page. Then in a Java app I'm decoding base64, decrypting, and unzipping using GZIPInputStream. I get errors in the last step.

But everything works fine if I skip the compression step (just pass plain text to the encrypt function). Everything also works fine if I skip encryption and just do compression. So those functions seem to work fine on both PHP and Java sides if I don't combine them.

public static function encrypt($str,$key,$iv) {
    $str=Crypto2::pkcs5Pad($str,mcrypt_get_block_size(MCRYPT_RIJNDAEL_128, MCRYPT_MODE_CBC));
    $encrypted=mcrypt_encrypt(MCRYPT_RIJNDAEL_128,$key,$str,MCRYPT_MODE_CBC,$iv);
    return $encrypted;
}

public static function pkcs5Pad ($text, $blocksize) {
    $pad = $blocksize - (strlen($text) % $blocksize);
    $padded=$text . str_repeat(chr($pad), $pad);
    return $padded;
}

Java functions:

public static byte[] decrypt(byte[] inputbuffer) throws Exception {
    Key key = new SecretKeySpec(keybyte, "AES");
    IvParameterSpec ivSpec = new IvParameterSpec(iv);
    Cipher c = Cipher.getInstance("AES/CBC/PKCS5Padding");
    c.init(Cipher.DECRYPT_MODE, key, ivSpec);

    c.getBlockSize();
    System.out.println("Block size="+c.getBlockSize());

    int outlen = c.getOutputSize(inputbuffer.length);
    System.out.println("Output length will be:"+outlen);
    byte[] result=c.doFinal(inputbuffer);
    return result;
}

  public static byte[] decodeBase64(String data) throws IOException{ 
    BASE64Decoder decoder = new BASE64Decoder(); 
    byte[] decodedBytes = decoder.decodeBuffer(data);

    return decodedBytes;
  }


   public static void unzipPrint(byte[] data) throws Exception{
    InputStream is=new GZIPInputStream(new ByteArrayInputStream(data));
    int ch2;
    while((ch2=is.read())!=-1) {
        System.out.print((char)ch2);
    }
   }

So if I do this in PHP: base64_encode(encrypt(gzencode($plain_text)));

and this in Java

unzipPrint(decrypt(decodeBase64(data)));

I get the dreaded: "java.util.zip.ZipException: oversubscribed dynamic bit lengths tree" during the unzip phase.

Again, if I skip the compression/decompression steps at both ends everything is fine. And if I skip encryption at both ends then compression/decompression works fine.

EDIT: Ok weird, but after checking byte by byte the resulting byte array of the compressed data (after decoding base64 and decryption), I found a SINGLE byte that was off (compared to the original PHP byte array) by a value of 1. It was byte number 14 (index 13 in Java) and it had a value of 110 instead of 111. I have absolutely no idea how this could be the case.

So if I change that single byte to from 110 to 111, then I can successfuly use GZIPOutputStream to uncompress the data.

So I know what is wrong but not why.

EDIT 2: This is SOLVED ->Thanks to comment by Owlstead I double checked the IV values and found that there was minor descrepancy between the php and java code. How this can lead to only one byte of difference in the resulting decrypted data I have no idea.

That was one wasted day over a single 0x13 instead of 0x12 in my IV.

I would verify the strings are truly the same before/after just compression or just encryption. They might *look* the same, but verify with a checksum. — goat, Apr 22 '12 at 14:47
How do you pass the IV? Are you sure the first block decrypts correctly? — Maarten Bodewes, Apr 22 '12 at 16:26
@owlstead> IV is hardcoded in both php and java files. Also as indicated I can successfully encrypt/decrypt from php/java if I skip the compression step. — Fraggle, Apr 22 '12 at 16:46
@chris: See my recent Edit. One byte difference between the original compressed data and the decoded/decrypted compressed data. No idea why still. — Fraggle, Apr 22 '12 at 20:30
@owlstead-> YOU WIN THE PRIZE. Problem is solved. You were right to double check the IV. I found one value was different. How can different IV lead to successful decryption of all but one byte of data (see my edit above)? I would have thought it would have messed up everything. Instead my other successful tests lead me to think the problem had nothing to do with encryption code. — Fraggle, Apr 22 '12 at 21:20
Obviously hardcoding the IV is not the way to go, but this was just a test that ended up not working even though I tried to keep things simple. — Fraggle, Apr 22 '12 at 21:28
The vector is the cipher text for all but the first block, where it is the IV. As the cipher text did not change, nor the key, all but the next blocks decrypt correctly. See wikipedia CBC schematic to see why. — Maarten Bodewes, Apr 23 '12 at 00:14

score 2 · Accepted Answer · answered Apr 23 '12 at 00:18

2

You should check the IV, as it may change only the first block of cipher text, which holds the ZIP header.

(to close the question, glad you got it solved, any day that you solve an issue is not one wasted in my opinion :) )

answered Apr 23 '12 at 00:18

Maarten Bodewes

90,524
13
150
263

Passing gzencoded data to mcrypt_encrypt, can't unzip after decryption

1 Answers1