Android Decryption: Last block (16 bytes) is missing in result file

Question

I'm currently working on a decryption algorithm to decrypt a PDF I'm receiving from the server for an eBook reader app. There is equivalent code which runs on iOS which works perfectly, I'm now trying to make the code run on Android.

Just some details, the decryption is AES and runs in ECB mode. The encryption key is a hex string array which I convert into a byte array (by converting every two chars into a byte, eg: "FF" becomes 255, etc).

What I'm experiencing is very interesting. I'm comparing the result files after decryption from both iOS and Android, and I'm noticing that consistently, the Android decryption files are 16 bytes shorter than the iOS decryption files, specifically in the end. All other bytes are identical (I will post below some examples).

This difference is causing my eBook reader to refuse to open the PDFs, while it successfully opens the iOS books.

Here is my decryption code:

private void performDecryption(DocumentModel document)
    {                               
        byte[] keyBytes = generateByteArray(document.getEncryptionKey());


        SecretKeySpec skeySpec = new SecretKeySpec(keyBytes, "AES");

        File encryptedDocument = new File(getBookFolderDocumentName(document, document.getFileSuffix()));
        File decryptedDocument = new File(BOOK_FOLDER + document.getGeneratedAssetName() + "_decrypted" + "." + document.getFileSuffix());

        decryptedDocument.mkdirs();
        if (decryptedDocument.exists())
            decryptedDocument.delete();


        Cipher cipher = null;    

        try
        {

            cipher = Cipher.getInstance("AES/ECB/ZeroBytePadding");
            cipher.init(Cipher.DECRYPT_MODE, skeySpec);         
        } 
        catch (NoSuchAlgorithmException noSuchAlgorithmEx)
        {
            Log.e("Decryption", "NoSuchAlgorithmException: " + noSuchAlgorithmEx.getMessage());
        }
        catch (NoSuchPaddingException noSuchPaddingEx)
        {
            Log.e("Decryption", "NoSuchPaddingException: " + noSuchPaddingEx.getMessage());
        }
        catch (InvalidKeyException invalidKeyEx)
        {
            Log.e("Decryption", "InvalidKeyException: " + invalidKeyEx.getMessage());
        } 

        FileInputStream encryptedFileStream = null;
        FileOutputStream decryptedFileStream = null;


        try
        {

            encryptedFileStream = new FileInputStream(encryptedDocument);
            decryptedFileStream = new FileOutputStream(decryptedDocument);



            long totalFileSize = encryptedDocument.length();
            long totalDecrypted = 0;
            int lastPercentage = -1;
            int currentPercentage = 0;

            byte[] encryptedBuffer = new byte[4096];
            byte[] decryptedBuffer = new byte[4096];
            int encryptedLength = 0;
            int decryptedLength = 0;

            while((encryptedLength = encryptedFileStream.read(encryptedBuffer)) > 0)
            {   
                while (encryptedLength % 16 != 0) // the code never lands in this loop
                {                   
                    encryptedBuffer[encryptedLength] = 0;
                    encryptedLength++;
                }

                decryptedLength = cipher.update(encryptedBuffer, 0, encryptedLength, decryptedBuffer);


                while (decryptedLength % 16 != 0) // the code never lands in this loop
                {
                    decryptedBuffer[decryptedLength] = 0;
                    decryptedLength++;
                }

                decryptedFileStream.write(decryptedBuffer, 0, decryptedLength);


                totalDecrypted += encryptedLength;

                currentPercentage = (int)(((float)totalDecrypted / (float)totalFileSize) * 100f);

                if (currentPercentage != lastPercentage)
                {
                    lastPercentage = currentPercentage;
                    Log.i("Decryption", "Decrypting... " + currentPercentage + "%");
                }
            }




            Log.i("Decryption", "Finished decrypting!");
        }
        catch (FileNotFoundException fileNotFoundEx)
        {
            Log.e("Decryption", "FileNotFoundException: " + fileNotFoundEx.getMessage());
        }
        catch (IOException ioEx)
        {
            Log.e("Decryption", "IOException: " + ioEx.getMessage());
        } 
        catch (ShortBufferException e) 
        {       
            e.printStackTrace();
        }
        finally
        {

        }

        try 
        {                   
            encryptedFileStream.close();
            decryptedFileStream.close();
            cipherOutputStream.close();         
        } 
        catch (IOException e1) 
        {

        }


        document.setDecryptedFilePath(decryptedDocument.getAbsolutePath());



        Log.i("Decryption", "Finished!");
    }

Here are some samples from the pdf files (I used a hex reader to get these results):

Book 1 (iOS):

0D 0A 3C 3C 2F 53 69 7A 65 20 35 38 31 3E 3E 0D 
0A 73 74 61 72 74 78 72 65 66 0D 0A 31 31 36 0D 
0A 25 25 45 4F 46 0D 0A 08 08 08 08 08 08 08 08 <-- this block is missing in android.

Book 1 (Android):

0D 0A 3C 3C 2F 53 69 7A 65 20 35 38 31 3E 3E 0D 
0A 73 74 61 72 74 78 72 65 66 0D 0A 31 31 36 0D

Book 2 (iOS):

65 6E 64 6F 62 6A 0D 73 74 61 72 74 78 72 65 66 
0D 0A 34 30 36 32 35 33 36 0D 0A 25 25 45 4F 46 
0D 0A 0E 0E 0E 0E 0E 0E 0E 0E 0E 0E 0E 0E 0E 0E <-- this block is missing in android.

Book 2 (Android):

65 6E 64 6F 62 6A 0D 73 74 61 72 74 78 72 65 66 
0D 0A 34 30 36 32 35 33 36 0D 0A 25 25 45 4F 46

What I'm noticing is that the last bytes have several identical bytes at the end, equal to the amount of times they appear. In book 1 iOS, in the last block, the byte 08 appears exactly 8 times. In book 2 iOS, in the last block, the byte 0e (14) appears exactly 14 times, etc.

Other than that, I'm not sure what pattern is occurring, so I'm not sure how I can solve this.

I've already tried using the following different padding types:

ZeroBytePadding, NoPadding, PKCS5Padding, PKCS7Padding

Any ideas would be greatly appreciated.

score 4 · Accepted Answer · answered Oct 30 '13 at 08:25

4

Ok, I figured out the problem... It didn't have anything to do with the padding type or anything like that. Apparently, using Cipher.update() will work fine up until the final block of data. The update() method will omit this.

This means, that when finishing the decryption process, you MUST call doFinal(), or the final bytes won't get written.

Here's the code I added immediately after my large while loop which performs the decryption:

byte[] finalBytes = cipher.doFinal();

byte[] finalBytesPadded = new byte[16];

for (int i = 0; i < finalBytes.length; i++)
{
    finalBytesPadded[i] = finalBytes[i];
}

for (int i = finalBytes.length; i < 16; i++)
{
    finalBytesPadded[i] = (byte)(16 - finalBytes.length);
}

decryptedFileStream.write(finalBytesPadded);

I couldn'v probably made the code nicer, but there it is. Problem solved :)

answered Oct 30 '13 at 08:25

Gil Moshayof

16,633
4
47
58

Actually i would suspect that you also have an error in the iphone implementation, as you seem to get the padding in the output. Do you decrypt on iphone with no padding option? With padding option the decrypter methode can't know if the decrypted data is actual data or the final padding, until you call the final methode. It will hold on to the last piece of data in the buffer until it know if its padding or real data. – Ebbe M. Pedersen Oct 30 '13 at 10:54
The decryption is done by a 3rd party book supplier. We received the decryption guidelines from them, as well as sample decrypted books to compare with. The extra bytes at the end are supposed to be there according to them. – Gil Moshayof Oct 30 '13 at 15:38
Well, then coincidentally (!) that seems to look exactly like a [PKCS#7 Padding](http://en.wikipedia.org/wiki/Padding_(cryptography)#PKCS7) would look ... – Ebbe M. Pedersen Oct 30 '13 at 15:56
I tried using that padding, but the last bytes were still being omitted. The point here is that you must call doFinal after looping with update. – Gil Moshayof Nov 07 '13 at 11:34
As block ciphers don't have any length fields, the only way to know if the last few decrypted bytes is real data or padding, is when the doFinal() is called. When that is called it can verified that the padding is correct, and the padding is thrown away. Up until the doFinal call, the last 8 bytes in the buffer is potentially a padding, and is thuse not returned to you yet. If the decrypt is done with the no padding option, all decrypted data is real data, and can therefor be returned without the risk of it being padding. – Ebbe M. Pedersen Nov 07 '13 at 22:58
I'm quite sure that the 8 `08` bytes and the 14 `0E` bytes (14 in decimal) in the end of the decrypted data in IOS is actually the padding, and that you only get that back because the decrypt is running without padding. It properly don't matter much for a PDF viewer that there is a bit too much data. – Ebbe M. Pedersen Nov 07 '13 at 23:02
Like I said, I tried using all the paddings I specified and the last 16 bytes were never there. When I called doFinal(), it returned 2 bytes (for the 0E padded book), and 8 bytes (for the 08 padded book), and different variations for other books. There was no way to recreate this without calling doFinal at the end of the loop. – Gil Moshayof Nov 08 '13 at 00:14

score 0 · Answer 2 · edited May 23 '17 at 12:08

0

After some quick googling, it seems that you should use

cipher = Cipher.getInstance("AES/ECB/PKCS5Padding");

instead of

cipher = Cipher.getInstance("AES/ECB/ZeroBytePadding");

It looks like PKCS #7 padding is being used. http://en.wikipedia.org/wiki/Padding_(cryptography)#PKCS7

Android doesn't seem to have a PKCS #7 built in, but this guy says that PKCS #7 padding is compatible with PKCS #5 padding.

edited May 23 '17 at 12:08

Community

1
1

answered Oct 30 '13 at 07:54

Matthew Wesly

1,238
1
13
14

Hey, tnx for the answer, and I probably should'v specified this, but I've already tried all the different padding types. – Gil Moshayof Oct 30 '13 at 08:00
Well, that's unfortunate, that was my guess. I don't really know much about this, but I figured that I would try helping. Good luck! – Matthew Wesly Oct 30 '13 at 08:18
I found out what the problem was if you want to have a look :) – Gil Moshayof Oct 30 '13 at 08:26

Android Decryption: Last block (16 bytes) is missing in result file

2 Answers2