Python Base64 string shorter than Botan Base64 string

Question

I've managed to get AES/Rijndael [256bit key / 128bit block size] symmetric encryption working: encrypt with pycrypto and decrypting with Botan in C++.

However, when I try to base64 encode the encryption result in python, the resulting string is shorter than the same string generated by Botan using a Base64_Encoder. Example:

Botan Base64:

zjjxmJf5KPs183I/EvC+JuNbOdmbm4bWyhLsdZI8fuVUnKQAeSj0ivmKIYu7HBjM7gLgLV+xtSKcsCeQD7Gy4w==

Py-3k Base64:

zjjxmJf5KPs183I/EvC+JuNbOdmbm4bWyhLsdZI8fuVUnKQAeSj0ivmKIYu7HBjM

You can see that the strings are exactly the same up until the 64 character mark. If I try to decrypt the Python base64 string in Botan it complains about "not enough input".

How do I get the Python base64 string to be acceptable by Botan?

-- EDIT -- When decoding the Botan base64 encoded string in Python:

Botan Decoded:[b'\xce8\xf1\x98\x97\xf9(\xfb5\xf3r?\x12\xf0\xbe&\xe3[9\xd9\x9b\x9b\x86\xd6\xca\x12\xecu\x92<~\xe5T\x9c\xa4\x00y(\xf4\x8a\xf9\x8a!\x8b\xbb\x1c\x18\xcc\xee\x02\xe0-_\xb1\xb5"\x9c\xb0\'\x90\x0f\xb1\xb2\xe3']
Botan Encoded:[b'zjjxmJf5KPs183I/EvC+JuNbOdmbm4bWyhLsdZI8fuVUnKQAeSj0ivmKIYu7HBjM7gLgLV+xtSKcsCeQD7Gy4w==']

Thus, the Python pycrypto result:

Encryption result: b'\xce8\xf1\x98\x97\xf9(\xfb5\xf3r?\x12\xf0\xbe&\xe3[9\xd9\x9b\x9b\x86\xd6\xca\x12\xecu\x92<~\xe5T\x9c\xa4\x00y(\xf4\x8a\xf9\x8a!\x8b\xbb\x1c\x18\xcc'

Base64 encoded: b'zjjxmJf5KPs183I/EvC+JuNbOdmbm4bWyhLsdZI8fuVUnKQAeSj0ivmKIYu7HBjM

Python seems to be "omitting" something. But what?

-- EDIT 2 --

When I try to base64decode & decrypt the pycrypto result, Botan throws this:

Botan exception caught: Buffered_Operation::final - not enough input

So pycrypto is not producing "enough" output such that it can be decrypted by Botan.

-- EDIT 3 --- Code examples:

Python: changed sensitive info.

import sys
import base64
import binascii
from Crypto.Cipher import AES

plaintext = "097807897-340284-083-08-8034-0843324890098324948"

hex_key = b'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
key = binascii.unhexlify( hex_key )
hex_iv = b'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
iv = binascii.unhexlify( hex_iv )

aes_enc_bytes = AES.new(key, AES.MODE_CBC, iv).encrypt( plaintext )
aes_enc = base64.encodebytes(aes_enc_bytes )

print( "Encrypted:[{}]".format( aes_enc ) )

aes_dec = AES.new(key, AES.MODE_CBC, iv).decrypt( binascii.a2b_base64( aes_enc ) )
print( "Decrypted:[{}]".format( aes_dec ) )

C++ (Qt + Botan)

void botanDecryptor::decrypt()
{
    Botan::SymmetricKey key( private_key );
    Botan::InitializationVector iv( iv_value );
    try
    {
        // Now decrypt...
        Botan::Pipe dec_pipe(new Base64_Decoder, get_cipher("AES-256/CBC", key, iv, Botan::DECRYPTION));

        dec_pipe.process_msg( ciphertext );

        string decrypted = dec_pipe.read_all_as_string();

        cout << "Decrypted:[" << decrypted << "]" << endl;
    }
    catch(Botan::Exception& e)
    {
        cout << "Botan exception caught: " << e.what() << endl;
        return;
    }

-- EDIT 4 --

I decided to try and decrypt the Botan encrypted, base64 encoded string in python and it worked, but it added a bunch of what looks like padding:

Decrypted:[b'097807897-340284-083-08-8034-0843324890098324948\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10']

I then proceeded to add that padding to my pycrypto result before base64 encoding to produce the following, which Botan refuses to decrypt ;(

zjjxmJf5KPs183I/EvC+JuNbOdmbm4bWyhLsdZI8fuVUnKQAeSj0ivmKIYu7HBjMEBAQEBAQEBAQ\nEBAQEBAQEA==

-- ANSWER -- (system wouldn't allow me to self answer for another 5 hours!)

I've finally schlepped through all the documentation and found the answer! One needs to specify what padding method is to be used for the mode. I specified NoPadding e.g.

Pipe dec_pipe(new Base64_Decoder, get_cipher("AES-256/CBC/NoPadding", key, iv, Botan::DECRYPTION));

and viola! The output matches the pycrypto exactly. For reference: [http://botan.randombit.net/filters.html][1]

[1]: Botan Docs: Cipher Filters

Decode then reencode the botan string it using python give the exact string for me. — Kien Truong, Apr 04 '12 at 10:04
Dikei, you're onto something there. I also processed the string and got the same result. But that then means the encrypted result from pycrypto is not the same as that of Botan: I'll edit to show the example further — ButItWorkedYesterday, Apr 04 '12 at 10:09
@ j13r: I've used both the base64 and binascii modules with little variation and no remarkable success. — ButItWorkedYesterday, Apr 04 '12 at 10:15
The input in the second example is shorter than the first example. — Kien Truong, Apr 04 '12 at 10:25
@ Dikei: exactly. This is what puzzles me. The 256bit KEY and 128bit IV is exactly the same but pycrypto produces shorter output which makes it impossible for me to use the python output to decrypt in Botan :( — ButItWorkedYesterday, Apr 04 '12 at 10:31
@ Dikei: Code posted - Python to encrypt with pycrypto then C++ to decrypt with Botan. — ButItWorkedYesterday, Apr 04 '12 at 10:59
Remove the AES stuff and isolate the issue to base64. Post the simplest case where things go wrong. Perhaps it's also a UTF8/0-termination issue. — j13r, Apr 04 '12 at 11:32

score 0 · Answer 1 · answered Apr 04 '12 at 09:40

0

Base64 can have a padding at the end (see https://en.wikipedia.org/wiki/Base64#Decoding_Base64_with_padding)

Apparently Botans decoder doesn't support this.

Perhaps the binascii module can help you; alternatively you could add/remove the padding yourself.

answered Apr 04 '12 at 09:40

j13r

2,576
2
21
28

That was my initial idea too but the "additional" data in the result string [7gLgLV+xtSKcsCeQD7Gy4w==] decodes, via binascii.b2a_base64 to "N2dMZ0xWK3h0U0tjc0NlUUQ3R3k0dz09\n" which adds nothing to my understanding of its meaning or origin. – ButItWorkedYesterday Apr 04 '12 at 10:29

score 0 · Accepted Answer · edited Apr 18 '12 at 19:32

0

I've finally schlepped through all the documentation and found the answer!

One needs to specify what padding method is to be used for the mode. I specified NoPadding e.g.

Pipe dec_pipe(new Base64_Decoder, get_cipher("AES-256/CBC/NoPadding", key, iv, Botan::DECRYPTION));

and voila! The output matches the pycrypto exactly. Here's the reference.

edited Apr 18 '12 at 19:32

Msonic

1,456
15
25

answered Apr 18 '12 at 10:01

ButItWorkedYesterday

1
4

Python Base64 string shorter than Botan Base64 string

2 Answers2