Cipher decryption / encryption changing results

Question

I'm reverse engineering some code which is decrypting data, hoping I'll be able to encrypt it back and obtain the same data it started with, for reasons that would make this question too long and off-topic.

public void Test() throws Exception {

    String pk_enc = //...

    String hashStr_64 = //...

    byte[] hashStr_encrypted = Base64.decode(hashStr_64);

    X509EncodedKeySpec e = new X509EncodedKeySpec(Base64.decode(pk_enc));
    KeyFactory keyFactory = KeyFactory.getInstance("RSA");
    RSAPublicKey RSApublicKey = (RSAPublicKey) keyFactory.generatePublic(e);
    Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1PADDING");
    cipher.init(2, RSApublicKey); // '2' means decrypt
    byte[] hashStr_decrypted = cipher.doFinal(hashStr_encrypted);
    String hashStr_result = new String(hashStr_decrypted);

    // Now in reverse...
    Cipher cipher1 = Cipher.getInstance("RSA/ECB/PKCS1PADDING");
    // instantiating a new cipher or using the original one makes no difference
    cipher1.init(1, RSApublicKey); // '1' means encrypt
    byte[] hashStr_encrypted_reverse = cipher1.doFinal(hashStr_decrypted);
    String hashStr_64_reverse = Base64.encode(hashStr_encrypted_reverse);
}

All the code before // Now in reverse... cannot be changed, but that doesn't mean it's impossible to convert hashStr_result back to hashStr_64, right?

However, the code I've wrote after, that should do just that, doesn't work. hashStr_encrypted_reverse is different from hashStr_encrypted. Why is that and how can I fix it?

Another sign that something went wrong in the encryption is what happens if I try to decrypt again...

// Decrypt again
Cipher cipher2 = Cipher.getInstance("RSA/ECB/PKCS1PADDING");
cipher2.init(2, RSApublicKey);
byte[] hashStr_decrypted_again = cipher.doFinal(hashStr_encrypted_reverse);

This throws:

javax.crypto.BadPaddingException

I don't really care, but maybe it could help answer the question.

That's not how RSA works. One operation uses the public key, the other operation uses the private key. You can't do both operations with the public key. — President James K. Polk, Nov 03 '18 at 01:56
That's something I thought about, but you'll see that we have a *public* key actually *decrypting* data. My limited understanding is that with an RSA public key one can only encrypt, not decrypt. So I thought this works differently from the RSA I know (or that surely it would be possible to encrypt it back). Thanks for the tip: I'll research more in this direction! — Saturnix, Nov 03 '18 at 02:00

score 2 · Answer 1 · answered Nov 03 '18 at 02:09

As @JamesKPolk said in his comment, this isn't how RSA works. RSA is an asymmetric encryption algorithm: there are two keys, public and private. A symmetric algorithm (e.g., AES) has a single key, which is used for both encryption and decryption, and that key must be kept safe, except to the sending and receiving parties.

Why asymmetric encryption?

You can encrypt with a public key (typically someone else's key that they've shared with you), and they must use their private key to decrypt it. Anyone can have the public key (that's why it's public), but it cannot be used to read the encrypted message. This is where you are having your problem.
Also, you can encrypt a message with a private key (typically your own), and anyone else can use your public key to decrypt it. This is how digital signatures are implemented: for example, you would encrypt a digest of a document, and anyone can verify your signature if they have your public key, but no one else could have signed it.

score 2 · Accepted Answer · answered Nov 03 '18 at 02:34

Terminology will be confusing. There are 4 RSA operations, best described as: signing, verifying, encrypting, decrypting. Mapping these to a lower-level and using only the language of encryption and decryption, these map as follows:

sign-verify pair

signing -> encrypt with private key
verifying -> decrypt with public key

encrypt-decrypt pair

encrypting -> encrypt with public key
decrypting -> decrypt with private key.

As you can see, each pair of operations has the private key on one side and the public key on the other.

Cipher decryption / encryption changing results

2 Answers2