BadPaddingException: Data must start with zero - Why?

Question

I need to write a java code which encrypt/decrypt messages using a private/public key given by a customer. Here is what I did:

• I got from the customer the private key which is in PKCS#1 format (It starts with -----BEGIN RSA PRIVATE KEY-----)

• I converted it into PKCS#8 using the following command:

  openssl pkcs8 -topk8 -nocrypt -in private_pkcs1.key -out private_pkcs8.key
  

• Now I have the private key in PKCS#8 format (which starts with -----BEGIN PRIVATE KEY-----)

• I have extracted the public key out of the private key (tried with both formats, and I got the same public key) using the following command:

  openssl rsa -in private_pkcs8.key -pubout > public.key
  

• I have written the following method to read the private key the private_pkcs8.key file:

  private static PrivateKey readPrivateKey(String filename) throws IOException, NoSuchAlgorithmException, InvalidKeySpecException
  {
  File file = new File(filename);
  FileInputStream in = new FileInputStream(file);
  byte[] keyBytes = new byte[ new Long(file.length()).intValue() ];
  in.read(keyBytes);
  in.close();
  
  String strPrivateKey = new String(keyBytes, "UTF-8").replaceAll("(-+BEGIN PRIVATE KEY-+\\r?\\n|-+END PRIVATE KEY-+\\r?\\n?)", "");
  PKCS8EncodedKeySpec keySpec = new PKCS8EncodedKeySpec(DatatypeConverter.parseBase64Binary(strPrivateKey));
  KeyFactory keyFactory = KeyFactory.getInstance("RSA");
  return keyFactory.generatePrivate(keySpec);
  }
  

• I have written the following method to read the public key from the public.key file

  private static PublicKey readPublicKey(String filename) throws NoSuchAlgorithmException, InvalidKeySpecException, IOException
  {
  File file = new File(filename);
  FileInputStream in = new FileInputStream(file);
  byte[] keyBytes = new byte[ new Long(file.length()).intValue() ];
  in.read(keyBytes);
  in.close();
  
  String strPublicKey = new String(keyBytes, "UTF-8").replaceAll("(-+BEGIN PUBLIC KEY-+\\r?\\n|-+END PUBLIC KEY-+\\r?\\n?)", "");
  X509EncodedKeySpec keySpec = new X509EncodedKeySpec(DatatypeConverter.parseBase64Binary(strPublicKey));
  KeyFactory keyFactory = KeyFactory.getInstance("RSA");
  return keyFactory.generatePublic(keySpec);
  }
  

• I have written the following encrypt method

  private static byte[] encrypt(byte[] message, Key key) throws NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException, NoSuchProviderException, InvalidAlgorithmParameterException
  {
  Cipher cipher = Cipher.getInstance(ALGORITHEM);
  cipher.init(Cipher.ENCRYPT_MODE, key);
  
  return cipher.doFinal(message);
  }
  

• I have written the following decrypt method

  private static byte[] decrypt(byte[] message, Key key) throws NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException, NoSuchProviderException, InvalidAlgorithmParameterException
  {
  Cipher cipher = Cipher.getInstance(ALGORITHEM);
  cipher.init(Cipher.DECRYPT_MODE, key);
  
  return cipher.doFinal(message);
  }
  

• I have written the following main test application which tries to encrypt a simple string and decrypt it back.

  public static void main(String[] args) throws NoSuchAlgorithmException, InvalidKeySpecException, IOException, InvalidKeyException, NoSuchPaddingException, IllegalBlockSizeException, BadPaddingException, SignatureException, NoSuchProviderException, InvalidAlgorithmParameterException 
  {
  readKeys();
  
  String messageToEncrypt = "This is the string I want to test";
  
  log("Original Message: " + messageToEncrypt);
  byte[] originalBytes = messageToEncrypt.getBytes("UTF-8");
  
  byte[] encriptedBytes = encrypt(originalBytes, privateKey);
  byte[] decriptedMessage = decrypt(encriptedBytes, publicKey);
  
  log("Decripted Message: " + new String(decriptedMessage));
  }
  

• The encryption was OK, but when I try to decrypt it back to the original message I got the following exception:

  Exception in thread "main" javax.crypto.BadPaddingException: Data must start with zero
  at sun.security.rsa.RSAPadding.unpadV15(Unknown Source)
  at sun.security.rsa.RSAPadding.unpad(Unknown Source)
  at com.sun.crypto.provider.RSACipher.a(DashoA13*..)
  at com.sun.crypto.provider.RSACipher.engineDoFinal(DashoA13*..)
  at javax.crypto.Cipher.doFinal(DashoA13*..)
  at com.company.encryption.utils.RsaUtils.decrypt(RsaUtils.java:95)
  at com.company.encryption.utils.RsaUtils.main(RsaUtils.java:108)
  ERROR: JDWP Unable to get JNI 1.2 environment, jvm->GetEnv() return code = -2
  JDWP exit error AGENT_ERROR_NO_JNI_ENV(183):  [../../../src/share/back/util.c:820]
  

Note that the JVM also crashes, which is very strange.

• When I have replaced the private/public keys and used KeyPairGenerator instead, all was OK. This brings me to the conclusion that something is wrong with the keys, or the way I read the keys, or the algorithm I use to encrypt / decrypt. ( I have tried with "RSA", "RSA/ECB/PKCS1Padding" - with no luck)

I have spent more than 2 days on this issue. I made sure I use the same public/private pair of keys, I didn't try to convert byte[] to String. I have done everything "by the book"

Any help on this issue will be highly appreciated. Is there some question I could ask my customer about the keys, that will help me solve this mystery? Why the JVM also crashes?

Thanks Guy Hudara

Answer 1

In asymmetric ciphers one encrypts with public key and decrypts with private key. This way it ensures that anyone can encrypt the message with commonly known public key, but only holder of the private key may decrypt it.

byte[] encryptedBytes = encrypt(originalBytes, publicKey);
byte[] decryptedBytes = decrypt(encryptedBytes, privateKey);

Answer 2

I has the same problem. Solved by regenerating public key.