Reading encrypted private key in PKCS#8 format through bouncycastle, Java failing in docker container

Question

I am trying to read a PKCS#8 private key which looks like following:

key.k8 --> (Sample key. Passphrase - 123456):

Following code is being used to parse the private key:

 InputStream privateKeyInputStream = getPrivateKeyInputStream(); // reads the key file from classpath and share as DataStream
 logger.info("InputStreamExists --> {} ", privateKeyInputStream.available());
 PEMParser pemParser = new PEMParser(new InputStreamReader(privateKeyInputStream));
 Object pemObject = pemParser.readObject();
 if (pemObject instanceof PKCS8EncryptedPrivateKeyInfo) {
     // Handle the case where the private key is encrypted.
     PKCS8EncryptedPrivateKeyInfo encryptedPrivateKeyInfo = (PKCS8EncryptedPrivateKeyInfo) pemObject;
     InputDecryptorProvider pkcs8Prov =
            new JceOpenSSLPKCS8DecryptorProviderBuilder().build(passphrase.toCharArray());
     privateKeyInfo = encryptedPrivateKeyInfo.decryptPrivateKeyInfo(pkcs8Prov); // fails here
}

InputStream resourceAsStream = null;
    if ("local".equals(privateKeyMode)) {
      resourceAsStream = this.getClass().getResourceAsStream(privateKeyPath);
    } else {
      File keyFile = new File(privateKeyPath);
      logger.info(
          "Key file found in {} mode. FileName : {}, Exists : {}",
          privateKeyMode,
          keyFile.getName(),
          keyFile.exists());
      try {
        resourceAsStream = new DataInputStream(new FileInputStream(keyFile));
      } catch (FileNotFoundException e) {
        e.printStackTrace();
      }

When I am running this code through intelliJ on windows, the code works fine but when I run it through docker container I am getting following exception:

org.bouncycastle.pkcs.PKCSException: unable to read encrypted data: failed to construct sequence from byte[]: Extra data detected in stream
snowflake-report-sync    |      at org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo.decryptPrivateKeyInfo(Unknown Source) ~[bcpkix-jdk15on-1.64.jar!/:1.64.00.0]
snowflake-report-sync    |      at com.optum.snowflakereportsync.configuration.SnowFlakeConfig.getPrivateKey(SnowFlakeConfig.java:103) ~[classes!/:na]
snowflake-report-sync    |      at com.optum.snowflakereportsync.configuration.SnowFlakeConfig.getConnectionProperties(SnowFlakeConfig.java:67) ~[classes!/:na]

Following is Dockerfile used:

FROM adoptopenjdk/openjdk11-openj9:latest
COPY build/libs/snowflake-report-sync-*.jar snowflake-report-sync.jar
RUN mkdir /encryption-keys
COPY encryption-keys/ /encryption-keys/ #keys are picked from docker filesystem when running in container
EXPOSE 8080
CMD java -Dcom.sun.management.jmxremote -noverify ${JAVA_OPTS} -jar snowflake-report-sync.jar

Options tried:

Ensured that key file is being read while running in container. Logger "InputStreamExists --> {}" gives number of bytes
Ran dos2unix on key.k8 just to make sure there are no Window's "^M" characters which be could be causing issue as container is linux one : FROM adoptopenjdk/openjdk11-openj9:latest

Not sure what I am doing wrong but any help or pointers would be appreciated.

That's something I tried after googling it a bit.. doesn't make a difference though.. The problem is same even if you feed InputStream directly. — Sukhmeet Sethi, Jan 23 '22 at 12:04
What type of key is it? RSA, DSA, AES, ect? Maybe generate a dummy key of the same algorithm and provide that? — Magnus, Jan 27 '22 at 08:07
It's an RSA Key... I am able to extract the private key through Openssl and for now using that: `openssl pkcs8 -in encryption-keys/rsa_key.p8 -passin pass:xxxxxx` — Sukhmeet Sethi, Jan 27 '22 at 08:36
Could you please provide the command you used to generate the key? — Magnus, Jan 27 '22 at 08:56
Unfortunately I haven't generated the key but have got it from the service provider and idea is to pass it request while establishing connection with Snowflake cluster. — Sukhmeet Sethi, Jan 27 '22 at 09:43
@Magnus perhaps these commands can give a similar key: `openssl genpkey -out rsakey.pem -algorithm RSA -pkeyopt rsa_keygen_bits:2048` followed by `openssl pkcs8 -in rsakey.pem -topk8 -out enckey.pem` — Sukhmeet Sethi, Jan 27 '22 at 09:49
Thanks, unfortunately I haven't been able to reproduce the issue. If you generate a new key using that command, does your code still fail on Linux? If you could provide a minimal example that reliably reproduces the issue, I could look into it further. One thing you could try is specifying a fixed charset for the inpustreamreader, but that's just a guess. — Magnus, Jan 27 '22 at 10:32
Thanks for looking into it @Magnus. I tried with the key generated through the commands shared but the error was still the same. I am adding the key along with few other details in the question in case it helps to replicate — Sukhmeet Sethi, Jan 28 '22 at 07:53
By being unresponsive towards offered solutions, now half of your bounty has gone to waste, I automatically got the other half, simply because I had one upvote more than the other guy. Maybe you would have decided the same way, maybe not. Either way, it is a waste like this. I got 25 extra points, but no answer has been accepted and you let a computer decide about who should get (half of) your bounty. Hmm... — kriegaex, Jan 31 '22 at 09:33

kriegaex · Accepted Answer · 2022-01-31T01:18:09.517

Like @Bragolgirith suspected, BouncyCastle seems to have problems with OpenJ9. I guess it is not a Docker issue, because I can reproduce it on GitHub Actions, too. It is also not limited to BouncyCastle 1.64 or 1.70, it happens in both versions. It also happens on OpenJ9 JDK 11, 14, 17 on Windows, MacOS and Linux, but for the same matrix of Java and OS versions it works on Adopt-Hotspot and Zulu.

Here is an example Maven project and a failed matrix build. So if you select another JVM type, you should be fine. I know that @Bragolgirith already suggested that, but I wanted to make the problem reproducible for everyone and also provide an MCVE, in case someone wants to open a BC or OpenJ9 issue.

P.S.: It is also not a character set issue with the InputStreamReader. This build fails exactly the same as before after I changed the constructor call.

Update: I have created BC-Java issue #1099. Let's see what the maintainers can say about this.

Update 2: The solution to your problem is to explicitly set the security provider to BC for your input decryptor provider. Thanks to David Hook for his helpful comment in #1099.

BouncyCastleProvider securityProvider = new BouncyCastleProvider();
Security.addProvider(securityProvider);

// (...)

InputDecryptorProvider pkcs8Prov = new JceOpenSSLPKCS8DecryptorProviderBuilder()
  // Explicitly setting security provider helps to avoid ambiguities
  // which otherwise can cause problems, e.g. on OpenJ9 JVMs
  .setProvider(securityProvider)
  .build(passphrase.toCharArray());

See this commit and the corresponding build, now passing on all platforms, Java versions and JVM types (including OpenJ9).

Because @Bragolgirith mentioned it in his answer: If you want to avoid the explicit new JceOpenSSLPKCS8DecryptorProviderBuilder().setProvider(securityProvider), the call Security.insertProviderAt(securityProvider, 1) instead of simply Security.addProvider(securityProvider) would in this case also solve the problem. But this holds true only as long as no other part of your code or any third-party library sets another provider to position 1 afterwards, as explained in the Javadoc. So maybe it is not a good idea to rely on that.

**Update 2:** The solution to your problem is to explicitly set the security provider to BC for your input decryptor provider. See my updated answer. — kriegaex, Jan 31 '22 at 00:59
Thanks, it is an issue with openj9 and both solutions of either using different base image or providing security provider exclusively works. Also thanks for taking pain to create a BC java issue to get more pointers. — Sukhmeet Sethi, Jan 31 '22 at 13:39
Yes, it is an issue with OpenJ9, but not a bug in OpenJ9 or BC. It is actually a user error, relying on platform-dependent defaults too much. But this was really hard to diagnose for people like you and me who do not use BC often. I only used it once before for signing and encrypting e-mails, and never tested it on OpenJ9 before. But this really taught me a lesson, too. BTW, I added more diagnostic output to my example and quoted it in the GitHub read-me. You can see the difference in the cipher's crypto provider (SunJCE vs. BC) in the log output now, but I had to use reflection. — kriegaex, Feb 01 '22 at 03:36

Bragolgirith · Answer 2 · 2022-01-30T12:33:12.450

Edit:

On second thought, when creating the JceOpenSSLPKCS8DecryptorProviderBuilder, you're not explicitly specifying the provider:

new JceOpenSSLPKCS8DecryptorProviderBuilder()
    .setProvider(BouncyCastleProvider.PROVIDER_NAME) // add this line
    .build(passphrase.toCharArray());

It seems OpenJ9 uses a different provider/algo selection mechanism and selects the SunJCE's AESCipher class as CipherSpi by default, while Hotspot selects BouncyCastleProvider's AES class.

Explicitly specifying the provider should work in all cases.

Alternatively, when adding the BouncyCastleProvider you could insert it at the first preferred position (i.e. Security.insertProviderAt(new BouncyCastleProvider(), 1) instead of Security.addProvider(new BouncyCastleProvider())) so that it gets selected.

(It's still unclear to me why the provider selection mechanism differs between the different JVMs.)

Original post:

I've managed to reproduce the issue and at this point I'd say it's an incompatibility issue with the OpenJ9 JVM.

Starting from a Hotspot base image instead, e.g.

FROM adoptopenjdk:11-jre-hotspot

makes the code work.

(Not yet entirely sure whether the fault lies with the Docker image itself, the OpenJ9 JVM or BouncyCastle)

After just having updated my answer and the corresponding proof of concept (PoC) on GitHub in a similar way to how you updated yours, I only noticed now that yours has changed, too, because you did not add a comment notifying subscribed users about it. It seems as if we have arrived at the same conclusion independently. Let's see if the OP prefers my more verbose and PoC-supported one or your more compact and equally correct one. Kudos! — kriegaex, Jan 31 '22 at 01:25
Thanks @Bragolgirith. Your solution works too. Appreciate your effort. — Sukhmeet Sethi, Jan 31 '22 at 13:40

Reading encrypted private key in PKCS#8 format through bouncycastle, Java failing in docker container

2 Answers2