3

We have an Apache Tomcat-based Java web application running on JDK 1.8 and recently had to strenghten its security-related features to comply with certain regulations, thus we are now using BouncyCastleFipsProvider as our JVM's main security provider, configured in java.security as suggested in the BC documentation:

security.provider.1=org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider
security.provider.2=com.sun.net.ssl.internal.ssl.Provider BCFIPS
security.provider.3=sun.security.provider.Sun

I'm facing an issue now calling one of our third-party services through HTTPS, because its certificate chain cannot be validated. The exception we got is:

Caused by: sun.security.validator.ValidatorException: End user tried to act as a CA
        at sun.security.validator.SimpleValidator.checkBasicConstraints(SimpleValidator.java:320)
        at sun.security.validator.SimpleValidator.checkExtensions(SimpleValidator.java:237)
        at sun.security.validator.SimpleValidator.engineValidate(SimpleValidator.java:221)
        at sun.security.validator.Validator.validate(Validator.java:262)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)

This HTTPS call used to work fine, so I've debugged the SSL handshake and noticed that in the previous (working) version (default JVM security settings with Sun's security provider) the validation is done with another class: sun.security.validator.PKIXValidator. In the current (not working) setup, the SSLContext is provided by the 2nd provider in the above list, which is "Sun JSSE provider (FIPS mode, crypto provider BCFIPS)"

I'm not that experienced with JCE, JSSE, Bouncy Castle in depth, so I wonder if anyone knows if this PKIXValidator can be used with com.sun.net.ssl.internal.ssl.Provider(“BCFIPS”) to validate server certificates? If so, how do I enable that?

UPDATE: We've also set the below parameters in our java.security file, the reason I found for that in one of our documents is: "PKIX is not available as a trust manager algorithm." I believe this is something I should double-check.

ssl.KeyManagerFactory.algorithm=SunX509
ssl.TrustManagerFactory.algorithm=SunX509

UPDATE2: I've tried setting the above algorithm properties to PKIX, and that did the trick, the HTTPS connection can be established now.

Note: the certificate of this third party is valid, its chain is trusted by browsers and is issued by a well-known root CA. The issue is that they send their own cert twice in the chain, which makes the SimpleValidator fail.

Bonus question: What is the difference between the first two providers in the above list?

Thanks in advance.

skarfiol
  • 131
  • 10
  • 1
    The cert validator for SSL/TLS is controlled not by the SSLContext itself, but by the TrustManager (usally X509TrustManagerImpl) linked from the SSLContext. Look at (maybe debug) your code if any that sets the trustmanager in the context; if not set it should default to using PKIX (instead of Simple) but verify security property (not system property) `java.security.ssl.TrustManagerFactory.algorithm` – dave_thompson_085 Feb 21 '19 at 07:35
  • Thanks for your comment Dave. I've checked it and it seems we explicitly set SunX509 there, I've updated the OP with that. – skarfiol Feb 21 '19 at 10:25

1 Answers1

0

Setting these parameters in java.security solved the issue:

ssl.KeyManagerFactory.algorithm=PKIX
ssl.TrustManagerFactory.algorithm=PKIX

See also: https://bugs.openjdk.java.net/browse/JDK-8169745

skarfiol
  • 131
  • 10