1

Preamble

OpenJDK 11.0.2 2019-01-15

I am using mutual authentication when connecting to a Jetty (9.4.14.v20151114) server using TLSv1.2, with the cipher suite TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384.

My client uses standard Java TLS mechanisms:

javax.net.ssl.SSLContext sslContext = SSLContext.getInstance("TLS");
javax.net.ssl.KeyManager keyManager = ...;
javax.net.ssl.TrustManager trustManager = ...;
sslContext.init(new KeyManager[] { keyManager }, new TrustManager[] { trustManager }, null);
javax.net.ssl.SSLSocketFactory socketFactory = sslContext.getSocketFactory();
// ... and so on

My client uses a PKCS11 keystore backed by a smart card:

Provider pkcs11Provider = Security.getProvider("SunPKCS11").configure(configFile);
Security.addProvider(pkcs11Provider);
CallbackHandlerProtection callbackHandler = ...;
KeyStore keyStore = KeyStore.Builder.newInstance("PKCS11", provider, callbackHandler).getKeyStore();

The problem

Jetty sends a quite long list of supported signature algorithms for use in client authentication. The client decides which one of these to use by calling:

sun.security.ssl.SignatureScheme.getPreferableAlgorithm(List<SignatureScheme> schemes, PrivateKey signingKey, ProtocolVersion version);

This chooses the signature scheme RSASSA-PSS.

Later, in the method

java.security.Signature$Delegate.chooseProvider(int type, java.security.Key key, SecureRandom sr)

the client chooses a java.security.Provider$Service which supports the selected RSASSA-PSS algorithm. This provider for the RSASSA-PSS service is sun.security.rsa.SunRsaSign. The SunPKCS11 provider used in the keystore does not provide the RSASSA-PSS service.

The problem is that this SunRsaSign provider's Service does not support the PKCS11-based private key used for signing, and I get the following stack trace (copied by hand, beware typos):

java.security.InvalidKeyException: No installed provider supports this key: sun.security.pkcs11.P11Key$P11PrivateKey
    at java.base/java.security.Signature$Delegate.chooseProvider(Signature.java:1163)
    ...

The question

It seems the selected signing algorithm is implemented by a java.security.Provider that doesn't support the PrivateKey used in client authentication.

Is there any way to work around this? Can I make the client somehow select a client signature algorithm which works with the PKCS11 KeyStore (e.g. RSA_PKCS1_SHA512)? Can I configure the Jetty server to not suggest client signature algorithms which don't work?

Edit

Jetty SSLContextFactory configuration (may contain typos):

<New id="sslContextFactory" class="org.eclipse.jetty.util.ssl.SslContextFactory">
    <Set name="keyStorePath">keystore.jks</Set>
    <Set name="keyStorePassword">...</Set>
    <Set name="trustStorePath">truststore.jks</Set>
    <Set name="needClientAuth">true</Set>
    <Set name="includeProtocols">
        <Array type="java.lang.String">
            <Item>TLSv1.2</Item>
        </Array>
    </Set>
    <Set name="includeCipherSuites">
        <Array type="java.lang.String">
            <Item>TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384</Item>
        </Array>
    </Set>
</New>
Atuos
  • 501
  • 3
  • 12
  • What TLS versions do you have configured on the client and the server? What is your SslContextFactory configuration on the server? Have you dumped your server state after startup to see how it is configured? – Joakim Erdfelt May 08 '19 at 09:23
  • @JoakimErdfelt TLSv1.2. I added the SslContextFactory configuration. I don't know what to look for in the server to determine how client signature algorithms are suggested. – Atuos May 08 '19 at 10:26
  • I reported this as a bug: https://bugs.openjdk.java.net/browse/JDK-8223940 – Atuos May 21 '19 at 07:00

0 Answers0