6

My java 8 application is communicating with other system via rest, secured with TLS1.2. Last 2 java patches (261, 271) has broken the connection, because they have added some backward compatibility with TLS1.3. During the handshake, it started using some newer signature scheme - rsa_pss_rsae_sha256 instead of, previously workingrsa_pkcs1_sha256 (named SHA256withRSA in java8u251), which is not working because it's trying to reach my private key (during CertificateVerify handhake step), which is protected by HSM, thus it's not available to read it.

I would like to disable this new signature scheme, because the older one is still sufficient and it worked on previous java patch and it's also used in a few other connections my application.

I have found this solution - https://bugs.openjdk.java.net/browse/JDK-8227445 but when I set this setting by direct signature scheme name rsa_pss_rsae_sha256, it didn't work. Do you know what name should I pass there to disable this specific signature scheme (or all rsa_pss_* signature schemes group)?

Marcin Mikołajczyk
  • 731
  • 6
  • 19
  • Which http client are you using? Can you post an example of your client ssl configuration? – Hakan54 Nov 30 '20 at 07:37
  • ```CloseableHttpClient``` with ```SSLConnectionSocketFactory``` (supported protocols - "TLSv1.2", supported cipher suites ```TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 ``` and a ```DefaultHostnameVerifier``` – Marcin Mikołajczyk Nov 30 '20 at 08:14
  • I was able to disable this scheme (not returning rsa_pss* algorithms in ClientHello anymore), but it still tries to use it, because while consuming CertificateRequest, the server side sends ```"supported signature algorithms": [ecdsa_secp256r1_sha256, rsa_pss_rsae_sha256, rsa_pkcs1_sha256,...``` and it's trying to connect using those algorithms in the order the server has returned. – Marcin Mikołajczyk Nov 30 '20 at 14:49
  • I would try to use `java.security` file, which will configure TLS for whole JVM. On JDK 8 and earlier, edit the `/lib/security/java.security`. All Java TLS changes are in the roadmap - https://java.com/en/jre-jdk-cryptoroadmap.html. You can find in the release notes/additional information what was changed. I'm not sure if you can override default cipher order in the java somehow - it needs some debugging/testing. – Jan Garaj Jan 22 '21 at 19:50
  • I have modified java.security file but with no luck – Marcin Mikołajczyk Jan 25 '21 at 12:39
  • The RSA PSS signature scheme differs from RSA PKCS1 only in the way the message is hashed and padded. Your HSM certainly knows how to encrypt a byte array with your private key, So maybe the solution would be to use a `SSLContext` (or other interface) implementation that supports RSA PSS? – Piotr P. Karwasz Jan 25 '21 at 12:45
  • @MarcinMikołajczyk did you ever get a solution to this problem, I too am hitting this issue now. – Paul Whelan Sep 10 '21 at 16:25
  • 1
    @Paul, I have reported this issue to Oracle team and they have fixed the jdk.certpath.disabledAlgorithms property in .291 patch. I have also got an update from nCipher software (hsm) to handle the RSA PSS algorithm without having to update java. – Marcin Mikołajczyk Sep 12 '21 at 16:48
  • Thanks @MarcinMikołajczyk I appreciate the reply. I'll look into the .291 patch that could get me out of a hole in relation to this issue. – Paul Whelan Sep 13 '21 at 08:22

0 Answers0