Handling multiple certificates in Netty's SSL Handler used in Play Framework 1.2.7

Question

I have a Java Key Store where I store certificates for each of my customer's sub-domain. I am planning to use the server alias to differentiate between multiple customers in the key store as suggested here. Play framework 1.2.7 uses Netty's SslHandler to support SSL on the server-side. I tried implementing a custom SslHttpServerContextFactory that uses this solution.

import play.Play;

import javax.net.ssl.*;
import java.io.FileInputStream;
import java.net.InetAddress;
import java.net.Socket;
import java.security.KeyStore;
import java.security.Principal;
import java.security.PrivateKey;
import java.security.Security;
import java.security.cert.X509Certificate;
import java.util.Properties;

public class CustomSslHttpServerContextFactory {

  private static final String PROTOCOL = "SSL";
  private static final SSLContext SERVER_CONTEXT;

  static {

    String algorithm = Security.getProperty("ssl.KeyManagerFactory.algorithm");
    if (algorithm == null) {
      algorithm = "SunX509";
    }

    SSLContext serverContext = null;
    KeyStore ks = null;
    try {
      final Properties p = Play.configuration;

      // Try to load it from the keystore
      ks = KeyStore.getInstance(p.getProperty("keystore.algorithm", "JKS"));
      // Load the file from the conf
      char[] certificatePassword = p.getProperty("keystore.password", "secret").toCharArray();
      ks.load(new FileInputStream(Play.getFile(p.getProperty("keystore.file", "conf/certificate.jks"))),
          certificatePassword);

      // Set up key manager factory to use our key store
      KeyManagerFactory kmf = KeyManagerFactory.getInstance(algorithm);
      kmf.init(ks, certificatePassword);
      TrustManagerFactory tmf = TrustManagerFactory.getInstance(algorithm);
      tmf.init(ks);

      final X509KeyManager origKm = (X509KeyManager) kmf.getKeyManagers()[0];
      X509KeyManager km = new X509KeyManagerWrapper(origKm);

      // Initialize the SSLContext to work with our key managers.
      serverContext = SSLContext.getInstance(PROTOCOL);
      serverContext.init(new KeyManager[]{km}, tmf.getTrustManagers(), null);
    } catch (Exception e) {
      throw new Error("Failed to initialize the server-side SSLContext", e);
    }

    SERVER_CONTEXT = serverContext;
  }

  public static SSLContext getServerContext() {
    return SERVER_CONTEXT;
  }

  public static class X509KeyManagerWrapper implements X509KeyManager {
    final X509KeyManager origKm;

    public X509KeyManagerWrapper(X509KeyManager origKm) {
      this.origKm = origKm;
    }

    public String chooseServerAlias(String keyType,
                                    Principal[] issuers, Socket socket) {
      InetAddress remoteAddress = socket.getInetAddress();
      //TODO: Implement alias selection based on remoteAddress

      return origKm.chooseServerAlias(keyType, issuers, socket);
    }

    @Override
    public String chooseClientAlias(String[] keyType,
                                    Principal[] issuers, Socket socket) {
      return origKm.chooseClientAlias(keyType, issuers, socket);
    }

    @Override
    public String[] getClientAliases(String s, Principal[] principals) {
      return origKm.getClientAliases(s, principals);
    }

    @Override
    public String[] getServerAliases(String s, Principal[] principals) {
      return origKm.getServerAliases(s, principals);
    }

    @Override
    public X509Certificate[] getCertificateChain(String s) {
      return origKm.getCertificateChain(s);
    }

    @Override
    public PrivateKey getPrivateKey(String s) {
      return origKm.getPrivateKey(s);
    }

  }

}

But, this approach did not work for some reason. I get this message in my SSL debug log.

X509KeyManager passed to SSLContext.init():  need an X509ExtendedKeyManager for SSLEngine use

This is the SSL trace, which fails with "no cipher suites in common". Now, I switched the wrapper to:

public static class X509KeyManagerWrapper extends X509ExtendedKeyManager

With this change, I got rid of the warning, but I still see the same error as before "no cipher suites in common" and here is the SSL trace. I am not sure why the delegation of key manager won't work.

Some more information that may be useful in this context.

• Netty uses javax.net.ssl.SSLEngine to support SSL in NIO server.
• As per the recommendation in this bug report, it is intentional that X509ExtendedKeyManager must be used with an SSLEngine. So, the wrapper must extend X509ExtendedKeyManager.

This is hindering me to move further with the custom alias selection logic in X509KeyManagerWrapper. Any clues on what might be happening here? Is there any other way to implement this in Netty/Play? Appreciate any suggestions.

Answer 1

SSLEngine uses the chooseEngineServerAlias method to pick the certificate to use (in server mode) - not the chooseServerAlias method.

The default chooseEngineServerAlias implementation actually returns null, which is what causes the "no cipher suites in common" message - you need a certificate to know which cipher suites can be used (e.g. ECDSA can only be used for authentication if the certificate has an ECC public key, etc.) There are actually some cipher suites which can be used without a certificate, however, these are typically disabled as they are vulnerable to MITM attacks.

Therefore, you should also override chooseEngineServerAlias, and implement your logic to select the certificate based on the IP address there. As Netty only uses SSLEngine, what chooseServerAlias does doesn't matter - it'll never be called.

Java 8 also has support for server-side SNI, which allows you to use several certificates across many hostnames with a single IP address. Most web browsers support SNI - the notable exceptions are IE running on Windows XP and some old versions of Android, however, usage of these is declining. I have created a small example application demonstrating how to use SNI in Netty on GitHub. The core part of how it works is by overriding chooseEngineServerAlias - which should give you enough hints, even if you want to use the one certificate per IP address technique instead of SNI.

(I posted a similar answer to this on the Netty mailing list, where you also asked this question - however, my post seems to have not yet been approved, so I thought I'd answer here too so you can get an answer sooner.)