Java https having problems with handshaking

Question

I am in the process of writing an https server in java that will accept and respond to ajax requests. I have this all working with http connections, but https is proving difficult to set up. If I use openssl I can hit the server and get a response as expected: openssl s_client -connect localhost:5001 But ajax calls from the browser fail. I'm not sure where to go from here.

This is the stacktrace from the server after an attemped ajax call:

javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at sun.security.ssl.SSLSocketImpl.writeRecord(Unknown Source)
at sun.security.ssl.AppOutputStream.write(Unknown Source)
at sun.nio.cs.StreamEncoder.writeBytes(Unknown Source)
at sun.nio.cs.StreamEncoder.implFlushBuffer(Unknown Source)
at sun.nio.cs.StreamEncoder.implFlush(Unknown Source)
at sun.nio.cs.StreamEncoder.flush(Unknown Source)
at java.io.OutputStreamWriter.flush(Unknown Source)
at java.io.BufferedWriter.flush(Unknown Source)
at com.myDomain.HttpsServer.main(HttpsServer.java:223)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(Unknown Source)
... 11 more

The ajax call:

var command, req;
  command = {
    command: "getStatus",
  };
  command = JSON.stringify(command);
  req = $.ajax("https://localhost:5001", {
    data: command,
    dataType: "jsonp",
    timeout: 1000
  });

The Java Server:

private static SSLServerSocket createSSLSocket(){
    SSLServerSocketFactory sslServerSocketFactory =
        (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
    try{
      SSLServerSocket sslSocket =
        (SSLServerSocket) sslServerSocketFactory.createServerSocket(port, 10, InetAddress.getByName("127.0.0.1"));
      return sslSocket;
    } catch (Exception e){
        e.printStackTrace();
    }
    return null;
}

public static void main (String args[]) throws Exception {
    SSLServerSocket sslSocket;
    sslSocket = createSSLSocket();

    while(running) {
        SSLSocket connected = (SSLSocket) sslSocket.accept();
        try{
         BufferedWriter w = new BufferedWriter(
            new outputStreamWriter(connected.getOutputStream()));
             BufferedReader r = new BufferedReader(
                new InputStreamReader(connected.getInputStream()));
             w.write("HTTP/1.0 200 OK");
             w.write("foo");
             w.newLine();
             w.flush();  //THIS IS WHERE THE ACTUAL EXCEPTION IS THROWN (LINE 223)
             w.close();
             r.close();
             connected.close();
        } catch (Exception e){
         e.printStackTrace();
        }
    }
}

This is being run with:

java -Djavax.net.ssl.keyStore=mySrvKeystore -Djavax.net.ssl.keyStorePassword=123456 myApp

And it looks like mySrvKeystore is being used properly when that is done with the debug option.

UPDATE

Here is some more information from debug output:

*** ECDH ServerKeyExchange
 Server key: Sun EC public key, 256 bits
   public x coord:      59120686551233854673577061225846672012454441193286172303206804252170042475984
   public y coord: 64356797475544123011351526783519675095229374542555548418334080869325161950574
   parameters: secp256r1 [NIST P-256, X9.62 prime256v1] (1.2.840.10045.3.1.7)
 *** ServerHelloDone
 main, WRITE: TLSv1 Handshake, length = 1317
 main, received EOFException: error
 main, handling exception: javax.net.ssl.SSLHandshakeException: Remote host closed connection      during handshake
 %% Invalidated:  [Session-1, TLS_ECDHE_RSA_WITH_RC4_128_SHA]
 main, SEND TLSv1 ALERT:  fatal, description = handshake_failure
 main, WRITE: TLSv1 Alert, length = 2
 main, called closeSocket()

Does it fail in all browsers? There is a question with the same stacktrace: http://stackoverflow.com/questions/4827190/exception-at-start-of-request-clientauth-ssl — Serxipc, May 25 '12 at 10:14
Looks like it is a problem specific to chrome. check this guy out: http://stackoverflow.com/questions/7535154/chrome-closing-connection-on-handshake-with-java-ssl-server — Lee Quarella, Jun 26 '12 at 13:07

score 0 · Answer 1 · answered Jun 03 '12 at 20:53

It can be tricky determining a handshake failure like this. Try turning on the SSL/TLS debugging using:

-Djavax.net.debug=true

You may also want to add a trust store on the server:

-Djavax.net.ssl.trustStore=mySrvKeystore

The ciphersuite TLS_ECDHE_RSA_WITH_RC4_128_SHA may not be supported by the browser you are using. It could be the client/server attempted to negotiate several and failed on this last one. You can print out your enabled cipher suites on the server using:

String[] cipherSuites = sslSocket.getEnabledCipherSuites();
for (int x = 0; x < cipherSuites.length; x++) {
   System.out.println(cipherSuites[x]);
}

You can also consider limiting the ciphersuites the server offers using sslSocket.setEnabledCipherSuites().

See Using AES in JSSE for some old examples from Sun.

Generally a modern browser can connect with a ciphersuite like TLS_RSA_WITH_AES_128_CBC_SHA

Java https having problems with handshaking

1 Answers1