Java 8: How to perform timeout for SSLSocket.startHandshake()

Question

We have below thread to perform SSLHandshake but in some edge cases I noticed ((SSLSocket) clientSocket).startHandshake(); is blocked forever and its not going to the next block of while loop code where SSL_HANDSHAKE_TIMEOUT is 1500 milli seconds and it works fine, Am wondering if adding clientSocket.setSoTimeout(90000); would fix this issue or should it handled in a different way?

MainServerHandshakeThread

public class MainServerHandshakeThread implements com.ssltunnel.utilities.threading.Shutdown, Runnable {
    private final Socket clientSocket;
    private static final org.slf4j.Logger LOG = LoggerFactory.getLogger(MainServerHandshakeThread.class.getName());
    private boolean done;

    public MainServerHandshakeThread(Socket clientSocket) {
        this.clientSocket = clientSocket;        
    }

    private void handshake() throws CertificateExpiredException, InterruptedException, IOException {

        long start = System.currentTimeMillis();

        ((SSLSocket) clientSocket).setNeedClientAuth(true);
        MainServerHandshakeHandler handshake = new MainServerHandshakeHandler();
        ((SSLSocket) clientSocket).addHandshakeCompletedListener(handshake);
        ((SSLSocket) clientSocket).startHandshake();


        while (!handshake.isDone() && !done) {
            Thread.sleep(10);
            long duration = System.currentTimeMillis() - start;
            if (duration>SSL_HANDSHAKE_TIMEOUT) {
                done = true;
                LOG.warn("Handshake timeout");
            }
        }
        long stop = System.currentTimeMillis();        
        serialNumber = handshake.getSerialNumber();
        LOG.info("MainServer Handshake Handshake done in ms: " + ((stop - start))+" For serialNumber "+serialNumber );        

    }

    @Override
    public void run() {
        try {
            handshake();
        } catch (CertificateExpiredException ex) {
            LOG.error("Client Certificate Expired", ex.getMessage());
            SocketUtils.closeQuietly(clientSocket);
        }
        catch (InterruptedException ex) {
            LOG.error("Interrupted waiting for handshake", ex);
            SocketUtils.closeQuietly(clientSocket);
        } 
        catch (IOException ex) {
            LOG.error("IO Error waiting for handshake", ex);
            SocketUtils.closeQuietly(clientSocket);
        }
        finally {
            LOG.debug("Handshake thread is done");
            done = true;
        }

    }

    @Override
    public void shutdown() {
        if (clientSocket!=null) {
            SocketUtils.closeQuietly(clientSocket);
        }
    }
}

According to [the documentation for `SSLSocket.startHandshake()`](https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocket.html#startHandshake--): _This method is synchronous for the initial handshake on a connection and returns when the negotiated handshake is complete._ — Sean Bright, Oct 09 '19 at 15:58
It would be better not to perform the initial handshake explicitly at all, let it be done automatically, and set a read timeout on the socket. NB 'Non-blocking timeout' is a contradiction in terms. — user207421, Oct 09 '19 at 19:33
Thanks for the response, how can I do the handshake automatically? can you please post it as answer? — OTUser, Oct 09 '19 at 19:35
You don't 'do the handshake automatically'. You 'let it be done automatically', as I said. That's what 'automatic' means. — user207421, Oct 09 '19 at 21:08
@user207421 Am quite new to socket world and I don't really understand what you meant by 'let it be done automatically', I really appreciate if you can post it as answer — OTUser, Oct 11 '19 at 16:02
@RanPaul: he means "just don't call `startHandshake()` at all". It will be *called automatically* by the JVM when you first try to *read* from or *write* to the socket. That plus a timeout (as you suggested) and it should be fine. — Matthieu, Oct 12 '19 at 07:33
Btw, did you try what you suggested? Setting `setSoTimeout()`? — Matthieu, Oct 12 '19 at 07:36
You might want to check [that other question](https://stackoverflow.com/questions/57190490/is-it-possible-to-create-multiple-ssl-serversocket-on-the-same-port), especially the comments might give you some insights. — Matthieu, Oct 12 '19 at 07:45

Matthieu · Answer 1 · 2019-10-12T16:01:28.553

To summarize the comments (mostly from @user207421): yes, setting a socket timeout through socket.setSoTimeout(timeout) will be enough to trigger a SocketTimeoutException (a subclass of IOException) if the handshake process doesn't complete after "some time" (but not necessarily the specified timeout¹).

That is simply explained because setSoTimeout() works at the socket level, below the SSL handshake: the handshake protocol performed by startHandshake() involves several reads/writes from the socket Input/OutputStream, which will trigger the timeout itself. In other words: it is not a "handshake timeout" per se, but the "read timeout" on all read operations performed by the handshake itself.

Also, note that you don't need to call startHandshake() yourself: the JVM will do that automatically when you first try to read or write from your SSLSocket (which is something you usually do early anyway after you get such a socket from an SSLServerSocket anyway).

¹: The timeout specified by setSoTimeout(timeout) is for a single read(). So the handshake process can timeout (in the worst case) after the number of read() it performs, times the timeout value you specified.

Thanks for the response, since we cache the socket connections at server side we can not defer the `handshake` and rely on JVM to do the handshake automatically, We have to performa the handshake manually — OTUser, Oct 14 '19 at 20:22
@RanPaul no problem, it's a facility to ease genericity: when you get a `Socket`, you might in fact get an `SSLSocket` (which is a subclass), so you don't need to worry about SSL specifics when reading/writing. There's nothing wrong with calling `startHandshake()` manually, it's just not mandatory. You can still call `setSoTimeout()` and get the handshake process to timeout. — Matthieu, Oct 14 '19 at 21:56

Ariel Carrera · Answer 2 · 2019-10-12T14:21:44.637

Most implementations of SSLSocket support setHandshakeTimeout() but it is not part of the class definition. I suggest you to look for example at Android code. This is a good example of how to deal with it: ssl socket factory wrapper and ssl socket factory

Check how they try to set a handshake timeout if exist:

    private void setHandshakeTimeout(SSLSocket sslSocket, int timeout) {

    try {

        // Most implementations of SSLSocket support setHandshakeTimeout(), but it is not

        // actually part of the class definition. We will attempt to set it using reflection.

        // If the particular implementation of SSLSocket we are using does not support this

        // function, then we will just have to use the default handshake timeout.

        sslSocket.getClass().getMethod("setHandshakeTimeout", int.class).invoke(sslSocket,

                timeout);

    } catch (Exception e) {

        LogUtils.w(LogUtils.TAG, e, "unable to set handshake timeout");

    }

}

And look at how they get a connection:

    SSLSocket sslsock = (SSLSocket)

        ((sock != null) ? sock : createSocket());



    if ((localAddress != null) || (localPort > 0)) {



        // we need to bind explicitly

        if (localPort < 0)

            localPort = 0; // indicates "any"



        InetSocketAddress isa =

            new InetSocketAddress(localAddress, localPort);

        sslsock.bind(isa);

    }



    int connTimeout = HttpConnectionParams.getConnectionTimeout(params);

    int soTimeout = HttpConnectionParams.getSoTimeout(params);



    InetSocketAddress remoteAddress;

    if (nameResolver != null) {

        remoteAddress = new InetSocketAddress(nameResolver.resolve(host), port);

    } else {

        remoteAddress = new InetSocketAddress(host, port);

    }



    sslsock.connect(remoteAddress, connTimeout);



    sslsock.setSoTimeout(soTimeout);

    try {

        hostnameVerifier.verify(host, sslsock);

        // verifyHostName() didn't blowup - good!

    } catch (IOException iox) {

        // close the socket before re-throwing the exception

        try { sslsock.close(); } catch (Exception x) { /*ignore*/ }

        throw iox;

    }



    return sslsock;

Hope it helps.

Java 8: How to perform timeout for SSLSocket.startHandshake()

2 Answers2