How is OkHttp performing parallell HTTP Requests with seemingly synchronous HTTP connections, without the use of threading?

Question

I've done some performance testing with the OkHttp library and found it to be great. It did 80 requests to http://httpbin.org/delay/1, which deliberately pauses 1 sec for each request, in 4.7s on my HTC One phone. I've looked at the code and I'm trying to find out why it is so fast. The developers (Square Inc) advertise Connection Pooling and Async calls, both which I believe contribute to the good performance.

I come from the .NET world and in .NET 4.5 you have a true async HTTP library with an Async GetResponse-method. By yielding the thread to the OS while waiting for the response, you free up resources to initiate more HTTP requests, or other stuff. The thing is I cannot see the same pattern with OkHttp (or any other HTTP library for Android I've looked into). So how can I still execute 80 1-second-request in 4 seconds? It's not thread-based, right? I'm not firing up 80 (or 20) threads?

To be specific, in com.squareup.okhttp.Call.beginRequest(), I see no yielding of the thread between the sendRequest and getResponse calls:

if (canceled) return null;

try {
    engine.sendRequest();

    if (request.body() != null) {
        BufferedSink sink = engine.getBufferedRequestBody();
        request.body().writeTo(sink);
    }

    engine.readResponse();
} catch (IOException e) {
    HttpEngine retryEngine = engine.recover(e, null);
    if (retryEngine != null) {
        engine = retryEngine;
        continue;
    }

    // Give up; recovery is not possible.
    throw e;
}

Response response = engine.getResponse();

So again, how is making 80 "parallell" calls possible?

I don't need to know this in order to use the library, but async programming interests me and I'd really like to understand how OkHttp/SquareInc has solved this.

Answer 1

I did some testing myself by linking the OkHttp source into my project and injecting logging into the core request class - Call.java. What I found was that OkHttp indeed uses a thread for each call and does not yield while waiting for the response as I wrongly assumed. The only reason it is faster than for instance Volley, is that Volley has hard-coded a thread limit of 4, while OkHttp uses Integer.MAX_VALUE (Dipatcher.java line 58):

executorService = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60, TimeUnit.SECONDS,
      new LinkedBlockingQueue<Runnable>(), Util.threadFactory("OkHttp Dispatcher", false));

Here is an exerpt of the LogCat log when I queued and excecuted 80 requests "asynchronously":

05-31 12:15:23.884  27989-28025/nilzor.okhttp I/OKH﹕ Starting request 1
05-31 12:15:23.884  27989-28026/nilzor.okhttp I/OKH﹕ Starting request 2
05-31 12:15:24.044  27989-28199/nilzor.okhttp I/OKH﹕ Starting request 79
05-31 12:15:24.054  27989-28202/nilzor.okhttp I/OKH﹕ Starting request 80
05-31 12:15:25.324  27989-28025/nilzor.okhttp I/OKH﹕ Getting response 1 after 1436ms
05-31 12:15:26.374  27989-28026/nilzor.okhttp I/OKH﹕ Getting response 2 after 2451ms
05-31 12:15:27.334  27989-28199/nilzor.okhttp I/OKH﹕ Getting response 79 after 3289ms
05-31 12:15:26.354  27989-28202/nilzor.okhttp I/OKH﹕ Getting response 80 after 2305ms

The third column on format xxxxx-yyyyy indicates process id (x) and thread id (y). Notice how each requests gets its own thread and how the same thread handles the response. Full log. This means that we have 80 blocking threads while waiting for responses, which is not how true async programming should be done.

In OkHttp / Square Inc's defense, they never claim to have true end-to-end async HTTP communication, they only provide an async interface to the consumer. Which is fine. And it also performs well and does a bunch of other stuff. It is a good library, but I misinterpreted it for having true async HTTP communication.

I have since understood to look for the keywords "NIO" to find what I'm looking for. Libraries like AndroidAsync and Ion seems promising.

Answer 2

OkHttp doesn't use async sockets at this time. If you're using the async API with enqueue(), the Dispatcher will spin up multiple threads and make multiple concurrent requests. If you use the same OkHttp client for all requests it will limit itself to 5 connections per host.