4

Android uses Apache's HTTP Components library to perform HTTP requests and exposes an API that doesn't support asynchronous requests or pipelining. We're writing an app that would benefit from pipelining so we are using Hotpotato to perform those requests. In an effort to reduce the size of the APK (Hotpotato and Netty add ~2-4MB to the APK size) we're looking to implement our own on top of HttpCore and HttpNIO.

The Apache NIO extensions docs have an obscure reference to pipelining, mentioning that "non-blocking HTTP connections [are] fully pipelining capable", and there's a bug on the HttpClient code that mentions pipelining support, but there's no mention of how to implement it.

How do I use Apache's HTTP Components to implement support for HTTP pipelining and persistent connections on top of Android's existing Apache HTTP components libraries?

Colin 't Hart
  • 7,372
  • 3
  • 28
  • 51
magneticMonster
  • 2,373
  • 6
  • 30
  • 46

1 Answers1

3

Most likely you are not going to like the answer, but so be it. The reason why support for HTTP pipelining is lacking is that HTTP pipelining is simply not useful outside a very limited number of use cases. HTTP pipelining is applicable (or recommended by the HTTP spec) for non-idempotent HTTP methods only. This effectively precludes pipelining of POST requests. Pipelining can be marginally useful for browsers that need to retrieve a large sets of static files using GET requests while being restricted to only two simultaneous HTTP connections to the same host. In this case HTTP pipelining may produce marginal performance improvements. At the same time I contend that an HTTP agent using a moderately sized pool of persistent connections (no more than five) will outperform pipelining HTTP agent. The extra complexity of HTTP pipelining is simply not worth the trouble and this is the reason why there is no great urgency to add out-of-box support for HTTP pipelining to HttpClient and HttpCore.

Having said all that non-blocking HTTP connections of HttpCore NIO are fully asynchronous and always function in a full duplex mode. HttpCore imposes no restriction as to how many requests can be written out or how many responses can be received in one go. It is the responsibility of the protocol handler to correlate HTTP requests and responses into logically related sequences of message exchanges. Standard HTTP protocol handlers do not pipeline HTTP messages in order to be able to support the expect-continue handshaking for POST requests (expectation verification and pipelining are pretty much mutually exclusive). However, there is nothing that prevents you from building a custom NHttpClientHandler class and making it pipeline requests. You can start off by taking the source code of HttpAsyncClientProtocolHandler [1], rip out the expect-continue handshaking code and add queuing of incoming and outgoing HTTP messages.

Hope this helps

[1] http://svn.apache.org/repos/asf/httpcomponents/httpcore/trunk/httpcore-nio/src/main/java/org/apache/http/nio/protocol/HttpAsyncClientProtocolHandler.java

ok2c
  • 26,450
  • 5
  • 63
  • 71
  • Thanks for the response, but my use case would in fact benefit from pipelining (hundreds of small request/responses over a high-latency mobile phone connection). I'll take a look at the protocol handler example you gave but it's not a whole lot to go off of... – magneticMonster Sep 26 '11 at 14:40