0

I've got an XML-over-HTTP service that has been working quite well for a long time, and I've been asked to provide some sample PHP code to connect to it. I've written the code and it connects, etc. just fine, except that it's waiting for the keep-alive timeout (5s) before completing.

When I connect using a Java-based client, I can see that Tomcat (through httpd) is using chunked encoding:

DEBUG: Request properties: DEBUG: Accept-Charset: UTF-8, ISO-8859-1, * DEBUG: Cache-Control: private, no-cache, no-store, no-transform DEBUG: Accept: application/xml, text/xml, text/plain DEBUG: User-Agent: My Awesome Java Client DEBUG: Pragma: no-cache DEBUG: Accept-Encoding: gzip, deflate DEBUG: Content-Type: application/xml DEBUG: Received first response packet after 219ms DEBUG: Dumping HTTP response headers DEBUG: Transfer-Encoding: chunked DEBUG: Keep-Alive: timeout=5, max=100 DEBUG: null: HTTP/1.1 200 OK DEBUG: Server: Apache/2.2.22 (Debian) DEBUG: Connection: Keep-Alive DEBUG: Date: Fri, 03 Feb 2017 15:33:13 GMT DEBUG: Content-Type: application/xml;charset=UTF-8 DEBUG: Read response in 7ms

When I use the PHP client, here's what I get in the response headers:

array(6) { [0]=> string(15) "HTTP/1.1 200 OK" [1]=> string(35) "Date: Fri, 03 Feb 2017 15:37:01 GMT" [2]=> string(30) "Server: Apache/2.2.22 (Debian)" [3]=> string(30) "Keep-Alive: timeout=5, max=100" [4]=> string(22) "Connection: Keep-Alive" [5]=> string(43) "Content-Type: application/xml;charset=UTF-8" }

Note that there isn't any Transfer-Encoding: chunked response header.

I can confirm that the PHP client is making an HTTP/1.1 request (and the response is using HTTP/1.1 as you can see) but chunked encoding seems not to be in use, here.

I believe I am sending the same request headers from the PHP client. Here's what the server is seeing:

2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] ========== Request from x.y.113.203 2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] host: example.com 2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] content-length: 135 2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] content-type: application/xml 2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] user-agent: My Awesome PHP Client 2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] accept: application/xml, text/xml, text/plain 2017-02-03 10:44:59,287 [catalina-exec-2] TRACE MyServlet- [1d] Accept-Charset: UTF-8, ISO-8859-1, * 2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d] Accept-Encoding: gzip, deflate 2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d] Cache-Control: private, no-cache, no-store, no-transform 2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d] pragma: no-cache 2017-02-03 10:44:59,288 [catalina-exec-2] TRACE MyServlet- [1d] connection: keep-alive

And the same from the Java client:

2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] ========== Request from x.y.113.203 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] user-agent: My Awesome Java Client 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] content-type: application/xml 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] accept: application/xml, text/xml, text/plain 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] Accept-Encoding: gzip, deflate 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] Accept-Charset: UTF-8, ISO-8859-1, * 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] Cache-Control: private, no-cache, no-store, no-transform 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] pragma: no-cache 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] host: example.com 2017-02-03 10:46:33,684 [catalina-exec-3] TRACE MyServlet- [1e] connection: keep-alive 2017-02-03 10:46:33,685 [catalina-exec-3] TRACE MyServlet- [1e] content-length: 135

I'm using the same "call" payload and the response should be identical for the two clients.

If I add Connection: close to the headers from the PHP client then the call completes immediately without the 5-second delay.

PHP is documented to support chunked encoding, but the server doesn't look like it's even trying to use chunked encoding, since the response headers don't say anything about it. The PHP code looks like this:

$options = array(
    'http' => array(
        'header' => array('Content-Type: application/xml',
                          'User-Agent: My Awesome PHP Client',
                          'Accept: application/xml, text/xml, text/plain',
                          'Accept-Charset: UTF-8, ISO-8859-1, *',
                          'Accept-Encoding: gzip, deflate',
                          'Cache-Control: private, no-cache, no-store, no-transform',
                          'Pragma: no-cache',

// NOTE: Connection: close makes multiple calls less efficient, but file_get_contents seems to stall otherwise
//                              'Connection: close'
'Connection: keep-alive',
        ),
        'follow_location' => 0, // Don't follow redirects
        'protocol_version' => '1.1',
        'ignore_errors' => TRUE, // Does not dump ugly errors to stderr
        'timeout' => 0.01, // TODO: adjustable
        'method'  => 'POST',
        'content' => $message,
    ),
    'ssl' => array(
        'disable_compression' => TRUE,
        'ciphers' => '!aNULL:!eNULL:!EXPORT:!DSS:!DES:!3DES:!SSLv2:!RC4:!MD5:ECDHE:ECDH',
    )
);

$context = stream_context_create($options);

$result = file_get_contents($url, false, $context);

var_dump($http_response_header);

Any ideas as to what might be happening? I believe if chunked-encoding is used, the client won't stall waiting for more information when the response is in fact complete.

Please note that the response must be either chunked or timed-out, because the dynamic and streaming nature of the response cannot have its content-length pre-computed and sent in an HTTP response header.

Christopher Schultz
  • 20,221
  • 9
  • 60
  • 77
  • IME usually the host web server decides about chunking or not based on whether the CGI sets a content-length, whether the client advertises support for HTTP/1.1 or not etc. If the server is emitting chunked responses without setting Transfer-Encoding: chunked, that would seem to be a bug in the server. The server also needs to know when the CGI has stopped generating output so it can emit the terminating chunk (0 length). Is this CGI or Fast-CGI? – Adrien Feb 07 '17 at 20:20
  • @Adrien The server is Java-based ("Tomcat through httpd") and we never set a `Content-Length` header as mentioned at the end of the question. The server is *not* using chunked-encoding. Rather, I *want* the server to use chunked-encoding, but it's not. – Christopher Schultz Feb 07 '17 at 23:17
  • This appears to be happening only with the PHP-based client. Using the exact same "call" (an identical XML document POSTed to the server with all the same request headers), the Java client ends up getting chunked-encoding and the PHP client does not. I can see no difference between what goes over the wire, so I'm at a loss to explain why the PHP client is causing the server to behave differently. – Christopher Schultz Feb 07 '17 at 23:18
  • The only requirement for a server to send a chunked response is that the client advertise http/1.1 as it's mandatory to implement. You might need to wireshark it to see what is going on. I guess it's possible php is messing up the response before you see it in the PHP script. – Adrien Feb 08 '17 at 10:19
  • Yeah, that's what I thought: `HTTP/1.1` by definition supports chunked encoding. I've been avoiding Wireshark because we are using ECDHE-based TLS even in our development environments. I'll have to temporarily disable that to get a good wire capture. – Christopher Schultz Feb 08 '17 at 15:12

0 Answers0