Enabling HTTP/2 makes the site much slower in nginx

Question

After enabling HTTP/2 to my website, I found that the performance dropped dramatically. The download speed becomes much slower, and large image requests block other API calls.

Here is an example webpage to illustrate this problem:

<img src="img.jpg">
<script>
for(var i=0;i<50;i++)
    setTimeout(()=>{
        fetch('foo.txt');
    },i*100);
</script>

(img.jpg is an image of ~700KB and foo.txt is a small text file. Everything is served directly from nginx.)

Here is the timeline graph when HTTP/2 is NOT enabled (listen 443 ssl):

... and when HTTP/2 is enabled (listen 443 ssl http2):

You can see that HTTP/2 is causing longer loading time of both img.jpg and foo.txt.

Here is the site config:

server {
    listen 443 ssl http2; # when performing http2 example
    # listen 443 ssl; # when performing http1 example
    server_name h2.test.**********;
    root /home/******/h2-test/;
}

I am using nginx 1.14.2 on Ubuntu 16.04.6 LTS. Do you have any suggestions for troubleshooting this problem?

Answer 1

Following is mostly a "theoretical" kind of answer:

I suppose what you observe is the double-fold issue: the nature of your test (requesting particularly small files), and, Chrome behavior/implementation of HTTP/2.

If you take a look at your HTTP/2 image, you can see that quite many (18) requests end at the same time. This may tell you that even though they don't start at the same time, they run in parallel.

By the nature of HTTP/2 multiplexing:

it’s even possible to intermingle parts of one message with another on the wire

So it appears as if this is what Chrome is doing - sees an unfinished request to server, and attempts to make another request over the same multiplexed connection, in parallel.

For some reason, your server has some sort of "scaling" issue with the parallel requests.

You said that foo.txt is a small file, but how exactly small it is?

There is http2_chunk_size directive (default value is 8k), which would mean that HTTP/2 response is spit in chunks of that size, and if the requests indeed run in parallel, and foo.txt is smaller than 8k, then it will result in "waiting" of one request for foo.txt to complete another request for foo.txt, until a total of 8k chunk is accumulated for output to browser.

I would suggest playing with that directive and lowering it to be smaller than requested file sizes, and/or repeating the tests with a text file that is larger than 8k.

Finally, if the connection is not reliable, you might be hitting the problem of TCP HOL blocking, which is worse on HTTP/2. To quote here:

But are there scenarios where it could worsen the situation? Yes. The reason is that applications using HTTP/2 can send many more requests over a single TCP connection due to the pipelining/multiplexing features. Unexpectedly large variations in latency or a loss affecting the segment at the TCP head of line makes HTTP/2 more likely to hit HOLB, and its impact is larger as well. Basically, the receiver stands idle till the head is recovered, while all the subsequent segments are held by the TCP stack. This means that an image might be downloaded successfully and still not show up due to HOLB.