2

I am trying to see the performance improvement of HTTP/2 compared to HTTP/1.1 from header compression. I have simulated a network of 300ms latency and 50 KB/s bandwidth (I tried few other combinations as well).

I did multiple attempts of tests which sends different number of requests to sever varying from 1 to 100.

In each test, I load my page multiple times and measure time between request sent and first byte of response received (I use Navigation Timing API for this). There is a reduction of this time between the very first request and consequent requests. However similar reduction is seen with HTTP/1.1 as well. So there is no visible gain compared to HTTP/1.1. There are considerable improvements to total page load time, but I have no way to say that it is from header compression or multiplexing. So I think measuring time between request sent and first byte of response received should give an accurate measure. But I am unable to experience that.

Find my sample test results

enter image description here

What should be the correct way to measure performance improvement from header compression ?

Thanks.

Lakmali Erandi Baminiwatta
  • 2,295
  • 15
  • 18
  • Do you have the wen site available on the internet? First of all you should test TLS problems. On online web site I'd recommend to make test on https://www.ssllabs.com/ssltest/. Then you can use http://www.webpagetest.org/ to test the site without HTTP/2 and then with HTTP/2. – Oleg Mar 27 '16 at 15:20
  • Hi Oleg, I am using my own website deployed on Jetty. Can you clarify more about what is meant by TLS problem here? I am comparing performance of HTTP/2 with HTTPS. Hence I assume it will be beneficial for HTTP/2 over HTTPS due to less TLS overhead in HTTP/2. – Lakmali Erandi Baminiwatta Mar 29 '16 at 11:06
  • **Could you post the URL to your web site?** HTTP/2 works mostly only over HTTPS (`h2` protocol). The performance problems which you described could come from TLS. It's difficult to speak about the theory of many possible problems. It's better if you just post the example of the URL which have the described problems. The problem with "the first byte of response received" looks for me really like the TLS problem. – Oleg Mar 29 '16 at 11:24
  • Thanks for the info Oleg. My site also works on HTTPS (ex:https://:8443/app). My site is an local deployment and is not accessible from outside. I have updated my question with some sample test results which I ran today under different network conditions. As per my knowledge impact from header compression start from the 2nd request onward due to redundant headers. So as per the results i gained, it shows only around 3-4% improvement from header compression which doesn't look consistent either. – Lakmali Erandi Baminiwatta Mar 29 '16 at 17:13
  • You are welcome! You post too few details about your server. I don't know till now even which operation system and which web server in which version you use. Look at [the article](https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-to-first-byte/), which describes detailed an example of tuning of the problem of "Time To First Byte" on nginx. Look at [here](https://youtu.be/0EB7zh_7UE4). In general TCP tuning, OCSP stamping (if OCSP are used) and other TLS performance tuning (see [here](https://istlsfastyet.com/)) should solve your problem. – Oleg Mar 29 '16 at 17:28
  • Thanks again. I will go through the links. My OS is Ubuntu (14.04), and I use Jetty (9.3) as the web server. I have deployed a simple service which loads set of images. My client in google chrome (49). – Lakmali Erandi Baminiwatta Mar 29 '16 at 17:53
  • I didn't worked with Jetty. I recommend you to try to ask http://stackoverflow.com/users/1215076/sbordet to help you. He is Jetty developer and he posted some answers about HTTP/2 too. Online tools like https://www.ssllabs.com/ssltest/, https://www.htbridge.com/ssl/ and http://www.webpagetest.org/ could available **online** could essential simplify the troubleshooting of your problem. Without the tools you will have to verify a lot of things manually. I can't help you offline especially if I have no practical experience with Jetty. I still suppose, your main problem is TCP/TLs configuration. – Oleg Mar 29 '16 at 19:41
  • I would strictly recommend you to prepare the web server with an empty HTML page **with the same configuration of Ubuntu (14.04), and I use Jetty (9.3)**, which will be available online and to test the server with respect of https://www.ssllabs.com/ssltest/, https://www.htbridge.com/ssl/ and http://www.webpagetest.org/. It will save a lot of your time and point you to the problems in TLS configuration or your server. You can get free SSL/TLS certificate from https://letsencrypt.org/ or from https://startssl.com/. – Oleg Mar 30 '16 at 11:38

2 Answers2

2

If you want to know the performance improvements from header compression alone, you need two scenarios whose only difference is header compression. Since HTTP/1.1 doesn't support header compression, you need to make the experiment using HTTP/2. In one, you ask your HPACK encoder to not use any sort of compression (no Huffman encoding, no use of tables at all).

The way I see it, you need quite elaborate server support to measure this over a full-blown website. And you probably want a full-blown website so that your headers are interesting enough. My best suggestion therefore is that you check out NginX source code and edit it's code to basically have HPACK slack as much as possible.

Failing such a comprehensive solution, you can simply measure how many bytes are used in HTTP/2 frames dedicated to headers versus how much bytes are used for headers in HTTP/1.1 fetches of your site. Right now Wireshark and some parsing can help you there. We are planning to add such counters to ShimmerCat, but unfortunately is not there yet (there are quick command line switches to test different protocols though....).

dsign
  • 12,340
  • 6
  • 59
  • 82
  • Your points make sense. For first point, actually I wanted to find a way to measure how much performance improvement we can gain from Header compression in HTTP/2 compared to HTTPS(HTTP/1.1 over ssl). So I am taking measurements of the same web page loading over two protocols. As you have pointed out, my headers might be not that interesting enough. Agree with your suggestion to measure bytes. But I still prefer to find a way to get time vise improvements by adjusting my test suit. But couldn't find a way yet. – Lakmali Erandi Baminiwatta Mar 29 '16 at 11:34
0

After increasing the header size ~2 KB and emulating low bandwidth and high latency conditions, I was able to see TTFB reduction between first and second requests.

Lakmali Erandi Baminiwatta
  • 2,295
  • 15
  • 18