HTTP/2 Server Push results in duplicate requests

Question

A response for a document with the following headers enters Nginx:

link: </picture.jpg>; as=image; rel=preload
link: </_next/static/chunks/commons.4e96503c89eea0476d3e.module.js>; as=script; rel=preload
link: </_next/static/runtime/main-3c17cb16bbbc3efc4bb5.module.js>; as=script; rel=preload
link: </_next/static/runtime/webpack-0b10894b69bf5a718e01.module.js>; as=script; rel=preload
link: </_next/static/Q53NXtgLT1rgpqOOsVV6Q/pages/_app.module.js>; as=script; rel=preload
link: </_next/static/Q53NXtgLT1rgpqOOsVV6Q/pages/index.module.js>; as=script; rel=preload

With the help of HTTP/2 Server Push the requests are Pushed to the client but 5 out of the 6 requests download two times (once with the push and once triggered by the document). The Network tab in Chrome Dev Tools looks like this: I've tested if the Type is set properly and it seems alright. What could be the issue?

Consecutive requests (chrome cache enabled) result in a similar way as well:

What could be wrong? I'm pretty sure the request should not duplicate

@edit I tried doing the Server Push without Nginx (talking directly to Node.js backend instead of the backend attaching link headers for Nginx). It works without an issue. The problem pops up when I use Nginx.

Btw. I do know that one should not push all the contents via Server Push, especially images, but I did it just for a clear test. If you look closer it seems that only the scripts get duplicated and the picture downloads only once.

Yes, the preload link and script tags are tagged with cross-origin anonymous. I've tried changing these to use-credentials as well but the problem persists. — Dawid, Jan 05 '20 at 20:11
I’m surprised that didn’t fix it. You shouldn’t need the preload links btw as you now have preload headers which do the same thing. Try removing them completely. — Barry Pollard, Jan 05 '20 at 20:34
Yes I'll try to do this as well They're automatically placed by the library that I use - Next.js — Dawid, Jan 05 '20 at 20:48

Dawid · Accepted Answer · 2020-01-06T19:56:09.030

The core of the problem is actually Chromium. This thing only fails in Chromium from what I can see.

The problem with Nginx is in the implementation of http2_push_preload.

What Nginx seeks is a header with Link: </resource>; as=type; rel=preload. It reads it and serves the files via push, unfortunately when the browser (I only tested Chrome actually) receives the document with the Link header as well as the Push it conflicts resulting in a significant slowdown and downloads the resources that were seen while parsing the document instead.

# This results in HTTP/2 Server Push and the requests get duplicated due to the `Link` headers that were passed along
location / {
    proxy_pass http://localhost:3000;
    http2_push_preload on;
}

# This results in Resource Hints getting triggered in the browser.
location / {
    proxy_pass http://localhost:3000;
}

# This results in a regular HTTP/2 (no push)
location / {
    proxy_pass http://localhost:3000;
    http2_push_preload on;
    proxy_hide_header link;
}

# This result in a valid HTTP/2 Server Push (proper)
location / {
    proxy_pass http://localhost:3000;
    http2_push /commons.4e96503c89eea0476d3e.module.js;
    http2_push /index.module.js;
    http2_push /_app.module.js;
    http2_push /webpack-0b10894b69bf5a718e01.module.js;
    http2_push /main-3c17cb16bbbc3efc4bb5.module.js;
}

It seems Nginx does not work well with this feature yet...

If only I could remove the Link headers and use http2_push_preload...

Anyway I got it to work with the usage of H2O H2O did let me delete the headers while preserving HTTP/2 Server Push

// h2o.conf
  [...]
  proxy.reverse.url: "http://host.docker.internal:3000/"
  header.unset: "Link"

Works alright with H2O: I hope Nginx fixes the way http2_push_preload works and allows for more control.

Along the side, I think that Chromium should deal with the issue anyway instead of downloading 2 times as many bytes.

I confirm that `http2_push_preload on` doesn't play well with `proxy_hide_header`. How can this still be the case? Does it mean that anyone trying to use `http2_push_preload` on nginx today gets broken push? — cherouvim, Feb 19 '21 at 20:11
Last time i checked it was still broken. I've switched to H2O as the HTTP/2 push server. It's far superior in that manner with http2-casper. — Dawid, Feb 20 '21 at 11:16
Did you still have nginx behind, or did you point H2O directly towards your backend? — cherouvim, Feb 21 '21 at 14:47

score 1 · Answer 2 · answered Jan 05 '20 at 20:40

Your HTML is requested on a normal “credentialed” connection. It then pushes the JPG and JS on that connection.

Your page then also loads the via anonymous cross-origin settings. So it cannot use the pushes resources and requests them again.

See here for more details: https://jakearchibald.com/2017/h2-push-tougher-than-i-thought/#requests-without-credentials-use-a-separate-connection

Btw recommendations are only to push a small amount and not every resource needed by the page. If you even want to use push at all that is since it’s complicated and the gains haven’t really be proven to be worth the complexity on the whole.

I tried pushing the same version of the app without Nginx (handling HTTP2 Push solely on my backend) and it works without an issue this way. See the @edit to a pic. Could I be missing something in the `nginx.conf`? — Dawid, Jan 05 '20 at 21:02

HTTP/2 Server Push results in duplicate requests

2 Answers2