1

I'm using a self compiled nginx/1.13.8 with the additional modules brotli and headers-more-nginx-module but my bug occurs independently from activating brotli or not. Server is running Debian 9. Most of the time everything works but sometimes one or a few requests (e.g. to css/js ressources) results in the following errors. All requests are served through http/2:

chrome: ERR_SPDY_PROTOCOL_ERROR

firefox: loading failed

safari: kCFErrorDomainCFNetwork-Fehler 303

edge: (same bug, can't test it right now; going to update this later)

My nginx SSL Config (that seems to be fine (A+) according to ssllabs):

ssl_certificate      "/etc/letsencrypt/live/***/fullchain.pem";
ssl_certificate_key  "/etc/letsencrypt/live/***/privkey.pem";
ssl_protocols TLSv1.2;
ssl_dhparam /etc/ssl/dhparam.pem;

ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout  10m;

    #raymii.org/s/tutorials/Strong_SSL_Secruity_On_nginx.html
ssl_ciphers  'EECDH-AESGCM:EDH+ESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers  on;

Due to I'm new to servers and server-management I have no clue how I can debug this problem. All I know is that the error most-likely didn't happened with the nginx from the debian repo but I'm not certain.

My guess is that it has something to do with the ciphers because since I changed them from their last value the error occurs less often. Server-Log seems fine: for example:

**MY-IP** - - [23/Jan/2018:10:33:06 +0100] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /portal HTTP/2.0" 200 383 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
**MY-IP** - - [23/Jan/2018:10:33:06 +0100] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /styles.a01bb74b47d88d296c44.bundle.css HTTP/2.0" 200 24238 "***" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
**MY-IP** - - [23/Jan/2018:10:33:06 +0100] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /inline.bfe190f13378e2257d4e.bundle.js HTTP/2.0" 200 731 "***" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
**MY-IP** - - [23/Jan/2018:10:33:06 +0100] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /polyfills.74b809925dee18bd9f89.bundle.js HTTP/2.0" 200 19182 "***" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
**MY-IP** - - [23/Jan/2018:10:33:06 +0100] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /scripts.1cd17589767e3c3fbdfe.bundle.js HTTP/2.0" 200 40807 "***" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"

**MY-IP** - - [23/Jan/2018:10:33:06 +0100] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /main.c0a6975cd3e3b14f7b2a.bundle.js HTTP/2.0" 200 0 "***" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
--> The one that failed in this case! - looks fine?

By the way this happens on different devices with different operating systems.

phip1611
  • 111
  • 5
  • whats the error log? – Ilham Sulaksono Jan 23 '18 at 10:39
  • Oh, I missed that! It says: ```2018/01/23 14:05:25 [crit] 15699#0: *1 open() "/var/lib/nginx/proxy/1/00/0000000001" failed (13: Permission denied) while reading upstream, client: 89.16.*.*, server: ***.tld, request: "GET /scripts.1cd17589767e3c3fbdfe.bundle.js HTTP/2.0", upstream: "http://127.0.0.1:8080/scripts.1cd17589767e3c3fbdfe.bundle.j‌​s", host: "juniorzeit.de", referrer: "https://***.tld/portal"``` – phip1611 Jan 23 '18 at 13:16

2 Answers2

0

I think I found the solution (but I'm not 100% sure!). As the error log says (see my comment under the question) there are permission denied problems in /var/lib/nginx/proxy/**. all directories and files there belonged to "nobody" while nginx was running as "nginx". I changed the owner of the nginx-process to "nobody", now it works.

Anyway, how could it worked 90% of the time and 10% of the time it crashed? In my opinion permission denied should either hit all the time or never... but sometimes?

phip1611
  • 111
  • 5
  • Okay I think I know what went wrong! Before I used my self-compiled version I used the version from the debian-repo. Both used `/var/lib/nginx/proxy/**` but with *different owners* which caused permission denied problems. I deleted `/var/lib/nginx/proxy/` and nginx recreated the directory after a restart. – phip1611 Jan 23 '18 at 14:14
0

Depending on the size of the file being proxied nginx can buffer to disk.

My guess is that the failing files are larger than the ones that succeed. So only in those cases it tries to buffer it and fails at it.

What you are getting aren't SSL errors but http/spdy protocol errors. Most likely because the size set in the Content-length header mismatches the amount transferred. I hope that answers your question of why it fails.

evilBunny
  • 54
  • 4