I'm trying to setup an HTTP2 node server on elastic beanstalk using nginx. I have been reading some articles and I came across this one: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/nodejs-platform-proxy.html . Where it explains how to change the nginx configuration. I modified the http version to use http 2.0, as follows:
files:
/etc/nginx/conf.d/proxy.conf:
mode: "000644"
owner: root
group: root
content: |
upstream nodejs {
server 127.0.0.1:8081;
keepalive 256;
}
server {
listen 8080;
if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
set $year $1;
set $month $2;
set $day $3;
set $hour $4;
}
access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour healthd;
access_log /var/log/nginx/access.log main;
location / {
proxy_pass http://nodejs;
proxy_set_header Connection "";
proxy_http_version 2.0;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
gzip on;
gzip_comp_level 4;
gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
location /static {
alias /var/app/current/static;
}
}
/opt/elasticbeanstalk/hooks/configdeploy/post/99_kill_default_nginx.sh:
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash -xe
rm -f /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf
service nginx stop
service nginx start
container_commands:
removeconfig:
command: "rm -f /tmp/deployment/config/#etc#nginx#conf.d#00_elastic_beanstalk_proxy.conf /etc/nginx/conf.d/00_elastic_beanstalk_proxy.conf"
Basically I have just modified the line proxy_http_version 2.0;. But it does not work! The environment is not healthy and I'm seeing the following logs:
access.log
172.16.1.51 - - [03/Nov/2021:15:44:45 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.0.132 - - [03/Nov/2021:15:45:03 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.1.51 - - [03/Nov/2021:15:45:15 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.0.132 - - [03/Nov/2021:15:45:34 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.1.51 - - [03/Nov/2021:15:45:45 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.0.132 - - [03/Nov/2021:15:46:04 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.1.51 - - [03/Nov/2021:15:46:15 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
217.216.95.140 - - [03/Nov/2021:15:46:22 +0000] "GET /health HTTP/1.1" 009 26 "-" "curl/7.64.1" "-"
217.216.95.140 - - [03/Nov/2021:15:46:32 +0000] "GET /health HTTP/1.1" 009 26 "-" "curl/7.64.1" "-"
172.16.0.132 - - [03/Nov/2021:15:46:34 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.1.51 - - [03/Nov/2021:15:46:45 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.0.132 - - [03/Nov/2021:15:47:04 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
172.16.1.51 - - [03/Nov/2021:15:47:15 +0000] "PRI * HTTP/2.0" 400 157 "-" "-" "-"
The requests from curl are mine, trying to debug the problem. The other ones are the health checker of my load balancer. In curl I'm using the public IP of the EC2 instance, getting the following:
curl --http2 --insecure --verbose http://18.232.139.145/health
* Trying 18.232.139.145...
* TCP_NODELAY set
* Connected to 18.232.139.145 (18.232.139.145) port 80 (#0)
> GET /health HTTP/1.1
> Host: 18.232.139.145
> User-Agent: curl/7.64.1
> Accept: */*
> Connection: Upgrade, HTTP2-Settings
> Upgrade: h2c
> HTTP2-Settings: AAMAAABkAARAAAAAAAIAAAAA
>
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 26)
* Closing connection 0
And from the erros.log:
2021/11/03 15:44:00 [error] 4395#4395: *33 connect() failed (111: Connection refused) while connecting to upstream, client: 205.185.120.103, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "18.232.139.145:80"
2021/11/03 15:44:03 [warn] 4395#4395: *35 using uninitialized "year" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:03 [warn] 4395#4395: *35 using uninitialized "month" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:03 [warn] 4395#4395: *35 using uninitialized "day" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:03 [warn] 4395#4395: *35 using uninitialized "hour" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:15 [warn] 4395#4395: *36 using uninitialized "year" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:15 [warn] 4395#4395: *36 using uninitialized "month" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:15 [warn] 4395#4395: *36 using uninitialized "day" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:15 [warn] 4395#4395: *36 using uninitialized "hour" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:33 [warn] 4395#4395: *37 using uninitialized "year" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:33 [warn] 4395#4395: *37 using uninitialized "month" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:33 [warn] 4395#4395: *37 using uninitialized "day" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:33 [warn] 4395#4395: *37 using uninitialized "hour" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:45 [warn] 4395#4395: *38 using uninitialized "year" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:45 [warn] 4395#4395: *38 using uninitialized "month" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:45 [warn] 4395#4395: *38 using uninitialized "day" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:44:45 [warn] 4395#4395: *38 using uninitialized "hour" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:03 [warn] 4395#4395: *39 using uninitialized "year" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:03 [warn] 4395#4395: *39 using uninitialized "month" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:03 [warn] 4395#4395: *39 using uninitialized "day" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:03 [warn] 4395#4395: *39 using uninitialized "hour" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:15 [warn] 4395#4395: *40 using uninitialized "year" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:15 [warn] 4395#4395: *40 using uninitialized "month" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:15 [warn] 4395#4395: *40 using uninitialized "day" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:15 [warn] 4395#4395: *40 using uninitialized "hour" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:34 [warn] 4395#4395: *41 using uninitialized "year" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:34 [warn] 4395#4395: *41 using uninitialized "month" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:34 [warn] 4395#4395: *41 using uninitialized "day" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:34 [warn] 4395#4395: *41 using uninitialized "hour" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:45 [warn] 4395#4395: *42 using uninitialized "year" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:45 [warn] 4395#4395: *42 using uninitialized "month" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:45 [warn] 4395#4395: *42 using uninitialized "day" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:45:45 [warn] 4395#4395: *42 using uninitialized "hour" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:04 [warn] 4395#4395: *43 using uninitialized "year" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:04 [warn] 4395#4395: *43 using uninitialized "month" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:04 [warn] 4395#4395: *43 using uninitialized "day" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:04 [warn] 4395#4395: *43 using uninitialized "hour" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:15 [warn] 23073#23073: *1 using uninitialized "year" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:15 [warn] 23073#23073: *1 using uninitialized "month" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:15 [warn] 23073#23073: *1 using uninitialized "day" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:15 [warn] 23073#23073: *1 using uninitialized "hour" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:22 [error] 23073#23073: *2 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 217.216.95.140, server: , request: "GET /health HTTP/1.1", upstream: "http://127.0.0.1:8080/health", host: "18.232.139.145"
2021/11/03 15:46:32 [error] 23073#23073: *4 upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 217.216.95.140, server: , request: "GET /health HTTP/1.1", upstream: "http://127.0.0.1:8080/health", host: "18.232.139.145"
2021/11/03 15:46:34 [warn] 23073#23073: *6 using uninitialized "year" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:34 [warn] 23073#23073: *6 using uninitialized "month" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:34 [warn] 23073#23073: *6 using uninitialized "day" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:34 [warn] 23073#23073: *6 using uninitialized "hour" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:45 [warn] 23073#23073: *7 using uninitialized "year" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:45 [warn] 23073#23073: *7 using uninitialized "month" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:45 [warn] 23073#23073: *7 using uninitialized "day" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:46:45 [warn] 23073#23073: *7 using uninitialized "hour" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:04 [warn] 23073#23073: *8 using uninitialized "year" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:04 [warn] 23073#23073: *8 using uninitialized "month" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:04 [warn] 23073#23073: *8 using uninitialized "day" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:04 [warn] 23073#23073: *8 using uninitialized "hour" variable while logging request, client: 172.16.0.132, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:15 [warn] 23073#23073: *9 using uninitialized "year" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:15 [warn] 23073#23073: *9 using uninitialized "month" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:15 [warn] 23073#23073: *9 using uninitialized "day" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
2021/11/03 15:47:15 [warn] 23073#23073: *9 using uninitialized "hour" variable while logging request, client: 172.16.1.51, server: , request: "PRI * HTTP/2.0"
Any idea what's going on? Do you need more information?
- Note: the server is using fastify with http2 enabled (in local it works great)
