Add headers in nginx config on AWS Elastic Beanstalk

Question

I'm trying to have the nginx on my Elastic Beanstalk application add some headers to every response.

I've edited my 00_application.conf file (in .ebextensions/nginx/conf.d/elasticbeanstalk) to include:

location / {
     # snip other config
     proxy_set_header    Strict-Transport-Security "max-age=86400; includeSubDomains";
     proxy_set_header    Referrer-Policy           "no-referrer";
     proxy_set_header    Content-Security-Policy   "default-src 'none'";
     proxy_set_header    X-Content-Type-Options    "nosniff";
     proxy_set_header    X-Frame-Options           "deny";
     proxy_set_header    X-XSS-Protection          "0";
}

However, when I make a request, only some of these headers are coming through: I'm seeing the X-Frame-Options: DENY and X-Content-Type-Options: nosniff, but the X-XSS-Protection header has the value 1; mode=block and the three other headers aren't present at all.

What am I doing wrong, and how do I do it correctly?