6

Why does a variable not work in proxy_pass?

This works perfectly:

location /foo/ {
  proxy_pass http://127.0.0.1/;
}

This doesn't work at all:

location /foo/ {
  set $FOO http://127.0.0.1/;
  proxy_pass $FOO;
  add_header x-debug $FOO;
}

I see the x-header: http://127.0.0.1/ but the result is 404 so I don't know where it's proxying to but it's not identical to the first example.

Source where it is explained that using a variable in proxy_pass will prevent NGINX startup errors when the upstream is not available.

UPDATE: The issue is the upstream path rewriting. I expect it to rewrite /foo/blah to the upstream at /blah removing the /foo prefix. It works fine with static host/uri entries but not with a variable.

Marc
  • 13,011
  • 11
  • 78
  • 98
  • Did you add a `resolver` directive to your nginx configuration? – Ivan Shatsky Feb 21 '22 at 10:26
  • I'm confused, a resolver is a reference to a DNS Server. How does that apply to resolving variables? In fact I use 127.0.0.1 (instead of localhost) so I don't need DNS... – Marc Feb 21 '22 at 10:34
  • Tested this in a sandbox with OpenResty 1.17.8.2 (based on nginx 1.17 core) and it works like a charm for me without any additional `resolver`. Cannot text with the vanilla nginx right now... – Ivan Shatsky Feb 21 '22 at 11:01
  • @IvanShatsky the issue was the path stripping. We want inbound /foo/bar/ to route to upstream /bar/. – Marc Apr 15 '22 at 19:02
  • 1
    Indeed, and `set $FOO http://127.0.0.1; proxy_pass $FOO/;` didn't cut the `/foo` URI prefix either. However your final solution is somewhat over-complicated, the more simple `rewrite ^/foo(/.*) $1 break; proxy_pass http://$FOO;` will do the same slightly more efficiently. – Ivan Shatsky Apr 16 '22 at 08:33

2 Answers2

7

The final answer, much aided by @MSalters was more complicated than I could imagine. The reason is that NGINX works differently with variables than with statically entered hostnames - it does not even use the same DNS mechanism.

The main issue is that path handling and prefix stripping does not work the same with variables. You have to strip path prefixes yourself. In my original example:

location /foo/ {
  set $FOO 127.0.0.1;
  rewrite /foo/(.*) /$1 break;
  proxy_pass http://$FOO/$1$is_args$args;
}

In my example I use an IP address so no resolver is required. However, if you use a host name a resolver is required so add your DNS IP there. Shrugs.

For full disclosure, we are using NGINX inside Kubernetes so it gets even more complicated. The special points of interest are:

  1. Add a resolver directive with the IP of the cluster's DNS service (in my case 10.43.0.10). This is the ClusterIP of the kube-dns service in the kube-system namespace.
  2. Use a FQDN even if your NGINX is in the same namespace since the DNS can only resolve FQDN apparently.
location /foo/ {
  set $MYSERVICE myservice.mynamespace.svc.cluster.local;
  rewrite /foo/(.*) /$1 break;
  proxy_pass http://$MYSERVICE/$1$is_args$args;
  resolver 10.43.0.10 valid=10s;
}

NOTE: Due to a BUG (which is unfortunately not acknowledged by NGINX maintainers) in NGINX, using $1 in URLs will break if the path contains a space. So /foo%20bar/ will be passed upstream as /foo bar/ and just break.

Marc
  • 13,011
  • 11
  • 78
  • 98
0

The idea of variables in nginx.conf is to delay evaluation. An nginx.conf configuration is parsed for two different reasons: on startup, and later when a request is made. In the second parse, request-dependent variables are filled in.

This parser is not very smart, and this behavior is not officially specified. The trick in your link is a proper hack.

The work-around I use in production is proxy-pass http://$FOO. The concatenation of the literal string http and the variable $FOO does happen when a request is made, showing that variables do work in proxy-pass. Why it doesn't work with a plain substitution, I don't know. And since it's an undocumented hack, this might change from version to version. It would be nice if nginx itself was smarter.

[EDIT]

In some cases, the part of a request URI to be replaced cannot be determined: ... When variables are used in proxy_pass: location /name/ { proxy_pass http://127.0.0.1$request_uri;} In this case, if URI is specified in the directive, it is passed to the server as is, replacing the original request URI.

This different behavior if a variable is present is spelled out in the manual. You could try a rewrite directive, which modifies the URI to be sent.

MSalters
  • 173,980
  • 10
  • 155
  • 350
  • Still doesn't work with that workaround - it's proxying differently and not removing the path prefix `/foo`. There is no equivalence. Tried `set $FOO 127.0.0.1/; proxy_pass http://$FOO;` and `set $FOO 127.0.0.1; proxy_pass http://$FOO/;` with/without slash. – Marc Feb 21 '22 at 11:36
  • I've updated my answer - the issues is not that it doesn't work but that it doesn't work the same as before stripping the `/foo` path before proxying. – Marc Feb 22 '22 at 11:57
  • @Marc: See the manual; it's a documented quirk. – MSalters Feb 22 '22 at 12:37
  • Yep it's quriky. I can't get it to resolve a hostname as a variable which it resolves just fine when statically entered. `some.host could not be resolved (3: Host not found)` If I just enter `http://some.host` it works fine. – Marc Feb 22 '22 at 14:37
  • Adding a resolver seems to do the trick as NGINX doesn't do normal DNS when it sees a variable. `resolver 10.43.0.10 valid=10s;` (for Kubernetes) – Marc Feb 22 '22 at 15:00
  • @Marc: Ah. My config had `resolver 127.0.0.11` (Docker). – MSalters Feb 22 '22 at 15:59