How to forward the URL's all parameters through a proxy_pass with nginx?

Question

Nginx config:

location /proxy/ {
     if ($request_method = HEAD) { return 200; }

     if ( $arg_address != "" ) {
      proxy_pass $arg_address;
      return 301 $arg_address;
      }  

     proxy_ssl_verify       off;
     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header X-Real-IP $remote_addr;
 }

this urls works:

https://localhost/proxy/?address=https://exemple.com/transfer/file.txt ==>> https://exemple.com/transfer/file.txt

or

https://localhost/proxy/?address=https://exemple.com/transfer/file.txt?host-id=1 ==>> https://exemple.com/transfer/file.tx?host-id=1

if I add multiple parameters, it will be truncated to the first "&"

https://localhost/proxy/?address=https://exemple.com/transfer/file.txt?host-id=1&password=123456&date=xxxxxx ==>> https://exemple.com/transfer/file.txt?host-id=1

How can I transfer the entire url?

score 0 · Answer 1 · answered May 20 '21 at 12:19

Before we get to the answer, may I ask what do you want to achieve? Do you want to proxy the request or to generate HTTP 301 redirect? With the following construction

if ( $arg_address != "" ) {
    proxy_pass $arg_address;
    return 301 $arg_address;
}

you'll always get a redirect because the directives from ngx_http_rewrite_module are executed before any others, so the proxy_pass directive is useless here. ngx_http_rewrite_module is very special and different from most of the other modules. Although nginx configuration in general is declarative, rewrite module evaluates its instructions imperatively. This is always a source of confusion for every nginx novice. You can read more about the rewrite module internal implementation here.

If you want to proxy the request instead of generating a redirect, you'll need to remove return and add a resolver directive to your configuration. Here you can read why it is required.

A "dirty hack" solution

Being that said, get back to the question. Of course, when nginx receives the request

https://localhost/proxy/?address=https://example.com/transfer/file.txt?host-id=1&password=123456&date=20210520

arg_NAME variables will be filled the following way:

arg_address => https://example.com/transfer/file.txt?host-id=1
arg_password => 123456
arg_date => 20210520

It is correct and expected behavior.

What you can do to preserve all the other query arguments? The most simple is to assume that all query arguments following the address one are subject to pass to the upstream and use a map directive to get the required string:

map $args $address {
    ~(?:^|&)(address=.*)    $1;
}

server {
    ...
    location /proxy/ {
        ...
        if ($address) {
            # 'proxy_pass $address' or 'return 301 $address' here
        }  
        ...
    }
    ...
}

Here is more strict check where we take the rest of the query string only if there is a question mark after address query parameter and only $arg_address value otherwise:

map $args $address {
    ~(?:^|&)(address=[^&?]+\?.*)    $1;
    default                         $arg_address;
}

Reliable solution

While the answer above is generally workable, I'd rather try to design my proxy solution using URL encoding on address query argument to avoid reserved characters usage as part of the query argument value. The above request being URL-encoded would look like

https://localhost/proxy/?address=https%3A%2F%2Fexample.com%2Ftransfer%2Ffile.txt%3Fhost-id%3D1%26password%3D123456%26date%3D20210520

The bad thing is that "vanilla" nginx doesn't have an ability to URL-decode an arbitrary string. However it can be done using OpenResty/lua-nginx-module:

location /proxy/ {
    ...
    set_by_lua_block $address { return ngx.unescape_uri(ngx.var.arg_address) }
    if ($address) {
        # 'proxy_pass $address' or 'return 301 $address' here
    }  
    ...
}

or set-misc-nginx-module:

location /proxy/ {
    ...
    if ($arg_address) {
        set_unescape_uri $address $arg_address;
        # 'proxy_pass $address' or 'return 301 $address' here
    }  
    ...
}

Perhaps the same can be done using njs, but I didn't use it and can't give you an example.

How to forward the URL's all parameters through a proxy_pass with nginx?

1 Answers1