filter out info in nginx log_format

Question

I want to log the $request coming in but want to filter out sensitive information such as passwords or keys.

I tried the following and it doesn't work:

http {
  log_format xxx '$filtered_request';

  location /xxx {
    set $filtered_request $request;
    if ($filtered_request ~ (.*)password=[^&]*(.*)) {
      set $filtered_request $1password=****$2;
    }

    access_log /var/log/xxx.log xxx;
  }
}

This saves empty lines to the log file.

In fact, the following doesn't work either:

http {
  log_format yyy '$yyy';

  location /foo {
    set $yyy 'abc';
    access_log /var/log/yyy.log yyy;
  }
}

The result is still empty lines.

How to use custom variables in log_format?

I'm using nginx/1.2.5

UPDATE: I notice that the set $yyy 'abc'; actually does something, but the value is not reflected in the log. i.e.:

http {
  log_format yyy '$request $arg_password';
  location /foo {
    set $arg_password 'filtered';
    access_log /var/log/yyy.log yyy;    
  }
}

The $arg_password becomes empty with the set ... statement, and log the arg ?password=asdf as asdf if the set ... statement is commented out.

Answer 1

It turns out there are some rewrite going on in that location:

location /foo {
  ...
  rewrite ^(/foo)... ... break;

  set $filtered_request $request;
  ...
}

If I move the set statements before the rewrite, it works fine:

location /foo {
  ...

  set $filtered_request $request;
  ...

  rewrite ^(/foo)... ... break;

  ...
}

Doesn't know why it had the side effect of making the variable empty. But at least the problem at hand is solved.