How to mask sensitive information of POST body in nginx log?

Question

For the convenience of analyzing we save the $request_body field in access log. However, there are some sensitive information inside the post body, such as password or credit card number, exposed in the logs. How can we mask these information?

password=1234asdf  ->  password=****

If I write a nginx module to mask the data, should I write a new log module or should I manipulate the request body before the original log module called?
Or should I use nginx-lua to achieve this goal?
Or is there any other methods?

Answer 1

Use 'echo_read_request_body' command to get the HTTP POST data and then filter the password using 'map' and regex

map $request_body $req_body_start {
    "~(?<nopwd>.*)\&password=[^\&]*.+"  $nopwd;
    default        $request_body;
}

map $request_body $req_body_end {
    "~.*\&password=[^\&]*(?<nopwd1>.+)"  $nopwd1;
    default        '';
}

 map $request_body $req_body_pwd {
    "~.*\&password=[^\&]*.+"  '&password=****';
    default        '';
}

Then define log_format and use it on a server/location level:

log_format  logreqbody  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for" "$req_body_start$req_body_pwd$req_body_end"';

Here is more info: https://www.rstcloud.net/blog/30-how-to-hide-sensitive-post-data-in-nginx-log

Answer 2

One Expression Map that can mask the password field regardless of its position in the argument list

    map $request_body $request_body_masked {
            "~(?<pretext>.*)(?<pass0>password)=[^&]+(?<last>.*)$"  '$pretext$pass0=*****$last';
            default  '';
    }

Tested for following patterns

• password=papaya&id=someuser
• password=papaya&
• password=papaya
• &password=papaya
• id=someuser&password=papaya&remember=1
• id=someuser&password=papaya

However, need to benchmark the performance cost of running the regex map over the body of each request.

Answer 3

This is my version.

@Yury Sergeev answer is only effective if there is only a single password in the request.body. But if you have a registration page where you have confirmation_password and other passwords like &password=pass1234&confirmation_password=pass1234&withdrawal_password=pass1234 it will not work.

My version is even though manual and dirty.. But it will still hide those passwords

map $request_body $req_body_step_1 {
    "~(?<start_request>.*)\&(?<pwd>.*password)=[^\&]*(?<end_request>.+)"  "$start_request$end_request";
    default        $request_body;
}

map $req_body_step_1 $req_body_step_2 {
    "~(?<start_request>.*)\&(?<pwd>.*password)=[^\&]*(?<end_request>.+)"  "$start_request$end_request";
    default        $req_body_step_1;
}

map $req_body_step_2 $req_body_step_3 {
    "~(?<start_request>.*)\&(?<pwd>.*password)=[^\&]*(?<end_request>.+)"  "$start_request$end_request";
    default        $req_body_step_2;
}

map $req_body_step_3 $req_body_step_4 {
    "~(?<start_request>.*)\&(?<pwd>.*password)=[^\&]*(?<end_request>.+)"  "$start_request$end_request";
    default        $req_body_step_3;
}

map $req_body_step_4 $req_body_step_5 {
    "~(?<start_request>.*)\&(?<pwd>.*password)=[^\&]*(?<end_request>.+)"  "$start_request$end_request";
    default        $req_body_step_4;
}

map $req_body_step_5 $req_body_step_6 {
    "~(?<pwd>.*password)=[^\&]*(?<end_request>.*)"  "$end_request";
    default        $req_body_step_5;
}

log_format  main  '"Request-Body": $req_body_step_6';

Your request body like username=3124124&password=3124124&confirmation_password=3124124&withdrawal_password=24124134231&real_name=Dean&phone=13412345678&email=xxx@yahoo.com

will become username=312412&real_name=Dean&phone=13412345678&email=xxx@yahoo.com