Improve performance of Nginx with http auth and a lot of requests

Question

I've got an PHP side running via fpm behind an Nginx server. For $reasons we need an http base auth in front of that setup, so I ended up in a setup like:

#… server section ….

    auth_basic "Restricted";
    auth_basic_user_file /path/to/htpasswd;

#… some more locations …

location ~ \.php$ {


    fastcgi_pass 127.0.0.1:9001;
    fastcgi_split_path_info ^(.+\.php)(/.*)$;
    include /etc/nginx/fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param HTTPS off;
    fastcgi_param APPLICATION_ENV production;
}

Which is working — but very slow. It's only processing one request after the other by 100% cpu utilization. If I remove the http_auth it's working fast.

My question is: How to improve the setup to ensure even with http_auth the performances is okish?

For reference:

# nginx -V
nginx version: nginx/1.8.1
built with OpenSSL 1.0.2j  26 Sep 2016
TLS SNI support enabled
configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --pid-path=/run/nginx.pid --lock-path=/run/lock/nginx.lock --with-cc-opt=-I/usr/include --with-ld-opt=-L/usr/lib --http-log-path=/var/log/nginx/access_log --http-client-body-temp-path=/var/lib/nginx/tmp/client --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --with-ipv6 --with-libatomic --with-pcre --with-http_realip_module --add-module=external_module/ngx_devel_kit-0.2.19 --add-module=external_module/lua-nginx-module-0.9.15 --add-module=external_module/modsecurity-2.9.1-nginx-207c85d/nginx/modsecurity --with-http_ssl_module --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --user=nginx --group=nginx

When you say removing "http_auth" makes it fast, there is nothing in the config file that looks like "http_auth". Please be precise about your scenario. — Tim, Nov 23 '16 at 18:29
I removed it and it was fast. Simple binary test. I was refering to auth_basic so maybe wrong wording. — frlan, Nov 23 '16 at 18:30
Please edit your question so it's clear. Also suggest using good grammar: "For $reasons" should be "To control costs" or similar. Show a curl, access log, and error log. — Tim, Nov 23 '16 at 18:32

score 0 · Answer 1 · answered Apr 21 '21 at 09:32

I had same the same issue using Nginx and basic HTTP Auth. With htpasswd I was using the bcrypt option with too many rounds. I tried the max of 17 (with option -C 17) but the server did not like that at all. The CPU went to 100% and each page took a minute to load.

htpasswd -B -C 17 -c /etc/nginx/.htpasswd username

The -C option, which is only used when bcrypt -B is used, sets the computing time used for the bcrypt algorithm (higher is more secure but slower, default: 5, valid: 4 to 17). The cost of computing a bcrypt password hash value increases with the number of rounds specified by the -C option.

The default encryption algorithm htpasswd uses is a version of MD5 modified for Apache. But with the -B option you can use bcrypt. Bcrypt is considered very secure right now.

I ended up settling on 7 rounds. Which seems to work well, CPU doesn't spike and page load time is fine.

htpasswd -B -C 7 -c /etc/nginx/.htpasswd username

score 0 · Accepted Answer · answered Nov 24 '16 at 08:49

The root cause for my issue was not direct Nginx caused, but algorithm I used for htpasswd. As this file is checked again and again in my config above, it's kind of important to use an algorithm not being this resource hungry. I was originally using a sha512-based algorithm invoked by Python hashlib

passlib.hash.sha512_crypt.encrypt(password)

which was too much. When changing to simpler algorithm via direct call of htpasswd

htpasswd /path/to/passwdfile myusername

the performance issue were gone.

Improve performance of Nginx with http auth and a lot of requests

2 Answers2