Here is my nginx config file
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log error;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
stream {
server {
listen *:443 ssl;
ssl_certificate /etc/nginx/ssl/server.crt;
ssl_certificate_key /etc/nginx/ssl/server.key;
ssl_session_cache shared:SSL:20m;
ssl_session_timeout 60m;
proxy_pass 127.0.0.1:1935;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DHE+AES128:!ADH:!AECDH:!MD5;
}
}
I am not happy with it because nginx consumes too much cpu. How to tune it properly? Here I have rtmp listener at 127.0.0.1:1935. So nginx is only terminating ssl and passes by rtmp further.
This is how cpu consumption looks for 1cpu server. About 50% is eaten by nginx and the rest by rtmp server. I would like to make nginx consume less CPU.
Nginx version
nginx -V
nginx version: nginx/1.15.2
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
built with OpenSSL 1.1.0f 25 May 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.15.2/debian/debuild-base/nginx-1.15.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'
The load you see is provided by 500 simultaneous stream uploads by command
ffmpeg -re -stream_loop -1 -i /dolbycanyon.mp4 -acodec copy -vcodec copy -f flv rtmps://rtmp:443/live/live139
from another server.
I generated certificate files /etc/nginx/ssl/server.crt and /etc/nginx/ssl/server.key by command
openssl req -config ./openssl.conf -x509 -nodes -days 365 -newkey rsa:2048 -keyout selfsigned.key -out selfsigned.crt
where openssl.conf is
[req]
prompt = no
distinguished_name = req_distinguished_name
req_extensions = v3_req
[req_distinguished_name]
C = US
ST = California
L = Los Angeles
O = Our Company Llc
#OU = Org Unit Name
CN = Our Company Llc
#emailAddress = info@example.com
[v3_req]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = example.com
DNS.2 = www.example.com
Update 1
So when I use nginx ssl termination, it can stand maximum 512 streams for 1 CPU. When I stream directly to RTMP port 1935 (without nginx) it's 970 simultaneous streams. So I would like to optimize nginx to get a number a bit closer to 970 streams rather than 512 streams.
Update 2
Looks like the problem is not in ssl termination but in nginx forwarding itself. When I use simple forwarding (without ssl):
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log error;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
stream {
server {
listen *:1936;
proxy_pass 127.0.0.1:1935;
}
}
It gives 511 simultaneous streams. It's almost the same amount as when I used ssl: 512. And I have the same cpu consumption ratio: about 50% for nginx and about 50% for rtmp server itself. In this case, I upload stream to 1936 (without ssl):
ffmpeg -re -stream_loop -1 -i /dolbycanyon.mp4 -acodec copy -vcodec copy -f flv rtmp://rtmp:1936/live/live6
Update 3
1) I checked /etc/security/limits.conf file. It was empty. I added a couple lines to it:
web soft nofile 65535
web hard nofile 65535
2) Also, I added file ULIMIT="-n 65535" to file /etc/default/nginx
3) And added line worker_rlimit_nofile 65535; to /etc/nginx/nginx.conf:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log error;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
worker_rlimit_nofile 65535;
stream {
server {
listen *:1936;
proxy_pass 127.0.0.1:1935;
}
}
But the result is still the same: 512 connections. By the way, I've done 1) for both host and Docker container. I run all this stuff inside a container which is living inside gcloud compute instance.
