1

Here is my nginx config file

user  nginx;
worker_processes  auto;


error_log  /var/log/nginx/error.log error;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}


stream {
    server {
        listen *:443 ssl;

        ssl_certificate             /etc/nginx/ssl/server.crt;
        ssl_certificate_key         /etc/nginx/ssl/server.key;

        ssl_session_cache shared:SSL:20m;
        ssl_session_timeout 60m;

        proxy_pass 127.0.0.1:1935;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DHE+AES128:!ADH:!AECDH:!MD5;
    }
}

I am not happy with it because nginx consumes too much cpu. How to tune it properly? Here I have rtmp listener at 127.0.0.1:1935. So nginx is only terminating ssl and passes by rtmp further.

This is how cpu consumption looks for 1cpu server. About 50% is eaten by nginx and the rest by rtmp server. I would like to make nginx consume less CPU. cpu consumption

Nginx version

nginx -V
nginx version: nginx/1.15.2
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) 
built with OpenSSL 1.1.0f  25 May 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.15.2/debian/debuild-base/nginx-1.15.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

The load you see is provided by 500 simultaneous stream uploads by command

ffmpeg -re -stream_loop -1 -i /dolbycanyon.mp4 -acodec copy -vcodec copy -f flv rtmps://rtmp:443/live/live139

from another server.

I generated certificate files /etc/nginx/ssl/server.crt and /etc/nginx/ssl/server.key by command

openssl req -config ./openssl.conf -x509 -nodes -days 365 -newkey rsa:2048 -keyout selfsigned.key -out selfsigned.crt

where openssl.conf is

[req]
prompt = no
distinguished_name = req_distinguished_name
req_extensions = v3_req

[req_distinguished_name]
C = US
ST = California
L = Los Angeles
O = Our Company Llc
#OU = Org Unit Name
CN = Our Company Llc
#emailAddress = info@example.com

[v3_req]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = example.com
DNS.2 = www.example.com

Update 1

So when I use nginx ssl termination, it can stand maximum 512 streams for 1 CPU. When I stream directly to RTMP port 1935 (without nginx) it's 970 simultaneous streams. So I would like to optimize nginx to get a number a bit closer to 970 streams rather than 512 streams.

Update 2

Looks like the problem is not in ssl termination but in nginx forwarding itself. When I use simple forwarding (without ssl):

user  nginx;
worker_processes  auto;


error_log  /var/log/nginx/error.log error;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}


stream {
    server {
        listen *:1936;
        proxy_pass 127.0.0.1:1935;
    }
}

It gives 511 simultaneous streams. It's almost the same amount as when I used ssl: 512. And I have the same cpu consumption ratio: about 50% for nginx and about 50% for rtmp server itself. In this case, I upload stream to 1936 (without ssl):

ffmpeg -re -stream_loop -1 -i /dolbycanyon.mp4 -acodec copy -vcodec copy -f flv rtmp://rtmp:1936/live/live6

Update 3

1) I checked /etc/security/limits.conf file. It was empty. I added a couple lines to it:

web soft nofile 65535
web hard nofile 65535

2) Also, I added file ULIMIT="-n 65535" to file /etc/default/nginx

3) And added line worker_rlimit_nofile 65535; to /etc/nginx/nginx.conf:

user  nginx;
worker_processes  auto;


error_log  /var/log/nginx/error.log error;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

worker_rlimit_nofile 65535;

stream {
    server {
        listen *:1936;
        proxy_pass 127.0.0.1:1935;
    }
}

But the result is still the same: 512 connections. By the way, I've done 1) for both host and Docker container. I run all this stuff inside a container which is living inside gcloud compute instance.

0
+50

512 is the default number of the worker_connections value, which defines the number of connections per process (in this case, 1CPUx512Connections). My thoughts on that is you might be hitting a default limit even though you've obviously increased the default worker_connections as defined in the docs around it. http://nginx.org/en/docs/ngx_core_module.html#worker_connections

Have you tried increasing a worker_rlimit_nofile value or checking that the /etc/security/limits.conf file has any limits in it?

  • Increasing worker_rlimit_nofile should probably be the first step here. 1024 is way too low. – Michael Hampton Aug 10 '18 at 16:26
  • This post is more suited as a comment than an answer. Remember the goal is to have Q&A which together can be referenced in the future to answer the same question. – Xalorous Aug 10 '18 at 17:58
  • @MichaelHampton, Rhys I tried to play with worker_rlimit_nofile and /etc/security/limits.conf, still 512 streams please see update 3 above. – Maxim Yefremov Aug 11 '18 at 10:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.