max file descriptors with nginx and IM erlang server

Question

I'm using nginx for serving static files and as a proxy for an Erlang IM server. Each user is using two long-polling http connections. OS is ubuntu server.

On peak hours I see:

> sudo more /proc/sys/fs/file-nr
1024 0 38001

On normal hours the number of file descriptors goes below 1024 but it never exceeds 1024. Users reporting on lost connections when I'm hitting the 1024 limit.

Why am I limited by 1024 file descriptors although the root user (erlang), www-data user (nginx) and open TCP connections are lower than 1024 each? Is there some global 1024 limit?

Do I need to use worker_rlimit_nofile with nginx or is it enough to increase the nofile in limits.conf for all users?

Thanks

More info that might be relevant:

> sudo nano /etc/security/limits.conf
...
root           hard    nofile          30000
root           soft    nofile          30000

> sudo sysctl fs.file-max
fs.file-max = 38001

> ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 4160
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4160
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Answer 1

Another user besides root is most likely hitting your file descriptor limit.

Try switching /etc/security/limits.conf to

...
*           hard    nofile          30000
*           soft    nofile          30000

That way every user has a upper limit of 30000.