Getting Too many Open files error for Postgres

Question

In Ubuntu 14.04 server on a JAVA application we are getting Too many Open files error for Postgres (Using Postgresql 9.5).

We set the following in /etc/security/limits.conf

* soft nofile 16384
* hard nofile 16384
root soft nofile 16384
root hard  nofile 16384
postgres soft nofile 16384
postgres hard  nofile 16384

Also set the following in /etc/sysctl.conf

kern.maxfiles=20480
kern.maxfilesperproc=18000

Also please see the following results while running as Postgres user:

-> ulimit -Hn
16384

-> ulimit -Sn
16384


-> cat /proc/sys/fs/file-max
100268

After restarting the server and checking the ulimit for Postgres its 100268. But while checking the limit for opened files under postgres process its still 1024 and 4096

# cat /proc/1072/limits

Max open files            1024                 4096                 files

When we restart postgres services it got changed to

#cat /proc/1759/limits
Max open files            16384                16384                files

But it seems its not affected because still we are getting "Too many Open files" error.

Also in Server the directories /etc/security/limits.d/ and /etc/security/conf.d/ and empty. So can anyone please guide me.

Answer 1

@dilyin Thanks for the update.

We updated ulimit value in startup script of PostgreSQL but still the issue was there.

Finally we fixed the issue by reducing max_files_per_process to 200 from default 1000. This parameter is in postgresql.conf file and this sets the maximum number of simultaneously open files allowed to each server subprocess.

Answer 2

Ah... The known problem.

Usually people start to edit "/etc/security/limits.conf" file, but forget that this file only apply to the actively logged in users through the pam system.

If you start the database manually using the init script the database proces will inherit your modified limits, but if the database is started at boot or started by something like systemd it will not.

There are files "/etc/defaults/$service for Debian and "/etc/sysconfig/$service for RedHat. These files are sourced by the init script before the daemon is run. Add ulimit -s unlimited or something like this into these files. The limits will be applied in the init script shell and will affect the database process.

Kernel maxfiles should be set as well.