Tag Info

Hot answers tagged

32

These limits came from a time where multiple "normal" users (not apps) would share the server, and we needed ways to protect them from using too many resources. They are very low for high performance servers and we generally set them to a very high number. (24k or so) If you need higher numbers, you also need to change the sysctl file-max option (generally ...


6

I tested this in Linux version 2.6.18-164.el5 - Red Hat 4.1.2-46. I could see that the ulimit is applied per process. The parameter is set at user level, but applied for each process. Eg: 1024 was the limit. Multiple processes were started and the files open by each one was counted using ls -l /proc/--$pid--/fd/ | wc -l There were no errors when the ...


6

Your operating system set limits on how many files can be opened by any running application on your host. You can extend the basic values usually 1024 easily by modifying 2 configuration files: # vi /etc/sysctl.conf fs.file-max = 32000 # vi /etc/security/limits.conf youruser soft nofile 10000 youruser hard nofile 30000 The hard ...


4

If the file descriptors are tcp sockets, etc, then you risk using up a large amount of memory for the socket buffers and other kernel objects; this memory is not going to be swappable. But otherwise, no, in principle there should be no problem. Consult the kernel documentation to try to work out how much kernel memory it will use, and/or test it. We run ...


4

On the process take a look at the lsof -p <pid> Notably you're looking for FD 13 i.e. from one of my httpd deployments. httpd 10865 apache 13w REG 8,5 1113187 59310687 /var/log/httpd/some_site.log I'd look at contention with obtaining an exclusive lock on the file, noted in FD 13 in this case, if other processes are contending for ...


3

Yes, the ulimit facility limits the number of open files and many other things like stack size, core dump size, etc.


3

In Linux, you can see if that option has changed the limits of the process by running: cat /proc/<PID>/limits Where <PID> is the process ID of nginex. Test on a development environment. If that option is not changing the limits or if you have a too restrictive hard limit you need to change the nofile in the /etc/security/limits.conf file.


3

The ulimit is for filehandles. It applies to files, directories, sockets, pipes epolls, eventfds, timerfds etc etc. At any point during the processes startup the limits might have been changed. Visit /proc/<pid>/limits and see if the values have been altered.


3

From the ulimit builtins man page The ulimit builtin provides control over the resources available to the shell and to processes started by it on systems that allow such control. Your lsof command lists all of the open files for all processes for all users on the system. You are not comparing like with like.


3

You could always just cat /proc/sys/fs/file-nr During the 'high load' situation to see how many file descriptors are in use. As to a maximum - it just depends on what you are doing.


2

I am not personally aware of any best practices. It's somewhat subjective depending on system function. Remember that 1024 you're seeing is a per-user limit and not a system-wide limit. Consider how many applications you run on this system. Is this the only one? Is the user that runs this application doing anything else? (IE do you have humans using this ...


2

You did not mention how many connections you have as a maximum when you start getting these errors. You can use tools/commands like: ps -ef | grep apache | wc -l lsof -p <apache pid> netstat -anp | grep 80 | grep -i ESTABLISHED | wc -l The first command tells you how many apache process on your system. The second command tells you how many opened ...


2

I'll answer my own question: the Linux 2.6 kernel sets the open file limit to 10% of the available memory. Source: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=fs/file_table.c 488 void __init files_init(unsigned long mempages) 489 { 490 int n; 491 492 filp_cachep = kmem_cache_create("filp", sizeof(struct ...


2

A bash(1) builtin, ulimit defines the maximum number of open files per process. This is not a system-wide setting in any way.


2

grep -lr pam_limits /etc/pam.d? If it's not return common-session, run the following command: echo -e "session\trequired\t\tpam_limits.so" > /etc/pam.d/common-session logging out and in again to see it works.


2

You have to add the ulimit line in the squid init script as well as increase max_filedescriptors in squid.conf. Those are the two essential steps. You no longer have to compile squid from source to increase this limit. That was something you had to do with really old squid versions.


2

I'd be surprised if your installation of Ubuntu didn't have by default more than 65536 available. Check your current setting with $ sudo sysctl fs.file-max The general rule of thumb is that you can increase the fs.file-max parameter by 64 for every 1MB of RAM. e.g. 2 gigabytes = 2048 * 1MB = 2048 * 64 = 131072 ...


2

It really depends on where the buffer is: if the application uses its own logging buffer then there's not way of forcing a flush. If the buffering is done by the C library then you can use LD_PRELOAD to disable the buffering. Assuming that the program is using fopen() to open its log file you can do something like this: #define _GNU_SOURCE 1 #include ...


1

2^20 (approximately one million) is the maximum file descriptor limit (NR_OPEN) that's supported under Linux. It would take many changes, including recompiling the kernel, to raise it. It would also break lots of user space applications that rely on the limit being low. I suspect you're completely barking up the wrong tree though. There's no correlation ...


1

The problem was in the launching script of daemon. It was using setuidgid to run the daemon under headless user.Looks like setuidgid will not install the resource limits which are set in limits.conf while changing user/group for the process. A daemon should take care of setting resource limits for itself via its launching scripts. By setting max FD limits ...


1

It seems like your reasoning is something like, "I have to lower that limit so I don't run out of precious descriptors". The truth is exactly the reverse -- if your server ran out of file descriptors, you need to raise that limit from 1,024 to something larger. For a realistic glassfish implementation, 32,768 is reasonable. Personally, I always raise the ...


1

If you have good memory and CPU's you can increase the no. of opened files limit via ulimit command ulimit -n 32667 (May be higher than this too)


1

I once got burned by something similar. One additional thing that you may want to check - that PHP is not sucking up the files either. If PHP is storing session information on-disk, a sudden overload of requests can suck up all the files too.


1

So it seems MongoDB 1.6.5 doesn't work with the limit nofile line. You need to put ulimit -n X (where X is the file descriptors you want) just after the script keyword. script ulimit -n 60000 PORT=27027 ... I guess the method using the pre-start block is for later versions of MongoDB. Also for note, MongoDB has a hard limit of ...


1

to see the top 20 file handle using processes: for x in `ps -eF| awk '{ print $2 }'`;do echo `ls /proc/$x/fd 2> /dev/null | wc -l` $x `cat /proc/$x/cmdline 2> /dev/null`;done | sort -n -r | head -n 20 the output is in the format file handle count, pid, cmndline for process example output 701 1216 /sbin/rsyslogd-n-c5 169 11835 postgres: spaceuser ...


1

What exactly are you trying to track down? The remote IP address(es) associated with the leaked FDs, the defective code, or something else? As you've already identified that there is a leak, contacting the engineers responsible for this java process seems like a reasonable next step.


1

This seems to me one of those questions best answered with "test it in a development environment". I remember years ago Sun got nervous when you messed with this, but not that nervous. It's limit at the time was also 1024, so I'm a little surprised to see that it's the same now for Linux, seems like it ought to be higher. I found the following link ...



Only top voted, non community-wiki answers of a minimum length are eligible