Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Server applications running on linux often require large quantities of open file handlers, for ex. HBase ulimit, Hadoop epoll limit

This wiki entry should serve as documentary for Linux file limits configuration.

  • What is soft vs. hard limit?
  • How to control the hard limit?
  • How to control the soft limit?
  • Does kernel fs.file-max and user ulimit -n related?

Please describe the Linux distribution under which your configuration is valid as various vendors configure stuff differently.


Update Based on lstvan answer:

For people looking to automate this, at least on Ubuntu servers you can put this in your machine installation scripts:

echo 'fs.file-max = 65000' > /etc/sysctl.d/60-file-max.conf
echo '* soft nofile 65000' > /etc/security/limits.d/60-nofile-limit.conf
echo '* hard nofile 65000' >> /etc/security/limits.d/60-nofile-limit.conf
echo 'root soft nofile 65000' >> /etc/security/limits.d/60-nofile-limit.conf
echo 'root hard nofile 65000' >> /etc/security/limits.d/60-nofile-limit.conf
share|improve this question
    
Beware of the curly single quotes in the above snippet if cutting-and-pasting! –  Ville Jan 29 at 22:22
1  
sorry about that :) should be OK now. –  Maxim Veksler Jan 30 at 9:41
    
I think that should be 'root soft nofile 65000' 'root hard nofile 65000' .. right? :) –  Ville Jan 30 at 23:32
    
Yeah, fixed yet another time. Most bug contained post I’ve ever had :) Thanks. –  Maxim Veksler Feb 1 at 21:23

2 Answers 2

up vote 6 down vote accepted

Your operating system set limits on how many files can be opened by any running application on your host. You can extend the basic values usually 1024 easily by modifying 2 configuration files:

# vi /etc/sysctl.conf

fs.file-max = 32000

# vi /etc/security/limits.conf

youruser       soft    nofile   10000
youruser       hard    nofile   30000

The hard and soft limits:

man 5 limits.conf

hard
for enforcing hard resource limits. These limits are set by the superuser and 
enforced by the Kernel. The user cannot raise his requirement of system resources
above such values.

soft
for enforcing soft resource limits. These limits are ones that the user
can move up or down within the permitted range by any pre-exisiting hard 
limits. The values specified with this token can be thought of as default values,
for normal system usage.

HTH

share|improve this answer
    
So (just to clarify for future searchers:) fs-max is the global limit of open files. Meaning the if fs-max is set to 1000000 and we have 2 users, each of them with a limit of hard 1000000. Then even though they both have a hard limit of 1M they can't open 1M FD in the same time. Am I correct? (faqs.org/docs/securing/chap6sec72.html) –  Maxim Veksler Aug 2 '10 at 18:11

If you're increasing the file limits on Ubuntu, you may use the asterisks to indicate 'all users' for soft and hard file limits. However, this will not include the root user, and hence you have to define it separately if you want the increased limits apply to the root user as well (and so you can set different limits "for the root user", and "for everybody else"), like so:

* soft nofile 4096
* hard nofile 10240
root soft nofile 4096
root hard nofile 10240

It is perhaps cleanest to add these into a separate file, such as /etc/security/limits.d/60-nofile-limit.conf

share|improve this answer
    
It is generally considered a bad practice to run server application under the root user, but I’ve added your comment to the summary above, for completeness and saving lost root-based souls. Thank you. –  Maxim Veksler Jan 29 at 18:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.