3

Problem

Got java.io.IOException: Too many open files in solr indexing through jenkins.

Did some googling and found we have to set the ulimit for the box in where we are running the job.

So set the ulimit in a linux box with spec

Linux x86_64 GNU/Linux

in both of the following fashions

  1. ulimit -n 1000000

  2. /etc/security/limits.conf

    userx soft nofile 1000000
    userx hard nofile 1000000
    

Given

  1. userx is the user through which the jenkins job is being executed.
  2. when doing ssh to the box as userx manually through terminal and check ulimit -n am getting 10000000

Question

But when executing the same ulimit -n through a jenkins job, only getting 1024 which is the default.

Any advice would be much helpful?

1
  • 2
    check the init scripts for Jenkins to ensure they're not resetting the ulimit. Also, ensure Jenkins has been restarted since making the changes to limits.conf Oct 8, 2012 at 20:42

2 Answers 2

4

Make sure pam_limits.so is enabled for su if your jenkins startup script is using "su" for user switching.

For Ubuntu Server, uncomment "session required pam_limits.so" line in /etc/pam.d/su file

sudo sed -i '/# session\s\+required\s\+pam_limits.so/ s/# *//' /etc/pam.d/su
1
  • Thanks @FlareCoder, my problem was bit amateur, please see my own answer. This will be definitely helpful for someone.
    – raksja
    Nov 14, 2012 at 22:37
2

By default Jenkins establish a SSH connection with each of the Node/Slave as userx.

I set the ulimit, took the node offline and bring it back and I thought the problem is on the session side.

So Disconnected the whole node from the jenkins and restarted the connection with it again. It brought back a new session for userx and it obviously worked.

My bad that I didn't look through the logs when I took the node offline.

Remember: Taking a jenkins node offline and bringing it back won't bring up a new SSH session.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .