Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We recently set up an HP ProLiant DL360p server for some heavy grunt-work. For one reason or another we disabled hyperthreading in the system BIOS when setting it up. Being a dual-CPU 8-core system, that gave us 16 hardware threads. 32 GB RAM. We are running Ubuntu 12.04 64-bit.

The bulk of the work done is by a VHDL "synthesizer", or compiler. It is a QT application, but typically run in command-line mode (no GUI). This compiler has been running well for the last few weeks, kicked off by a continuous integration system (Jenkins).

Today we re-enabled hyperthreading to get access to all 32 hardware threads. However now this compiler hangs with the following error in every case I can contrive:

QThread::start: Thread creation error: Resource temporarily unavailable

The process appears to be stalled, not busy on any CPU, and ctrl-c interrupts it.

I did some searching online and it would appear that this may be related to OS maximum thread limits, but I'm not sure how to go about changing this. The default is supposedly around 800 threads anyway, which should be more than enough for this compiler that runs only a small number (maybe 2?).

In the meantime I will have to disable hyperthreading, but I was wondering if this is a known issue with high-end servers running 64-bit Linux? Is there a known workaround? Or is it more likely a problem with that specific application?

share|improve this question
    
Turning off hyper-threading fixes the issue. But that's really only a workaround for now. –  meowsqueak Jul 3 '12 at 23:10

1 Answer 1

Could you describe how this process is started, and provide us with the output of ulimit -a from a context as close from that as possible?

As pthread_create(3) explains, you are running into:

ERRORS
       EAGAIN Insufficient  resources  to create another thread, or a system-
              imposed limit on the number of threads  was  encountered.   The
              latter  case  may  occur  in  two  ways:  the RLIMIT_NPROC soft
              resource limit (set via setrlimit(2)), which limits the  number
              of  process  for  a  real user ID, was reached; or the kernel's
              system-wide limit on  the  number  of  threads,  /proc/sys/ker‐
              nel/threads-max, was reached.
share|improve this answer
    
I can tell you (with HT currently disabled) that the binary is executed from a sh script (sets up environment) and a ulimit -a just before invoking the binary gives: time(seconds) unlimited file(blocks) unlimited data(kbytes) unlimited stack(kbytes) 131072 coredump(blocks) 0 memory(kbytes) unlimited locked memory(kbytes) 64 process 256966 nofiles 1024 vmemory(kbytes) unlimited locks unlimited . Command is /opt/diamond/1.4/synpbase/linux/mbin/synbatch -product synplify_pro -rxl isIWgMKg -batch build.tcl –  meowsqueak Jul 6 '12 at 2:20
    
OK, then it looks like a kernel bug. Can we have uname -a? –  Pierre Carrier Jul 6 '12 at 8:51
    
Linux build01 3.2.0-26-generic #41-Ubuntu SMP Thu Jun 14 17:49:24 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux –  meowsqueak Jul 8 '12 at 21:26
    
Is this bug documented anywhere else? –  meowsqueak Aug 23 '12 at 1:02
1  
Have you tried upgrading to a newer Ubuntu kernel (i.e. 3.5.0 from 12.10) or building your own? It does sound like a kernel/RCU bug! –  Panagiotis PJ Papadomitsos Nov 25 '12 at 17:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.