Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I've issue with apache. Everything begins about a week ago (so this may be related to latest system updates) - my monitoring tools SMS me about non-working website. I've checked it, and looks like request to trivial cgi (implemented in perl) took 30 seconds. After restarting apache problem was fixed for next few hours/days, and then happens again.

Initially I suspect php, because it was only update (5.3.15->5.3.18) which may affect apache. I've tried to disgrage it back to 5.3.15, but this won't fixed this issue.

Next, I did strace, and looks like delay happens while clone() syscall:

[pid 26659] 1356268557.675934 clone( <unfinished ...>
[pid 26659] 1356268587.684401 <... clone resumed> child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xa1033768) = 32169 <30.008452>

cgi itself runs after that and works ok and fast:

[pid 32169] 1356268587.686614 execve("/var/www/mysite.com/mycgi.cgi", ["/var/www/mysite.com/mycgi.cgi"], [/* 26 vars */]) = 0 <0.000776>

Any ideas why this may happens? How and why clone(2) may block for 30 seconds?! And successfully return after that… It looks like some timeout inside clone() because it always exactly 30 seconds.

I'm using up-to-date Hardened Gentoo Linux, kernel 3.5.4-hardened-r1, apache 2.2.23 (prefork, mod_cgi).

Update: I probably should add: this server isn't under high load. Apache childs use only about 120/40 MB (VIRT/RSS). Server has 2GB RAM, and only 400MB used (without buffers/cache).

share|improve this question
    
once it happens can you isolate the system and sniff both on default-route interface and loopback to see if there are no strange requests [eg forward/reverse dns, ldap/nis queries]? –  pQd Dec 23 '12 at 18:44
    
It's just happens again (I've rebooted system after posting this question just in case it's kernel bug which may gone after reboot). I can't really isolate this server, but I've checked traffic - as far as I can tell there is nothing unusual, and no activity which looks like relative to this 30 sec delay. –  Powerman Dec 23 '12 at 20:43
    
One more thing. When this issue begins, it doesn't affect all processes - initially only some apache childs got this 30 sec delay, but few minutes later all apache childs have this 30 sec delay. –  Powerman Dec 23 '12 at 22:56

1 Answer 1

up vote 1 down vote accepted

This happens because of CONFIG_GRKERNSEC_BRUTE=y kernel option.

share|improve this answer
    
How do you fix this? I think I'm having the same issue. –  Matt McCormick Jan 28 '13 at 7:38
    
Disable this option in kernel. –  Powerman Jan 28 '13 at 12:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.