Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

The problem doesn't occur very often, but still it surely exists and I'm not sure where to start from. I have grepped for the mongrel PIDs in /var/log/ and the only messages that contained them are these:

Jun  7 07:46:24 staging kernel: 4gb seg fixup, process mongrel_rails (pid 29498), cs:ip 73:00937a5c

It has something to do with Xen specific version of libc, but it's not critical, and the processes are still running with these messages accumulating in kern.log

I'm actually looking not only for specific solution (which probably couldn't be provided from the above description) but for any advice on how to set up monitoring or investigate such cases.

share|improve this question

4 Answers 4

up vote 0 down vote accepted

Not exactly an answer, but must you use mongrel? I switched to apache + passenger and never looked back.

share|improve this answer
    
heh, well we did as well, doesn't seem like we're going to miss these mongrels either :) –  dolzenko Jan 7 '10 at 18:42

We use nagios to monitor our mongrels (along with hundreds of other services).

It just checks to ensure that there are mongrel processes running on each of the required ports. If not, it re-starts them.

share|improve this answer
    
Heh, everybody seem to be doing it, right. But sounds so 2001 :) –  dolzenko Jun 8 '09 at 17:06

I had these messages when libc6-xen was not installed in the xen domU. So verify you have that package installed...

apt-get install libc6-xen

When another variant of the libc is used, it will still work but it will be slower as the kernel has to catch the bad operation and do the right thing instead. The message quoted is generated by the kernel precisely in that situation.

So you guessed it right, that doesn't explain why mongrel stops. Check mongrel's documentation to enable debugging logs if it has any. Otherwise you can always try to strace the process until it fails... the end of the log will give you hints on how it fails, and maybe you'll find why.

share|improve this answer
    
I have * libc6 * libc6-dev * libc6-xen installed, I wonder which one gets picked up. Not sure about strace, because the output will be enormous I guess, as it takes time for mongrels to fail, but speaking of debug logs it's probably something I should do. Anyway IMO without normal server monitoring the whole affair doesn't make much sense (probably it just eats all memory and gets killed by Linux?) –  dolzenko Jun 9 '09 at 19:41
    
Ok, it now I know that the formatting isn't respected in the comment field, sorry about that. –  dolzenko Jun 9 '09 at 19:42
    
I don't know the details of your installation, but if libc6-xen is installed it should get automatically be pickep up. Does mongrel run in a separate chroot or does it use some LD_LIBRARY_PATH custom setting where another copy of the libc6 is found first? –  Raphaël Hertzog Jun 10 '09 at 6:39
    
"chroot/LD_LIBRARY_PATH" nothing like this, but ldd /usr/bin/ruby1.8 shows that libc.so.6 => /lib/libc.so.6 (0x00191000) which probably says that the wrong libc version is used anyway, google confirms that there are such issues, but I guess that belongs to another question then. Thank a lot for your help! –  dolzenko Jun 11 '09 at 10:53

You might look at god for monitoring and managing your mongrels. It's quite flexible and you can use it to restart based on certain thresholds such as amount of memory, CPU usage, flapping and more. You might also consider monit, which I know of people using to replace god.

share|improve this answer
    
Monit has worked well for us in the past. Now we use EngineYard to host our RoR projects and we don't have to worry about the issue anymore. –  Jared Brown Jul 22 '09 at 12:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.