0

The problem doesn't occur very often, but still it surely exists and I'm not sure where to start from. I have grepped for the mongrel PIDs in /var/log/ and the only messages that contained them are these:

Jun  7 07:46:24 staging kernel: 4gb seg fixup, process mongrel_rails (pid 29498), cs:ip 73:00937a5c

It has something to do with Xen specific version of libc, but it's not critical, and the processes are still running with these messages accumulating in kern.log

I'm actually looking not only for specific solution (which probably couldn't be provided from the above description) but for any advice on how to set up monitoring or investigate such cases.

Improve this question

4 Answers 4

Reset to default
0

Not exactly an answer, but must you use mongrel? I switched to apache + passenger and never looked back.

Improve this answer
1
  • heh, well we did as well, doesn't seem like we're going to miss these mongrels either :)
    – Ev Dolzhenko
    Jan 7, 2010 at 18:42
0

We use nagios to monitor our mongrels (along with hundreds of other services).

It just checks to ensure that there are mongrel processes running on each of the required ports. If not, it re-starts them.

Improve this answer
1
  • Heh, everybody seem to be doing it, right. But sounds so 2001 :)
    – Ev Dolzhenko
    Jun 8, 2009 at 17:06
0

I had these messages when libc6-xen was not installed in the xen domU. So verify you have that package installed...

apt-get install libc6-xen

When another variant of the libc is used, it will still work but it will be slower as the kernel has to catch the bad operation and do the right thing instead. The message quoted is generated by the kernel precisely in that situation.

So you guessed it right, that doesn't explain why mongrel stops. Check mongrel's documentation to enable debugging logs if it has any. Otherwise you can always try to strace the process until it fails... the end of the log will give you hints on how it fails, and maybe you'll find why.

Improve this answer
4
  • I have * libc6 * libc6-dev * libc6-xen installed, I wonder which one gets picked up. Not sure about strace, because the output will be enormous I guess, as it takes time for mongrels to fail, but speaking of debug logs it's probably something I should do. Anyway IMO without normal server monitoring the whole affair doesn't make much sense (probably it just eats all memory and gets killed by Linux?)
    – Ev Dolzhenko
    Jun 9, 2009 at 19:41
  • Ok, it now I know that the formatting isn't respected in the comment field, sorry about that.
    – Ev Dolzhenko
    Jun 9, 2009 at 19:42
  • I don't know the details of your installation, but if libc6-xen is installed it should get automatically be pickep up. Does mongrel run in a separate chroot or does it use some LD_LIBRARY_PATH custom setting where another copy of the libc6 is found first?
    – Raphaël Hertzog
    Jun 10, 2009 at 6:39
  • "chroot/LD_LIBRARY_PATH" nothing like this, but ldd /usr/bin/ruby1.8 shows that libc.so.6 => /lib/libc.so.6 (0x00191000) which probably says that the wrong libc version is used anyway, google confirms that there are such issues, but I guess that belongs to another question then. Thank a lot for your help!
    – Ev Dolzhenko
    Jun 11, 2009 at 10:53
0

You might look at god for monitoring and managing your mongrels. It's quite flexible and you can use it to restart based on certain thresholds such as amount of memory, CPU usage, flapping and more. You might also consider monit, which I know of people using to replace god.

Improve this answer
1
  • Monit has worked well for us in the past. Now we use EngineYard to host our RoR projects and we don't have to worry about the issue anymore.
    – Jared Brown
    Jul 22, 2009 at 12:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .