3

I would like to be able to limit an installed binary to only be able to use up to a certain amount of RAM. I don't want it to get killed if it exceeds it, only that that would be the max amount that it could use. I would like the process to die once it reaches a certain amount of RAM, preferably before the server starts to swap heavily.

The problem I am facing is that I am running an Apache 2.2 server with PHP and some custom code that a developer is writing for us. The problem is that somewhere in there code they launch a PHP exec call that launches ImageMagick's 'convert' to create a resized image file.

I'm not privy to a lot of details to the project or the code, but need to find a solution to keep them from killing the server until they can find a way to optimize the code.

I had thought that I could do this with /etc/security/limits.conf and setting a limit on the apache user, but it seems to have no effect. This is what I used:

www-data hard as 500

If I understand it correctly, this should have limited any apache user process to a maximum to 500kb, however, when I ran a test script that would chew up a lot of RAM, this actually got up to 1.5GB before I killed it. Here is the output of 'ps auxf' after the setting change and a system reboot:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      5268  0.0  0.0 401072 10264 ?        Ss   15:28   0:00 /usr/sbin/apache2 -k start
www-data  5274  0.0  0.0 402468  9484 ?        S    15:28   0:00  \_ /usr/sbin/apache2 -k start
www-data  5285  102  9.4 1633500 1503452 ?     Rl   15:29   0:58  |   \_ /usr/bin/convert ../tours/28786/.….
www-data  5275  0.0  0.0 401072  5812 ?        S    15:28   0:00  \_ /usr/sbin/apache2 -k start

Next I thought I could do it with Apache's RlimitMEM setting, but get the same result of it not getting limited. Here is what I have in my apache.conf file:

RLimitMEM 500000 512000

It wasn't until many hours later that I figured out that if the process actually reached that amount that it would die with an OOM error.

Would love any ideas on how to set this limit so other things could function on the server, and all of them could play together nicely.

5
  • 3
    If it is not to be killed? then what? If the process tries to allocate some more memory and has already reached the limit? What should happen? Allocation failure? That would be equivalent to killing it as most programs abort if they fail to allocate some memory.
    – sch
    Oct 1, 2012 at 21:36
  • 2
    Beat the developers over the head with this. Calling exec() to run ImageMagick to resize an image smacks of incompetence. Oct 2, 2012 at 1:39
  • What Michael said. The ImageMagick libraries and documentation are readily available, as well as code examples in all major languages. The devs have no excuse for not doing it properly. Oct 2, 2012 at 6:27
  • @MichaelHampton: yea, not real happy about how they are doing it, but i'm only the guy who is trying to keep the server running and keep his pager from going off in the middle of the night.
    – Ken S.
    Oct 2, 2012 at 18:16
  • Maybe I am thinking about this wrong. I think i do want the process to die if it starts using too much memory. I would rather the process die than the server. From the above info about limits.conf and RLimitMEM, can anyone see what I am doing wrong with those?
    – Ken S.
    Oct 2, 2012 at 18:20

4 Answers 4

3

Create a new user, just for ImageMagick. Give it the memory limits you want. Set up the sudoers file in a way, that Apache may call ImageMagick as this user. Change the exec() call in your PHP code to run convert via sudo or replace the original convert binary with a wrapper script. If ImageMagick is eating all the memory, let it die and let the PHP script throw an appropriate error messages.

Or replace ImageMagick all together, it's a sheer memory hog (try stichting tiles of little pictures together. At one point you will remember why you are still having a swap partition).

1
  • Agreed. It's hard to set a ulimit on www-data as that can have an impact on all Apache processes rather than just the instances of ImageMagick that are going out of control.
    – Matt
    Oct 2, 2012 at 19:00
3

There is a rather new (2.6.24) kernel interface called cgroups that allows you to do this. You can put a user, group or PID(with children) under a cgroup, and give it X amount of RAM, CPU, IO (disk, network) etc. It's rather neat, and extremely powerful when used with KVM virtual machines. See wikipedia and the official docs for more information.

1

The simple answer is: You cannot. If the program cannot allocate more memory, it will crash. You can keep the server memory in certain borders by setting the workers and everything but this will limit the performance.

If you need more oxigen and cannot have more you will die too. There is no other outcome.

0

If Rlimitmeme option didn't work out then there seems to be little chance to get it in limit. Can you show me the OOM killer stack traces in a pastebin at least so that I can see which zones are getting depleted or how allocation is failing. Not a solution to the problem you have but some more usual debugging that I would rather do.

However, I really don't think of a way to limit a binary to use a certain amount of RAM. That's hardly possible, I guess.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .