Memory allocation and limits on debian 2.6.26

Question

On our server we have different users that need sometimes a big amount of memory (up to 40Gb). The total of our server is 128 Gb, but recently it crashed due to different users taking up together all memory. I couldn't even log in as a sysadmin, server was unresponsive.

For the moment, the only "solution" we found was limiting the amount of memory for each process. Alas it is then still possible to use up the complete 128 Gb, which will lead to another server crash.

Ideally, we want to reserve the last few Gb of the memory for the sysadmin, so that we can always get back on the server and kill some processes manually. In our department huge datasets are used, so we can't restrict too much.

Any idea how we can proceed on this?

Answer 1

Any idea how we can proceed on this?

You may find that some sort of virtualization solution is going to be the most effective way of limiting memory. In particular, the Linux Containers project is a very lightweight container mechanism that will let you apply hard resource limits with less overhead than a full virtualization solution.

You could conceivably put all non-admin users inside one container that has access to almost -- but not all -- of the available memory.