1

I want to be able to set up memory requirements per job.

For instance: to run 5 jobs, for which I know each will need 4 GB memory. I have 16 GB RAM on the Ubuntu server and 16 GB swap. I want to avoid using the swap. Can I do something like:

qsub -l mem_required_for_my_job=4G job1
qsub -l mem_required_for_my_job=4G job2
qsub -l mem_required_for_my_job=4G job3
qsub -l mem_required_for_my_job=4G job4
qsub -l mem_required_for_my_job=4G job5

? The jobs will require 4 GB at some moment, but not in the beginning.

How to tell SGE what my requirements are? How to avoid scheduling 5 x 4 GB when only 16 GB available?

I read the user guide and tried s_vmem, h_vmem, mem_free, mem_used. None of them is what I want. I do not want my jobs to get killed in the middle of the processing. I want them not to be scheduled, unless the maximum resources needed are available.

Can I do this?

Thank you all!

2 Answers 2

2

On our cluster we use h_vmem to enforce job memory allocation.

The thing you appear to be missing is setting the available amount as a consumable resource. In qconf -mc or in the qmon complexes dialog you need to set the resource as requestable and consumable.

Then, on each host with qconf -me you need to set the amount of available memory in complex_values.

For example we have host definitions that look like:

hostname              node004
load_scaling          NONE
complex_values        h_vmem=30.9G,exclusive=1,version=3.3.0
8
  • I tried with h_vmem, virtual_free and quotas. None of them carry the task in the way I want it. h_vmem protects the system, killing the jobs if they overcome their requirements. Both quota and virtual_free didn't do anything to prevent the memory exhausting. The OS killed the job in this case. I want to avoid killing the jobs. I would like to make them stay in the queue, waiting for their requested resources to be available. Is this possible?
    – grs
    Mar 2, 2011 at 22:28
  • I'm not sure what you mean. With the h_vmem limit, GE will only run the job on the node if it has enough of the h_vmem resource available to run it. It also sets the ulimit on virtual memory. If the job tries to allocate more than the limit, it will be killed by the OS. Mar 3, 2011 at 1:06
  • 1
    GridEngine will run exactly as many jobs as match the available complex resources. So if you define a node to have 16GB of h_vmem available, and you submit 5 jobs requesting 4GB, GridEngine will only place 4 of them on the node at once. Your job should always request the peak amount it expects to use. If you have swap space on your nodes, you can include that amount in the h_vmem complex value. Mar 3, 2011 at 2:22
  • 1
    The arguments are the same as the name of the complex. So if you have h_vmem configured as a consumable resource and under a host's complex_values, your command would look like: qsub -l h_vmem=4G job1. Mar 3, 2011 at 6:21
  • 2
    Also here's a quick blog post from gridengine.info regarding memory limits: gridengine.info/2009/12/01/… Mar 3, 2011 at 6:22
0

I think the what you are looking for is virtual_free

"If a job request for virtual_free exceeds the available amount, the job is not dispatched to a queue on that host." from https://web.archive.org/web/20100801220839/http://wikis.sun.com/display/GridEngine/Configuring+Complex+Resource+Attributes

See also http://markmail.org/message/2iw4esthqvs5rc2a

1
  • Please check the comment on the other answer.
    – grs
    Mar 2, 2011 at 22:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .