2

I'm managing a PBS/torque HPC cluster, and now I'm setting up another cluster with SLURM. On the PBS cluster, I can set a queue to accept only interactive jobs by qmgr -c "set queue interactive_q disallowed_types = batch" and to accept only batch jobs by qmgr -c "set queue cu disallowed_types = interactive".

However, I cannot find the equivalent setups with SLURM after browsing through the official SLURM documents.

How can I set up a partition on SLURM cluster to only accept interactive or batch jobs?

1 Answer 1

3

As of Slurm 20, there isn't any direct way to separate interactive jobs from batch jobs in a partition.

However, it is possible to use a job submit plugin which can distinguish the two in several ways; batch jobs have a job script (job_desc.script in Lua) associated with them where interactive jobs don't. Some interactive jobs have a PTY. I think there also may be a flag set that is new in Slurm 20.

Once the job submit plugin has identified a job as interactive or batch, it can either reject it for the given partition, or it can possibly change the partition it is being sent to, or it could modify the job in some other appropriate way (like assigning an "interactive" license to it or change the QOS) before final submission.

If you modify the job, then additional Slurm configuration can affect the job based on that.

4
  • This is indeed a good idea. Job submitted by sbatch will have its field BatchFlag set to 1. This is great, but according to github.com/SchedMD/slurm/blob/master/src/plugins/job_submit/lua/…, it does not seem that the lua plugin can access BatchFlag field. Writing C plugin is too troublesome, as the compilation of SLURM is done by the vendor, not me. Right now, I can only tell the job is interactive if job_desc.argv[job_desc.argc-1]=="bash". This is certainly not sufficient.
    – wdg
    Jan 21 at 9:17
  • 1
    I wrote my lua job submit plugin for slurm 19 or earlier, and checking for the script works fine, so the batchflag field is redundant anyway. Maybe they'll add it to a future version of the lua plugin. Look at job_desc.script
    – user10489
    Jan 21 at 12:25
  • Did you mean that for interactive jobs, job_desc.script is emtpy, but for batch jobs, they are not empty?
    – wdg
    Jan 21 at 14:28
  • 1
    Right. Interactive jobs have no script and job_desc.script will be empty / not set.
    – user10489
    Jan 22 at 0:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.