sched_setaffinity fails after I've used cset shield

Question

I am using cset shield to create a user shield with a set of cpus

 cset shield --cpu 1,3,5,7 --kthread on

The idea behind this is to reserve these cpus for my application code, and push all other tasks plus the movable kernel threads onto the other cpus.

In my application I create 4 threads, and in the context of each thread, attempt to use sched_setaffinity to pin each thread onto one of the reserved cpus.

int cpuNum = 1; // each thread gets one of the cpus
pid_t threadId = static_cast<pid_t>(syscall(SYS_gettid));

cpu_set_t cpuSet;
CPU_ZERO(&cpuSet);
CPU_SET(cpuNum, &cpuSet);

if (sched_setaffinity(threadId, sizeof(cpu_set_t), &cpuSet) == -1)
    perror("sched_setaffinity");

However, sched_setaffinity fails with Invalid argument

If I run my application using cset shield --exec ./TestApp then the pinning works

How come the naked sched_setaffinity call fails?

Answer 1

Your application code looks correct. Quickly, try this without --kthread on when you create your CPU shields. See if you get the same "Invalid argument" error. I'm wondering if there's an issue with the kernel threads.

Also, is this RHEL? What is the hardware setup (# of cores)? I've seen issues with the sched_setaffinity call on applications moved from Gentoo to Red Hat systems.