Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am using cset shield to create a user shield with a set of cpus

 cset shield --cpu 1,3,5,7 --kthread on

The idea behind this is to reserve these cpus for my application code, and push all other tasks plus the movable kernel threads onto the other cpus.

In my application I create 4 threads, and in the context of each thread, attempt to use sched_setaffinity to pin each thread onto one of the reserved cpus.

int cpuNum = 1; // each thread gets one of the cpus
pid_t threadId = static_cast<pid_t>(syscall(SYS_gettid));

cpu_set_t cpuSet;
CPU_ZERO(&cpuSet);
CPU_SET(cpuNum, &cpuSet);

if (sched_setaffinity(threadId, sizeof(cpu_set_t), &cpuSet) == -1)
    perror("sched_setaffinity");

However, sched_setaffinity fails with Invalid argument

If I run my application using cset shield --exec ./TestApp then the pinning works

How come the naked sched_setaffinity call fails?

share|improve this question
    
Can you provide the operating system version and distribution? Kernel version may also be helpful. –  ewwhite Jun 21 '12 at 5:13
    
@ewwhite This is suse 11. Will get the rest of the info for you tomorrow. –  Steve Lorimer Jun 21 '12 at 11:15
    
Oh bummer... because RHEL had a bug tied to the sched_setaffinity interface for awhile. I'm not sure on SuSE. –  ewwhite Jun 21 '12 at 12:01
add comment

1 Answer

Your application code looks correct. Quickly, try this without --kthread on when you create your CPU shields. See if you get the same "Invalid argument" error. I'm wondering if there's an issue with the kernel threads.

Also, is this RHEL? What is the hardware setup (# of cores)? I've seen issues with the sched_setaffinity call on applications moved from Gentoo to Red Hat systems.

share|improve this answer
    
Sadly doesn't work without --kthread on either. –  Steve Lorimer Jun 21 '12 at 22:53
    
Hardware setup is: 2 sockets, 6 cores per socket, Intel(R) Xeon(R) CPU X5680 @ 3.33GHz, hyperthreading enabled (but I enforce that no hyperthread cores can be pinned to) socket 0: cpus: 1, 3, 5, 7, 9, 11; hyperthread_cpus: 13, 15, 17, 19, 21, 23; socket 1: cpus: 0, 2, 4, 6, 8, 10; hyperthread_cpus: 12, 14, 16, 18, 20, 22 –  Steve Lorimer Jun 21 '12 at 22:56
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.