3

I use the isolcpus kernel parameter to restrict the CPUs the Linux scheduler should use to place processes.

cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-4.2.0-18-generic.efi.signed root=UUID=78e5b46f-6c1c-4e8f-b921-8c091c98209b ro isolcpus=4−23,28−47

From what I have read, this should place all processes where not explicitly specified by cpuset or similar on the cpus that are not in the isolcpus list.

However, the processes run on arbitrary CPUs, the supposedly isolated CPUs are still used. Here is a part if the output of ps -F ax:

UID        PID  PPID  C    SZ   RSS PSR STIME TTY      STAT   TIME CMD
message+  1185     1  0 10712  3804  11 20:36 ?        Ss     0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
root      1208     1  0  4411  5364  20 20:36 ?        S<L    0:00 /usr/bin/atop -a -w /var/log/atop/atop_20151114 600
root      1214     1  0  7135  3008   2 20:36 ?        Ss     0:00 /lib/systemd/systemd-logind
daemon    1221     1  0  4795  1912  42 20:36 ?        Ss     0:00 /usr/sbin/atd -f

As you can see in the PSR column, CPUs 11,20, and 42 are used, which should not be the case.

I have tried the following Ubuntu Versions so far with the result described above in all cases:

  • Ubuntu 14.04 Server
  • Ubuntu 15.04 Server, with Ubuntu 3.19 kernel as well as mainline kernel
  • Ubuntu 15.10 Server

Any ideas what might be the problem?

Update

Isolcpus does work when running Ubuntu in a VM without NUMA (Core i7). However, it still does not work on a server with NUMA (2x Xeon E3 v3 2690).

  • Chatter on the Internet indicates that they broke it. Are you absolutely 100% certain you need to use Ubuntu? – Michael Hampton Nov 14 '15 at 22:16
  • Well, we settled with Ubuntu for now. Are there alternatives that are known to work? Debian? – jgjl Nov 15 '15 at 15:12
  • Not finding a launchpad bug on the subject. If the kteam doesn't know its broken it can't fix it. – ppetraki Nov 15 '15 at 18:26
  • @ttry56. Is your application so sensitive that "everything" must be perfectly balanced or is this just in additional optimization that's nice to have? The last storage appliance we did, once the storage adapters were lined up with iscsi targets using cpuset, it really didn't mater where anything else went because they dominated the available resources on those numa nodes. Unless you got the performance numbers to back it up, you might be beating yourself up over nothing. Just trying to save you some headaches and unnecessary work. – ppetraki Nov 15 '15 at 18:46
  • Maybe a Debian kernel would work, but I'm surprised that you would seriously consider a Debian based distribution for this. They do not target enterprise use and a large number of design decisions over the years have made Debian particularly ill-suited overall for the enterprise. Ubuntu, of course, inherits most of those design issues. – Michael Hampton Nov 15 '15 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.