Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a problem about balancing irq. I took irq numbers from /etc/interrupts and assigned each irq to each of 24 CPUs as this tutorial says

for i in {143..166};do cat /proc/irq/${i}/smp_affinity; done
000001                                                                                                                                   
000002                                                                                                                                   
000004                                                                                                                                   
000008                                                                                                                                   
000010                                                                                                                                   
000020                                                                                                                                   
000040                                                                                                                                   
000080                                                                                                                                   
000100
000200
000400
000800
001000
002000
004000
008000
010000
020000
040000
080000
100000
200000
400000
800000

After that we tests server by sending about 10 million pps and observe output of top command.

top - 15:19:06 up  5:02,  8 users,  load average: 12.51, 12.86, 12.38
Tasks: 278 total,  17 running, 261 sleeping,   0 stopped,   0 zombie
%Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,  0.7 id,  0.0 wa, 99.3 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,  1.0 id,  0.0 wa, 99.0 hi,  0.0 si,  0.0 st
%Cpu2  :  0.0 us,  0.0 sy,  0.0 ni,  0.7 id,  0.0 wa, 99.3 hi,  0.0 si,  0.0 st
%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,  1.0 id,  0.0 wa, 99.0 hi,  0.0 si,  0.0 st
%Cpu4  :  0.0 us,  0.0 sy,  0.0 ni,  4.0 id,  0.0 wa, 96.0 hi,  0.0 si,  0.0 st
%Cpu5  :  0.0 us,  0.0 sy,  0.0 ni,  4.7 id,  0.0 wa, 95.3 hi,  0.0 si,  0.0 st
%Cpu6  :  0.0 us,  0.0 sy,  0.0 ni,  5.3 id,  0.0 wa, 94.7 hi,  0.0 si,  0.0 st
%Cpu7  :  0.0 us,  0.0 sy,  0.0 ni,  6.0 id,  0.0 wa, 94.0 hi,  0.0 si,  0.0 st
%Cpu8  :  0.0 us,  0.3 sy,  0.0 ni,  5.0 id,  0.0 wa, 94.7 hi,  0.0 si,  0.0 st
%Cpu9  :  0.0 us,  0.3 sy,  0.0 ni,  5.6 id,  0.0 wa, 94.1 hi,  0.0 si,  0.0 st
%Cpu10 :  0.0 us,  0.3 sy,  0.0 ni,  5.3 id,  0.0 wa, 94.4 hi,  0.0 si,  0.0 st
%Cpu11 :  0.0 us,  0.0 sy,  0.0 ni,  6.0 id,  0.0 wa, 94.0 hi,  0.0 si,  0.0 st
%Cpu12 :  0.0 us,  0.0 sy,  0.0 ni,  1.0 id,  0.0 wa, 99.0 hi,  0.0 si,  0.0 st
%Cpu13 :  0.0 us,  0.0 sy,  0.0 ni,  1.0 id,  0.0 wa, 99.0 hi,  0.0 si,  0.0 st
%Cpu14 :  0.0 us,  0.3 sy,  0.0 ni,  1.0 id,  0.0 wa, 98.7 hi,  0.0 si,  0.0 st
%Cpu15 :  0.0 us,  0.0 sy,  0.0 ni,  0.7 id,  0.0 wa, 99.3 hi,  0.0 si,  0.0 st
%Cpu16 :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu17 :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu18 :  0.3 us,  0.7 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu19 :  1.3 us,  0.3 sy,  0.0 ni, 98.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu20 :  0.3 us,  0.3 sy,  0.0 ni, 99.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu21 :  0.0 us,  0.3 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu22 :  0.7 us,  0.3 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu23 :  0.3 us,  0.3 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.3 hi,  0.0 si,  0.0 st
KiB Mem:  65947892 total,  1055364 used, 64892528 free,    24544 buffers
KiB Swap: 50321404 total,        0 used, 50321404 free,   348920 cached

My question is why only 16 of 24 CPU loaded? Is this normal? How can I balance irq to all CPUs?

share|improve this question
    
Why not just use irqbalance? –  ewwhite Apr 24 at 12:27
    
I also tried it. Result is same. –  ibrahim Apr 24 at 12:28
    
it would be helpful to see the change in /proc/interrupts over a period where you are applying the load - not all interrupts fire at the same frequency. –  symcbean Apr 24 at 15:44
    
in /proc/interrupts output, interrupt counts of CPU0-15 are too much and growing rapidly compare to CPU16-23 –  ibrahim Apr 25 at 5:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.