Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Currently I am seeing a lot of CPU usage on events/1 and I was wanting to know how to figure out what is causing that?

cat /proc/interrupts
            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7       CPU8       CPU9       CPU10      CPU11      CPU12      CPU13      CPU14      CPU15
   0:  575075290          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-edge      timer
   1:          5          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-edge      i8042
   8:          1          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-edge      rtc0
   9:          1          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   acpi
  18:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   ata_piix
  19:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   ata_piix
  20:         27          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb1
  21:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb5, uhci_hcd:usb8
  22:         48          0          0          0        203          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb4, uhci_hcd:usb7
  23:          2          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb3, uhci_hcd:usb6
  32:       7511          0          0          0          0          0          0     169991          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   megasas
  49:          1          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      eth0
  50:      14013      79342          0          0      13699       5817          0          0     916084          0          0          0      43447          0          0          0   PCI-MSI-edge      eth0-TxRx-0
  51:         21          0          0          0          0          0          0          0          0     491153          0          0          0          0          0          0   PCI-MSI-edge      eth0-TxRx-1
  52:         16          0          0          0          0          0          0          0          0          0          0          0          0     490363          0          0   PCI-MSI-edge      eth0-TxRx-2
  53:         15          0          0          0          0          0          0          0          0          0     512295          0          0          0          0          0   PCI-MSI-edge      eth0-TxRx-3
  54:         14          0          0          0          0          0          0          0          0          0          0          0          0          0    2137545          0   PCI-MSI-edge      eth0-TxRx-4
  55:         14          0          0          0          0          0          0          0          0          0          0     472322          0          0          0          0   PCI-MSI-edge      eth0-TxRx-5
  56:         14          0          0          0          0          0          0          0          0          0          0          0          0          0          0    1261400   PCI-MSI-edge      eth0-TxRx-6
  57:      46039     101974      16307          0     177472      22446          0          0      78146          0          0          0       3060          0          0          0   PCI-MSI-edge      eth0-TxRx-7
 NMI:     116990     104030      89718      76478      57282      42770      27106       7229      11177      12608      16074      17471      15123      17563      17220       9457   Non-maskable interrupts
 LOC:    1240959  513359079  608524106  453845650  545193366  480747439  402785555  456482461  620998409  526207907  405289993  406272537  426647321  459716091  532029492  578607757   Local timer interrupts
 SPU:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   Spurious interrupts
 PMI:     116990     104030      89718      76478      57282      42770      27106       7229      11177      12608      16074      17471      15123      17563      17220       9457   Performance monitoring interrupts
 IWI:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   IRQ work interrupts
 RES:       3518       3786       1864        535       5509       4985       1982       2083       2640       1854       1547       1067       2261       2259       1744       1742   Rescheduling interrupts
 CAL:      57062        229        228        228       3541        227        228        225        222        210        222        226        212        212        224        217   Function call interrupts
 TLB:      10184       9632       3623       3081      15017      12586       3242       2803       7624      33023       5225       4085       4565      45383       7271       4827   TLB shootdowns
 TRM:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   Thermal event interrupts
 THR:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   Threshold APIC interrupts
 MCE:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   Machine check exceptions
 MCP:       1185       1185       1185       1185       1185       1185       1185       1185       1185       1185       1185       1185       1185       1185       1185       1185   Machine check polls
 ERR:          0
 MIS:          0

Could this be a hardware issue?

EDIT:

ps auxf

http://pastebin.com/fF4zamew

share|improve this question
    
events/1 is a kernel thread. Post the ps auxf output first. interrupts comes way later in this kind of debugging process. –  Soham Chakraborty Dec 24 '13 at 4:22
    
Thanks. Added to post. –  Frank Thornton Dec 24 '13 at 4:38

1 Answer 1

up vote 0 down vote accepted

Okay, thanks for the added information. So, events/1 is the problem. This events/ mean events/CPU_NO. The events threadds were introduced as a means of the kernel to wake up and do work which replaced keventd. However, in the v3.9-rc8 git tree, I still see keventd and events/per_cpu both.

It is my assumption that irq 57 is hitting up CPU 1. Otherwise, in /proc/interrupts, I don't see anything unusual. The workload is pretty much evened out. Can you see what irq is 57.

Also, I honestly don't think we would be able to debug this problem using traditional methods of top, ps and all. In all subtlety, this will require a perf analysis. Let me know if you can get that. And if you do get that, you need to mail me that because you can put that much data here.

https://perf.wiki.kernel.org/index.php/Main_Page

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.