2

On CentOS 6.5, du -sk /home and sh -c (du -sk \/home) 2>/dev/null run because a scheduled disk usage for /home, but du is causing 100% CPU usage. Plus, before it's even finished, another du gets run and it piles up, bringing the system down to its knees.

I've since disabled the automated checks, but I can't run du -sk /home manually either. du runs fine (fast) on other directories (though /home is the largest - 4912620 - it shouldn't take as long as it does)

I would like to find out why is DU causing high CPU usage on /home


UPDATE 1: A full restart (host+guest) solved the problem.

UPDATE 2: Problem came back. While du is running, CPU usage is split nearly half and half between (us) and (sy). (wa) never creeps over 1%. Also, IOWAIT shows only disk writes?

5
  • It's likely to be iowait. Try running top or vmstat or iotop while doing du.
    – user130370
    Dec 28, 2013 at 17:32
  • 1
    @EricDANNIELOU: surely IO wait would cause the du to relinquish the cpu ?
    – user9517
    Dec 28, 2013 at 17:36
  • @lain I'm not sure about what you mean. 100% cpu can be caused by iowait (wa), userland (us) or kernel (sy).
    – user130370
    Dec 28, 2013 at 17:40
  • Iowait can definitely cause the CPU load to creep up. If the disks are slow, particularly network disks, then this is more common.
    – jaseeey
    Jan 7, 2014 at 11:45
  • Disk was an LVM volume for the guest KVM OS. OpCenter crew ran tests and found no physical problems with the disk subsystem.
    – Gaia
    Jan 7, 2014 at 19:42

2 Answers 2

1

I can't tell you why exactly, but here's what I'd do to narrow down what is going on:

cd /home
for i in * ; do echo "Starting in ${i}..." ; du -s ${i} ; done

This will run "du -s" for each command inside the /home directory, and show you what it is about to do. This is also assuming you are using sh or bash as your shell.

When you find which directory is causing the problem, move in one directory more, and see what happens there. Also remember this will not check for hidden ("dot") directories at the level you start the for loop from. You may need to check those as well, or modify the glob expression to include them.

10
  • There seem to be no abnormal delays or CPU spikes when I run the above command. If I run it for only /home with --max-depth=1 I also don't get a delay/spike. It's only when I run du -sk /home.
    – Gaia
    Jan 26, 2014 at 16:43
  • Do you have any .dot directories in /home? Jan 27, 2014 at 21:24
  • Directories named ".dot" ?
    – Gaia
    Jan 28, 2014 at 17:48
  • No, hidden files or directories inside /home Jan 28, 2014 at 17:54
  • yes, .htaccess, etc
    – Gaia
    Jan 28, 2014 at 19:20
0

I finally discovered what the problem was when I came across this post.

root> modprobe vhost_net root> echo vhost_net > /etc/modules

For some reason, the virtio interfaces were not loading the driver before. All is good now.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .