What is causing these flush processes?

Question

What are these flush processes?

$ ps aux | grep flush
root       710  0.0  0.0      0     0 ?        S     2012   2:29 [flush-202:1]
root     10732  6.2  0.0      0     0 ?        S    Apr14 453:33 [flush-202:80]
ubuntu   24009  0.0  0.0   8080   900 pts/3    S+   06:26   0:00 grep --color=auto flush

I notice that the flush-202:80 process often jumps to the top of top. This is on Ubuntu Server 12.04 running Apache, MySQL, and Solr processes. Googling the subject I found another SF post which mentions that these processes should never consume much CPU. The solution presented in that post mentions to reboot the system, which is exactly how I came to poke around in top and discover the processes in the first place: the system won't reboot.

Answer 1

The numbers tell you what the flush process is working on. Do

grep ^ /sys/class/block/*/dev

and you will see something like

:
/sys/class/block/sda1/dev:8:1
/sys/class/block/sda2/dev:8:2
/sys/class/block/sda5/dev:8:5
/sys/class/block/sda6/dev:8:6
/sys/class/block/sda/dev:8:0
:

compare this with your ps ax | grep flush

10213 ?        S      0:00 [flush-8:0]

and you can see that what flush is working on is my /dev/sda.

You could probably get more info by doing

grep "201:80" /proc/self/mountinfo
grep "201:1" /proc/self/mountinfo

and

lsof | grep "201,80"
lsof | grep "201,1"

to see what processess have opened what files on these devices.

What does top show you before and while this is going on? It would be best to track the problem down rather than do a reboot. Once you find the process that is causing the problem you might be able to tune /proc/sys/vm/dirty_background_ratio, /proc/sys/vm/dirty_expire_centisecs and /proc/sys/vm/dirty_ratio to compensate.