Tell me more ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.
up vote 2 down vote favorite
1

I'm having a problem with my VPS and disk IO. My server is running nginx + PHP-FPM + APC. The database is located on another dedicated VPS. I have several Wordpress MU sites living on the web server. The average IO rate is 6k block/sec.

I'm trying to understand what's causing the high IO.

Output of 'free -m': 

            total   used   free   shared   buffers   cached
Mem:         1005    973     31        0        96      568
-/+ buffers/cache:   307    697
Swap:         255      8    247

Output of 'iotop':

Total DISK READ: 0.00 B/s | Total DISK WRITE: 3.90 M/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 2150 be/4 root        0.00 B/s    0.00 B/s  0.00 % 65.25 % [flush-202:0]
 6694 be/4 www-data    0.00 B/s   19.64 K/s  0.00 %  0.00 % php-fpm: pool www
 6700 be/4 www-data    0.00 B/s   23.56 K/s  0.00 %  0.00 % php-fpm: pool www
 8646 be/4 www-data    0.00 B/s  424.12 K/s  0.00 %  0.00 % php-fpm: pool www
10974 be/4 www-data    0.00 B/s   19.64 K/s  0.00 %  0.00 % php-fpm: pool www

The 'flush-202:0' process sometimes hits IO of 99%. I've read this is the disk cache flushing process but don't really understand what causes it to run/how to fix it.

Any help would be appreciated.

Thanks

share|improve this question
Ok, so what filesystem are you using? I think this is caused by using too much data in buffers, looks like [flush] is initiated by "sync" command. Try running "sync" manually - does it hit 99% of I/O? What hardware are you running on? Is this Hardware RAID card with it's own cache? Ahh.. VPS.. You probably don't know :( Let's do some search: linux.livejournal.com/1707762.html?style=mine Issue vmstat -d: what is it saying? – user107914 Jan 22 '12 at 19:48

3 Answers

up vote 1 down vote

I'm not sure that iotop sample shows something unusual. It's not a problem for the flush process to be a high percentage of your i/o at any point in time if there isn't much i/o going on at that time. I would install atop, which can present real-time data like iotop but has the advantage of also logging samples throughout the day. A day after installing it, I would open the logged data with atop -r log_filename, then go through the samples with t until I found times when the i/o reported in the system-level output is high. Then I would switch the per- process output to disk with d to see what processes were generating the i/o activity.

share|improve this answer
up vote 0 down vote

It sounds like your answer can be found here:

limit linux background flush (dirty pages)

share|improve this answer
Thanks a lot for the link. I'm going to try and experiment with changing the dirty_bytes value. Do you have any idea what process of the server could be causing this? – Mem Feb 22 '11 at 18:48
The flush process is just something that writes out memory pages to disk. This happens in the background even if you are not short on memory. The kernel will copy pages to disk so that if it does need more memory it can drop the pages from memory without having to copy them to disk at that time. It should only be doing this when the disk is otherwise idle. – Zeki Feb 23 '11 at 1:59
up vote 0 down vote

You can acheive this with the pidstat program. Some distros dont come with it installed. But you can download the sysstate package from here and compile it. Dont install it but copy out the pidstat it compiles (or just run it in the current directory). You can pass the '-d' flag to get the output you want.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.