Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a server running apache, postgresql, and a lot of sites, and my graphs from my hosting provider indicate basically constant disk thrashing. 10k/s disk usage. This seems like a bad idea, but I'm not really sure why -- load is fine, server response time is reasonable. At first I thought that it was that the box was stuck in swap, because there were 2 application server threads for each site running, but reducing the number of sites running and the disk usage is still high. There's a little swap in use, but a lot of free memory. From top:

load average: 0.70, 0.28, 0.21

Cpu(s):  3.6%us,  1.4%sy,  0.0%ni, 93.9%id,  0.7%wa,  0.0%hi,  0.0%si,  0.4%st
Mem:   1731880k total,  1530124k used,   201756k free,    68360k buffers
Swap:  2359288k total,     1676k used,  2357612k free,   884892k cached

So I looked at vmstat, and it says swap isn't the issue:

root@foobar# vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0   1676 175220  68120 880856    0    0     1    20   20   15  4  1 94  1
 0  0   1676 174096  68140 881140    0    0     0  6484 1452  487  2  1 97  0
 0  0   1676 173848  68156 881420    0    0     0  5804  899  310  0  1 99  0
 0  0   1676 173600  68168 881768    0    0     0  5376  834  266  0  0 98  2
 0  0   1676 173228  68176 881880    0    0     0  5376  775  259  0  1 99  0
 1  0   1676 173044  68188 882152    0    0     0  3584  626  249  0  1 99  0
 0  0   1676 172796  68208 882424    0    0     0  6668 1450  593  2  1 97  0
 1  0   1676 181352  68224 882700    0    0     0  5760 2373 1038  3  2 95  0
 0  0   1676 181336  68228 882836    0    0     0  5376 3367 1289  7  1 91  0
 0  0   1676 181088  68236 883264    0    0     0  5376 1294  508  2  1 97  0
 0  0   1676 180848  68252 883440    0    0     0  5376 1217  410  1  1 98  0
 0  0   1676 180716  68264 883480    0    0     0  8444 1127  270  0  1 99  0
 0  0   1676 181584  68272 883488    0    0     0  5676 1285  624  1  1 98  0
 0  0   1676 180956  68272 883496    0    0     0  5376 1069  377  1  1 97  0
 0  0   1676 180964  68272 883496    0    0     0  5376  753  299  0  1 99  0

I tried increasing postgresql's shared_mem and work_mem settings, to no avail. I'm really not sure how to track down what's going on.

edit: here's iotop's big revelation:

26331 be/4 postgres    0.00 B/s    5.00 M/s  0.00 %  0.00 % postgres: stats collector process

running constantly, basically. Everything else is in the few kbps.

Any suggestions?

share|improve this question
    
Have you tried running iotop I feel that woud highlight the cause of the problem. –  James Park-Watt Nov 5 '12 at 19:30
    
holy shit yes, that shows me something. –  freyley Nov 5 '12 at 20:12

1 Answer 1

up vote 1 down vote accepted

iotop will highlight the cause of the problem.

share|improve this answer
    
it did. In this case, the problem was the postgres stats collector, and moving the stats collector to collect stats in an in-memory location reduced io usage to a reasonable level. –  freyley Nov 5 '12 at 20:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.