I have two processes running on my box => A and B.
Whenever there is heavy disk I/O activity caused by writing large files to the disk by process A, I see that file writes in process B are stalling intermittently. I noticed that I had the following virtual memory settings on my linux kernel.
vm.dirty_background_ratio = 5 vm.dirty_background_bytes = 0 vm.dirty_ratio = 20 vm.dirty_bytes = 0 vm.dirty_writeback_centisecs = 500 vm.dirty_expire_centisecs = 3000
Documentation states that 'dirty_background_ratio' is the percentage of the memory occupied by dirty pages before the kernel background flusher threads kick into action to flush the bytes to the disk. Given that I have 64 GB of memory, this background flush should kick off when the dirty page cache size reaches ~ 3.2 GB and I see that it is indeed the case when process A is writing to the disk.
However, as soon as this background flush starts, file writes in process B begin to stall intermittently for periods of about 10-20 seconds. Given that this disk flush is happening in the background, why does a write from process B stall given that all writes first go into the page cache and not directly to disk ?