Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I wrote a small script to print the memory usage during a large sequential write of a file.

#!/bin/bash
rm result
echo 3 > /proc/sys/vm/drop_caches
sync;
echo start
nohup time dd if=/dev/zero of=mem bs=1M count=2000 & 
for i in {1..200}
do
  sleep 0.2
  cat /proc/meminfo | grep Dirty >> result
  cat /proc/meminfo | grep Dirty
done
cat nohup.out
cat result

I should see the increase of the "Dirty" size from the beginning of the run. But when I ran the script, I often see a big delay (up to several seconds), during which the "Dirty" size does not increase, which possibly means the start of "dd" program is delayed. A sample problematic output is:

Dirty:                20 kB
Dirty:                20 kB
Dirty:                20 kB
Dirty:                20 kB
Dirty:                20 kB
Dirty:                24 kB
Dirty:                24 kB
Dirty:                24 kB
Dirty:                24 kB
Dirty:                28 kB
Dirty:                28 kB
Dirty:                28 kB
Dirty:                28 kB
Dirty:                28 kB
Dirty:             16528 kB
Dirty:            140608 kB
Dirty:            277228 kB
Dirty:            311768 kB
Dirty:            434308 kB
Dirty:            563352 kB
Dirty:            690952 kB
...

The length of the delay is uncertain, sometimes there's no delay at all. And in contrast, when I ran

time dd if=/dev/zero of=mem bs=1M count=2000

with some real time meminfo viewer, such as:

#!/bin/bash
clear
while true
do
  sleep 0.2
  tput home
  cat /proc/meminfo
done

I always see the "Dirty" size increases immediately. Is there something wrong with my script? I also doubt about how the "write" operation is executed by the OS, because I also tested the file read and detected the "Cached" field in /proc/meminfo, and it seems to have no delay at all.

Thanks,

share|improve this question
    
at linuxinsight.com/proc_sys_vm_drop_caches.html it says that sync should run first –  quamis Nov 8 '11 at 21:38
    
just wanted to point out that you can combine your two cats into one line with tee: cat /proc/meminfo | grep Dirty | tee -a result; this would append to 'result' and also output to STDOUT. Just FYI :) –  Jan Wikholm Jul 22 '13 at 7:34

2 Answers 2

You can determine what is taking up the additional time by either invoking the script with

/bin/bash -x /path/to/script

and watching the output. This will print each line of code of the script as it executes. Alternatively you could preface each command with the time command and what is taking the additional time will become readily apparent.

share|improve this answer
nohup time dd if=/dev/zero of=mem bs=1M count=2000 & 

do you have a "time" binary on your system? otherwise nohup wouldn't know how to run a bash internal by itself and it will fail

share|improve this answer
    
Yes I have the time binary: root@yonggang-laptop:/# which time /usr/bin/time –  yonggang Nov 8 '11 at 22:04
    
it may be because vm is dropping the caches while dd is running, for a while (those few seconds). I didn't check drop_caches handler, I am just assuming. I suppose if you comment the vm dropcaches enabler in your script you'll see dirty bufs size increasing faster? –  sysfault Nov 8 '11 at 22:11
    
Thanks for the reply. But that's not where the problem comes from, either. I have commented both the dropcaches and sync, and made sure the machine has finished flushing or freeing the pages before starting to write, but the big delays still exist. –  yonggang Nov 8 '11 at 23:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.