Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm running a Ubuntu Server 11.04 (amd64) on a machine with 1.6Ghz dual-core CPU, 4GB RAM, and 2x250GB 7200 SATA drives (both 3.0Gb/s and newly formatted from a fresh install). When I transfer relatively small files (~100MB) between the two drives, I achieve transfer rates that are (better) than expected, in the 300-350MB/sec range. However, transferring a 500MB file achieves about 50MB/sec, while a 1GB file achieves about 25MB/sec.

Here are some tests I ran that show this behavior:

/disk1 $ dd if=/dev/zero of=output-small.dat bs=1M count=128
 bytes (134 MB) copied, 0.388619 s, 345 MB/s
/disk1 $ dd if=/dev/zero of=output-med.dat bs=1M count=500
524288000 bytes (524 MB) copied, 20.0151 s, 26.2 MB/s
/disk1 $ dd if=/dev/zero of=output-large.dat bs=1M count=1000
1048576000 bytes (1.0 GB) copied, 45.7304 s, 22.9 MB/s

/disk2 $ dd if=/dev/zero of=output-small.dat bs=1M count=128
134217728 bytes (134 MB) copied, 0.372266 s, 361 MB/s
/disk2 $ dd if=/dev/zero of=output-med.dat bs=1M count=500
524288000 bytes (524 MB) copied, 10.0856 s, 52.0 MB/s
/disk2 $ dd if=/dev/zero of=output-large.dat bs=1M count=1000
1048576000 bytes (1.0 GB) copied, 37.1851 s, 28.2 MB/s

$ time cp /disk1/output-small.dat /disk2/test-small.dat
real    0m0.532s
user    0m0.000s
sys     0m0.410s

$ time cp output-med.dat /playpen/test-med.dat
real    0m30.879s
user    0m0.010s
sys     0m3.740s

$ time cp /disk1/output-large.dat /disk2/test-large.dat
real    0m26.643s
user    0m0.010s
sys     0m3.390s

Finally, when running the command pv < /disk1/output-large.dat > /disk2/test.dat, I initially see excellent transfer speeds, but then the rate drops off to the 1-10MB/sec range, periodically jumping (much) higher. I know this is not an particularly useful measurement of disk performance as I'm going through pv, but it is consistent performance with my other tests.

I have a couple questions:

  • What could be causing my transfer speeds to decrease with larger files? Shouldn't it be increasing?
  • What speeds should I expect to see with this kind of setup?
  • The time spent (presumably) waiting on IO (from time, real - sys) seems suspiciously high to me. Is it?
share|improve this question
add comment

2 Answers

up vote 1 down vote accepted

Small files fit in the page cache, thus the test finishes before stuff is on disk. Try adding the conv=fsync option to dd.

For the 1 GB file, perhaps you're running into some side-effects of writeback resulting in less throughput than the disks ought to be able to handle. You could try the dd options iflag=direct and oflag=direct to bypass the page cache and see how this affects the result.

share|improve this answer
add comment

You might be seeing an effect of file caching. You can set iflag or oflag to nocache, and retest, this way you'll see a raw, media based speed, not buffer-to-buffer speeds.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.