I'm running a Ubuntu Server 11.04 (amd64) on a machine with 1.6Ghz dual-core CPU, 4GB RAM, and 2x250GB 7200 SATA drives (both 3.0Gb/s and newly formatted from a fresh install). When I transfer relatively small files (~100MB) between the two drives, I achieve transfer rates that are (better) than expected, in the 300-350MB/sec range. However, transferring a 500MB file achieves about 50MB/sec, while a 1GB file achieves about 25MB/sec.
Here are some tests I ran that show this behavior:
/disk1 $ dd if=/dev/zero of=output-small.dat bs=1M count=128 bytes (134 MB) copied, 0.388619 s, 345 MB/s /disk1 $ dd if=/dev/zero of=output-med.dat bs=1M count=500 524288000 bytes (524 MB) copied, 20.0151 s, 26.2 MB/s /disk1 $ dd if=/dev/zero of=output-large.dat bs=1M count=1000 1048576000 bytes (1.0 GB) copied, 45.7304 s, 22.9 MB/s /disk2 $ dd if=/dev/zero of=output-small.dat bs=1M count=128 134217728 bytes (134 MB) copied, 0.372266 s, 361 MB/s /disk2 $ dd if=/dev/zero of=output-med.dat bs=1M count=500 524288000 bytes (524 MB) copied, 10.0856 s, 52.0 MB/s /disk2 $ dd if=/dev/zero of=output-large.dat bs=1M count=1000 1048576000 bytes (1.0 GB) copied, 37.1851 s, 28.2 MB/s $ time cp /disk1/output-small.dat /disk2/test-small.dat real 0m0.532s user 0m0.000s sys 0m0.410s $ time cp output-med.dat /playpen/test-med.dat real 0m30.879s user 0m0.010s sys 0m3.740s $ time cp /disk1/output-large.dat /disk2/test-large.dat real 0m26.643s user 0m0.010s sys 0m3.390s
Finally, when running the command
pv < /disk1/output-large.dat > /disk2/test.dat, I initially see excellent transfer speeds, but then the rate drops off to the 1-10MB/sec range, periodically jumping (much) higher. I know this is not an particularly useful measurement of disk performance as I'm going through pv, but it is consistent performance with my other tests.
I have a couple questions:
- What could be causing my transfer speeds to decrease with larger files? Shouldn't it be increasing?
- What speeds should I expect to see with this kind of setup?
- The time spent (presumably) waiting on IO (from
sys) seems suspiciously high to me. Is it?