Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I've seen in many places that throughput = bs * iops should be true. For example writing at 128k block size to a SAS disk that can support 190 IOPS should give a throughput of ~23 MBps - 23.75(MBs) = 128(BS)*190(SAS-15 IOPS)/1024.

Now when I tested it in a VM against a monster NetApp filer I got theses results:

# dd if=/dev/zero of=/tmp/dd.out bs=4k count=2097152
8589934592 bytes (8.6 GB) copied, 61.5996 seconds, 139 MB/s

To view the IO rate of the VM I used iostat and esxtop, and they both showed around 250 IOPS.

So to my understanding the throughput was supposed to be ~1000k: 1000(KBs) = 4(BS)*250(IOPS).

dd of 8GB is twice the size of RAM of course, so no page caching here.

What am I missing?

Thanks!

share|improve this question
add comment

3 Answers

That you are missing is the context. IOPS is FULLY RANDOM. A copy is not random but sequential. Hard discs get slow when the head is moved - the IOPS basically assumes, properly measured, IO that is randomly distributed over the complete disc platter (or at least a large part of it).

Yes, you are a lot faster when copying a disc. SADLY that is totally irrelevant unless your normal usage is only copying by ONLY ONE USER AT A TIME.

That is like measuring the top speed of a formula 1 car and then assuming that this is the average speed during a race- bad mews, formula 1 tracks have corners, cars mostly go a lot slower.

So, if you do not do totally degenerated patterns (in the technical term), i.e. only have one copy operation at a time, then the IO will be random (especially virtual machines- one may be sequential, 20 hitting the same disc is random) and the head spends most of the time moving, not doing IO operations.

dd of 8GB is twice the size of RAM

It still is pathetic, is is not? How l,large is the disc? (gb is only a small part, so the "random" part is very few movements (measured in length) compared to the real world scenario ;) Actually no randomm movement as you copy from a zero source, soiit is only writing, never moving the head. BAD ;)

ON TOP:

against a monster NetApp filer

ANY idea how much those large SAN items are able to optimize your IO? How much cache does it have? A "monster" filer would be one of the top models, which has 16+ gigabyt ememory for its own cache use. If it ireally a monster, your file is pathetic - wikipedia reads the top line of 2010 (!) having 192gb memory ;) Does not even realize when buffering 8gb. And deduplication (does it happen real time?) may eliminte pretty much all the write operations. Are you sure you did even measure disc based IOPS?

share|improve this answer
2  
No. But that if you measure the wrong fact you measure the wrong fact. As in: For discs, IOPS hare very depending on usage pattern (how much time is wasted moving the head), which is constant for a ram disc - and stuff like RAM caches, plenty of them in large SAN, mean that you have to make sure that those are not falsifying results. Your test is simply not measuring the IOPS budget of a disc under normal random load circumstances but under laboratory scenarios with near zero seek time. –  TomTom Apr 15 '12 at 20:00
1  
TomTom's dead on. You tested under unrealistically perfect conditions and got an unrealistically perfect result. –  David Schwartz Apr 15 '12 at 20:32
1  
No, this is true- the main problem is that the IOPS number for SSDD is constant regardless where it accesses, while the IOPS for HD goes down for random access. Also, IOPS always includes a specific size of IO. SSD for example often give it as IOPS and 4kb - because larger blocks take more time. But yes, that is the idea. –  TomTom Apr 16 '12 at 15:00
1  
No. Throughput = BS*IOPS is always true, just IOPS varies depending on usage pattern,. The IOPS numbers published assume either "random load" or are best case ("up to"). Point is tht stuff likeaverage seek time etc. play into the game - and never forget WHAT IOPS NUMBERS ARE FOR - estimating server workloads, and server workloads are NEVER sequential. File copy - random. Because 200 people hit the same platter array for various files at the same time. –  TomTom Apr 16 '12 at 19:34
1  
You are confusing two different statements. One is "BS * IOPS from specification = measured throughput". The other is "BS * IOPS actually done = measured throughput". The first one is only true under the conditions under which the specification's IOPS is valid. The second is always true. –  David Schwartz Apr 17 '12 at 8:54
show 17 more comments

There's an app called SQLIO, don't worry about the name it's actually got nothing to do with SQL it was just written by the SQL Server team at Microsoft, which will let you test your disk with random IO (read or write) and see just how much load the disks can handle. You can download it from Microsoft's site.

share|improve this answer
add comment

If you're going to use throughput = block size * IOPS, you have to use the block size of the I/O operations you are counting, not the block size of the file system, not the block size of the block device.

The 139MB/s is probably a bit higher than what you really got because I/O was likely continuing when the measurement stopped. The block cache was likely still flushing. So it seems like the most logical explanation is that the size of the underlying I/O operations you are counting is 512KB.

The block size of the I/O operations has to be some multiple of the block device's block size. I believe you say that's 128KB. So 512KB (4 block) operations are certainly possible.

512 * 250 = 128MB/s

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.