That you are missing is the context. IOPS is FULLY RANDOM. A copy is not random but sequential. Hard discs get slow when the head is moved - the IOPS basically assumes, properly measured, IO that is randomly distributed over the complete disc platter (or at least a large part of it).
Yes, you are a lot faster when copying a disc. SADLY that is totally irrelevant unless your normal usage is only copying by ONLY ONE USER AT A TIME.
That is like measuring the top speed of a formula 1 car and then assuming that this is the average speed during a race- bad mews, formula 1 tracks have corners, cars mostly go a lot slower.
So, if you do not do totally degenerated patterns (in the technical term), i.e. only have one copy operation at a time, then the IO will be random (especially virtual machines- one may be sequential, 20 hitting the same disc is random) and the head spends most of the time moving, not doing IO operations.
dd of 8GB is twice the size of RAM
It still is pathetic, is is not? How l,large is the disc? (gb is only a small part, so the "random" part is very few movements (measured in length) compared to the real world scenario ;) Actually no randomm movement as you copy from a zero source, soiit is only writing, never moving the head. BAD ;)
ON TOP:
against a monster NetApp filer
ANY idea how much those large SAN items are able to optimize your IO? How much cache does it have? A "monster" filer would be one of the top models, which has 16+ gigabyt ememory for its own cache use. If it ireally a monster, your file is pathetic - wikipedia reads the top line of 2010 (!) having 192gb memory ;) Does not even realize when buffering 8gb. And deduplication (does it happen real time?) may eliminte pretty much all the write operations. Are you sure you did even measure disc based IOPS?