I've heard it said that a 512byte data transfer over fibre channel is 10us so would an 8k (16x the data) take 160us? or is much of that 10us one time transmit overhead?
I'm interested in comparing NFS to FC. Over NFS I'm seeing about 350us for 8K with out jumbo frames. I figure it could be taken down lower than 350us with some modifications. (and it seems like most of that is from the TCP layer. I'm only looking at the NFS server and on the server only about 50us is from NFS layer, using dtrace for the measurements. I don't know the client side NFS costs but if it's similar, that would mean the bulk is from TCP).