Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My setup:

A hyper-v cluster with 2 nodes (AMD 24 cores, 64gb ram) connect via 2 procurve 25-10g to a p2000g3 SAN (12x300 GB enterprise SAS 10k 2.5"). Each node is an iSCSI initiator.


When I run a VM on a node and launch a basic crystamark disk benchmark, my MB/s are 80 for both read and write. This is poor.

What do you guys think ?

Update1: After a long day of testing here is my first conclusion. It seems i can't expect more than the bandwith of one gigabyte ethernet card. When no mpio and so only one connection, the bandwith is used at 100%, when 2 mpio paths, each path takes 50%, when 3 mpio paths 33%... Enabling flow control does nothing more ( less it seems..), no more for enabling jumbo frames. I can't imagine i'm stuck to 1gb, the p2000 has 2 controllers with 4 ports each. Please tell me it is possible to aggregate this :)

For recap my tests are: Running a crystalmark disk sequential (1000mb) on a vm give me between 80 and 100 mb/s, the same test with the storage vm locally on the hyperv host gives 250.

share|improve this question
What performance are you expecting? – ewwhite Jun 5 '12 at 14:11
I have no idea on what to expect. On a raid1 local disk where i move the vm i get 250mb/s with the same test so it point me i 'must' have a problem – Despe Jun 5 '12 at 15:04
If you are using the same test configured the same way, then it's your network/ethernet cards/drivers at fault. Is this a 10 Gb network configured according to the manual for storage? – Basil Jun 5 '12 at 15:32
What raid model? – 3molo Jun 5 '12 at 15:42
You may want to consider using this for your benchmarking – JGurtz Jun 5 '12 at 18:54

We had a very similar setup a few years ago. It's a complex setup with lots of moving parts that you have to look through. Here are the main areas to check for performance:

  • SAN: Make sure it's configured properly, verify your virtual disk layouts against the manufacturer specs, virtual disks that are spread out often cause problems. In addition make sure you have the proper settings to deal with two or more hosts (cluster) reading from the same virtual disks.
  • Switches: Check your jumbo frames settings, you typically want to make sure jumbo frames are enabled on your switches, SAN and servers for data connections only. Also make sure you are using a VLAN to segregate the traffic if both your network and data traffic share the same switches.
  • Server: Drivers and iSCSI, make sure they are up to date and setup properly. With multiple hosts you really want to use multipath drivers and configuration, again check with your manufacturer. You didn't post your NIC specs but make sure it supports 1GB or higher traffic as well.

That's some real basic starting points. Worst case call up your vendors, they typically have good white papers or case studies around this kind of stuff.

share|improve this answer
In fact i have a custom app latency problem (connected to an oracle vm). I run the basic sequential test on the vm and the same test with the vm and his disk put on one hyperv local disk (raid1) and results are 250, thats why i think it is problematic. Jumbo frames and flow control are not setup yet, i plan doing it tomorrow – Despe Jun 5 '12 at 14:48
You're going to have to walk through each part of the setup to determine where the problem is. There is no magic bullet to diagnosing this type of setup. Check the three areas I listed out, feel free to update the question with any additional information. – Brent Pabst Jun 5 '12 at 14:50

80MBps = 1Gbps Ethernet, you are either only using 1 NIC or you don't have MPIO setup correctly.

share|improve this answer
You point the right thing. I have mpio enabled and set to roundrobin but it seems to use only one nic. Is it possible to aggregate the 2 nics to double the bandwith ? – Despe Jun 6 '12 at 9:25
You might want to check the performance using a more realistic tool like sqlio before mucking with MPIO too much; it's possible you have it set correctly already. Running something like sqlio -kR -t8 -s15 -b64 -d$Driver_Letter sqlio.test should get you in the ballpark for peak performance. – Chris S Jun 6 '12 at 18:26

Your performance will vary based on the type of benchmark. If you're doing sequential reads or writes, you should be able to get a large amount of MB/s performance from the disks. This is not a useful benchmark because that's not the type of IO a virtual server does, usually.

If you're doing small block random IO, your writes will be cached until you run out of cache, and only a fraction of your reads will come from cache. The rest will all go to disk, and then your performance in terms of MB/s is irrelevant. The number of IO/s will be limiting you. Each 10k disk can add as many as a couple hundred IO/s, so with 12 of them, you can't get more than a few thousand.

edit: If you're using a 1Gb/s network switch, then the maximum throughput you can get is 80MB/s, which is right about what you're seeing. Add network ports on the switch and see if it improves.

share|improve this answer
Do you have some tools/tests i can run to compare with a local disk system ? – Despe Jun 5 '12 at 15:08
I always use IOMeter, but again, you need to configure the tests carefully. If everything between the disk and the servers are correctly configured, you will be limited to the disk's performance. The disks can do a certain maximum number of random IO/s, and a certain maximum number of MB/s. You will rarely max out MB/s while you are doing random IO, and will rarely max out IO/s while doing sequential IO. – Basil Jun 5 '12 at 15:29

if you have 2x1gig connections per server and 2x1gig connections per SAN node, then you have MPIO misconfigured on the server, or transmit load balancing on the SAN.

if you SAN does not support transmit load balancing to a single host then you're out of luck: the most you'll be able to get TO a host is a single link.

i can't find any documentation that the p2000 support transmit load balancing; i only find references to active/passive controllers.

share|improve this answer
I think my mpio setup is correct as i can monitor in my switches that 2 link are used (50% each) when 2 mpio path and 3 (33% each) when 3 there another way to confirm that ? Could you point me to an article explaining the san support for loadbalancing transmission ? Many thanks – Despe Jun 8 '12 at 15:04
when you say that you see 50% used for 2 links, are you looking at the stats for the ports the server is connected to, or the ports that the SAN is connected to? – longneck Jun 8 '12 at 15:08

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.