I have a pretty simple configuration: Windows Server 2012 Hyper V Host, with two VMs (both Server 2012), one for IIS, one for SQL Server. They're all connecting through the physical NIC
Last night I tried adding a virtual switch, and setting up virtual NICs on the two VMs (in addition to the physical NIC).
I hadn't switched over the servers to actually use the virtual NICs yet, but for some reason, this destroyed the connection speed between the two VMs -- even though the virtual NICs weren't being used. We're talking from 200-500 milliseconds to 30-90 seconds per request!
So this morning, after seeing the terrible impact it had on my site, I disabled the two virtual NICs, since that's all that I changed, and sure enough, speeds came back.
Ultimately I need to get them switched over to using the v-NICs, but I need to get past this issue first.
What could be causing this? And what can I do to troubleshoot it?
The setup is actually very simple (which may be part of my problem, I may be oversimplifying what is needed to get this set up).
I have a single server with a single (physical) NIC. Currently, this NIC is used for communication between the v-host and the internet, as well as between the two VMs and the internet. It is also used for intra-communication between the two VMs.
However, I'm trying to switch the intra-communication to go through the virtual swtich via vNICs, freeing up the physical NIC to be used only for external communication.
So I added an internal virtual switch to the v-host, then added a virtual NIC to each VM.
After doing this, that's when my connection speeds -- over the physical NIC -- dropped drastically. Disabling the vNICs (through the VMs' network configuration) restored the speed back to normal.
Below are screens of the VM configuration for the external physical NIC and internal vNICs (on just one VM, as they're both identically configured).
Another round of details
In digging a bit more on the issue, it appears that Broadcom NICs are notorious for performance issues. This server has dual Broadcom BCM5716C NICs. Here are some links to discussion of known issues:
So I started playing with settings, disabling various settings, one at a time. None of the changes seemed to have any impact on the issue -- if I enable the guest vNIC, the load speeds drop to 25+ seconds. Disable it again, they return to 250ms -- a factor of 100!
I also tried installing the latest drivers from Broadcom, no difference.
At this point, I feel it's an issue of compatibility between the Broadcom NIC and Hyper-V. But I don't want to order new NICs and go through the trouble of shutting the site down, installing and configuring them, only for the problem to persist.
So it would be nice if I could definitively rule in or out the NIC.
Physical NIC configuration on the switch manager
Virtual switch configuration on the switch manager
Physical NIC configuration on the DB VM
Virtual NIC configuration on the DB VM