3

I've setup a VM running Windows Server 2008 on a host node running Xen 3.4.3-5 and the following kernel: 2.6.18-308.1.1.el5xen #1 SMP Wed Mar 7 05:38:01 EST 2012 i686 i686 i386 GNU/Linux

The network speed on the VM is very slow--using the online speed tests I can only get it up to 8-9mbps. The line is 100mbps burstable and the host node has no problem achieving those speeds. If I setup a VM running CentOS (instead of Windows Server 2008), that CentOS VM has no problems achieving those speeds--only Win2K8 struggles.

I've done some pretty exhaustive troubleshooting, but nothing has helped:

  • New VM installations of Win2k8 do have the same network problem.
  • Upgrading to most recent kernel-xen did not help (2.6.18-308.1.1.el5xen).
  • Upgrading from xen 3.4.0 to xen 3.4.3-5 did not help.
  • Disabling Windows firewall, etc did not help.
  • Changing network card device config from auto negotiation to manual 100mbps full duplex did not help.
  • Changing the network receive buffer packet size did not help (tried all combos from 64k to 8k).

At this point I'm pretty much out of ideas--any help would be appreciated!

EDIT: I eventually resolved this problem. See my accepted answer below.

2 Answers 2

8

The problem is likely with the Windows guest and how it is communicating with the virtual NIC. Here are some things to try:

Disable TCP chimney offloading in the Windows guest. To determine if it is enabled run netsh int tcp show global. To disable it run netsh int tcp set global chimney=disabled

Disable TCP Offloading on the NIC in the Windows guest. In the guest Windows installation, go to device manager, open the NIC in question, go to the advanced tab and search for any option pertaining to TCP/UDP, checksum, largesend and/or TCP/IP offloading. Disable them. If it says "offloading" disable it.

Disable Receive Side Scaling in the Windows guest. Check if it is enabled with netsh int tcp show global. Disable it with netsh int tcp set global rss=disabled.

Disable NetDMA in the Windows guest. Check if it is enabled with netsh int tcp show global. To enable / disable it manually a registry key needs to be added. Check to see if it already exists with get-itemproperty -path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters -name EnableTCPA. If it does not exist and NetDMA is enabled, create the following registry key with PowerShell:

New-Item -Path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\EnableTCPA
New-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\" -Name "EnableTCPA" -Value 0 -PropertyType "DWord"

Disable Windows Network Task Offload in the Windows Guest. Check if the registry key exists using PowerShell: get-itemproperty -path HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters -name DisableTaskOffload. If it doesn't, add this registry key with PowerShell: New-ItemProperty "HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\" -Name "DisableTaskOffload" -Value 1 -PropertyType "DWord"

Disable Autotuning in the Windows Guest. Check first to see if it's enabled: Netsh interface tcp show global. If it is, disable it with: Netsh interface tcp set global autotuning=disabled

Disable Remote Differential Compression in the Windows Guest if it is enabled. Go to the run command and run appwiz.cpl, select "Turn Windows Features on or Off" and deselect remote differential compression.

Check if Anti-virus is set to scan live traffic. Antivirus tools can be set to scan all traffic coming and going from the host for anything malicious. If that is the case on your Windows guest, disable it temporarily to test.


EDIT 1

Some last ditch efforts on this include:

  • One of my guesses was that there was a port speed mismatch - but you addressed that. Unless there's more options you can frob with in this regard.
  • I usually recommend updating the network driver but this is less necessary in a VM since the virtual NIC is rather vanilla.
  • Install the latest guest tools into the Windows Guest.
  • I've seen some issues with IPv6 on Windows clients causing mysterious TCP/IP slowdowns. No idea why; I haven't researched it deeply. I suppose you could uninstall / remove v6 from the NIC.
  • Check TCP/IP stats in the Windows guest to see if there are lots of errors or other interesting numbers. netstat -es will do the trick.
  • Last, but certainly not least, install Network Monitor and inspect the traffic for anything unusual. CRC errors, reconnections, dropped packets... who knows. Anything that looks ugly.
3
  • Thanks for all these tips--I've applied everything you've listed here and speeds improved marginally (roughly 0.5 mbps), but still under 10mbps and well under the expected 100mbps. Do you have any other ideas or diagnostic steps I could take?
    – Elliot B.
    Apr 12, 2012 at 3:52
  • @Elliot.Bradshaw I updated my answer
    – Wesley
    Apr 12, 2012 at 4:00
  • @WesleyDavid Thanks very much, my download speed maxed out and my upload speed went from about 10 Mbit to 50 Mbit, which is a big improvement! Cheers!
    – Devator
    Aug 11, 2012 at 11:38
6

I solved this problem by installing Xen paravirtualization drivers for the NIC and other components in Windows. The default drivers on the Windows guest were interacting horribly with the Xen host node. I noticed that downloading a file in Internet Explorer on the VM would spike up the CPU on the host node to 75-90%. Installing James Harper's GPLPV drivers for Windows Server 2008 32bit resolved the issue.

If anyone has a similar problem, simply download and install the appropriate set of GPLPV drivers found here: http://wiki.univention.de/index.php?title=Installing-signed-GPLPV-drivers

However, for my particular server, installing the GPLPV drivers was not enough. The GPLPV drivers restored the download speeds to normal levels (~66mbps), but upload speeds were around 2mbps. I then re-tried the tips @WesleyDavid provided to disable all forms of offloading in the NIC config (via device manager) and that boosted upload speeds up to 36mbps. I've since installed GPLPV drivers on other systems and modifying the NIC config was not necessary.

Update:

Since I posted this question a year ago, I've installed Windows on a variety of different Xen machines. Every time I've installed James Harper's GPLPV drivers both network performance and general system performance has improved dramatically. I've never been able to achieve full download/upload speeds without the use of Harper's GPLPV drivers.

Anyone virtualizing Windows with Xen on a RHEL/CentOS host machine should definitely try out the drivers provided at http://wiki.univention.de/index.php?title=Installing-signed-GPLPV-drivers

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .