I am trying to copy a large amount (about 50 GB) of data over my network from a Hyper-V-hosted virtual machine running Ubuntu 11.04 (Natty Narwhal) to another (non-virtual) Ubuntu host that I plan to use for testing upgrades to one of our web applications. The problem I am having is with the virtual machine, which I shall refer to in what follows as "source.host". This machine is running 64-bit Ubuntu Server with the 2.6.38-8-server kernel and the Microsoft Linux Integration Components for Hyper-V kernel modules (hv_utils, hv_timesource, hv_netvsc, hv_blkvsc, hv_storvsc, and hv_vmbus) loaded. It uses a Hyper-V "synthetic network adapter" for its networking interface.

To do the copy, I log on to the machine with the data and run the following commands (Call the remote machine "destination.host".):

$ cd /path/to/data
$ tar -cvf - datafolder/ | ssh user@destination.host "cat > ~/data.tar"

This runs for a while and then suddenly stops after transferring somewhere from 2-6 GB. The terminal on the source.host machine displays a Write failed: broken pipe error. The odd part is this: after this occurs, the "source.host" machine is no longer able to talk to the rest of the network. I cannot ping any other hosts on the network from the "source.host" machine, and I cannot ping the "source.host" machine from any other host on the network. I am equally unable to access the any of the web services hosted on "source.host".

Running ifconfig on "source.host" shows the network adapter to be up and running as usual with the correct IP address and everything. I tried restarting the networking service with

$ /etc/init.d/networking restart

but the problem does not go away. Restarting the machine makes it capable of talking to the network again -- it can ping and be pinged by other hosts, and the web services are also accessible and usable as normal -- but attempting the copy operation again results in the same failure, requiring another restart.

As an experiment, I tried replacing the tar -- ssh pipeline above with a straight scp:

$ scp -r datafolder/ user@destination.host:~

but to no avail

Thinking that the issue might have to do with the kernel packet-send buffers filling up, I tried increasing the buffer size to 12 MB (up from the 128 KB default) with

# echo 12582911 > /proc/sys/net/core/wmem_max

but this also had no effect.

I'm guessing at this point that it might be a problem with the Microsoft synthetic network driver, but I don't really know. Does anyone have any suggestions?

Thank you very much in advance!

  • What kind of NIC do you have in the Hyper-V host server (these problems seem to be very common with cheap NICs like Realtek or Via)? – Chris S Jul 22 '11 at 3:58
  • Thanks for your reply. The host server NIC is a Broadcom card. – Anonymous Jul 22 '11 at 15:19
  • Has the firmware been updated this year? In Dec 2011 Broadcom released new firmware for most of their server NICs that fixed quite a few problems. – Chris S Jul 22 '11 at 16:27

Have you tried disabling broadcom teaming if enabled on the hyper-v network connection?

We have had to set ours to Active/Passive as the broadcom drivers + linux vm were causing mac address confusion and eventually causing the Linux guest to lose all network connectivity until rebooted/live migrated or virtual networks re-connected.

| improve this answer | |

Yoû're quite right with the hv_netvsc module. Some notable changes to this driver (including hv_vmbus) only whent into 2.6.39 and 3.0 where they are much more stable. Regular network stall or random crashes were quite normal on any Ubuntu and Debian VM I saw using these

Up till 2.6.38 at least, I saw random network loss quite regularly, after that it's much more stable - but still not perfect.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.