Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

One of the sites I work at uses VMWare virtual servers. For the most part they work very well, but copying large files onto them from other places on the network is very slow.

By large files I mean around 4GB in size (which isn't actually that big these days)

I have been watching my 4GB file copy for 90 minutes so far, and it still says about 2 hours to go.

Is there some VMWare trick for speeding up network file copies?

Technical details: Its VMWare ESX running on Linux. The network is a 1GB Lan. So far I've been trying to copy just using Windows Explorer, by remote desktopping into the virtual server and then connecting out to the network and copy/pasting the file.

For comparison, if I copied between two real machines on the same network, I would expect 4GB to copy in roughly 5 minutes. Going from 5 minutes to 240 minutes makes me think there is something going very wrong somewhere ....

share|improve this question
    
What is the speed of your connection? 10Mb, 100Mb, 1Gb? –  Joseph May 18 '09 at 12:26
    
Are you copying them from inside virtual machines, or from the hosts themselves? If they're in guests, what OS is it? What method of file copy are you using, FTP, drag & drop? If you clarify with these, you'll get much better answers. Hope that helps! –  Brent Ozar May 18 '09 at 12:27
    
Sorry for the lack of details - I dont administer the VMWare setup so I had to ask the relevant people - have now updated the question with more details. –  codeulike May 18 '09 at 12:37
    
So you're copying from within the guest OS? Yes that should be at near native speed. Perhaps there are too many guests competing for bandwidth? Is the 1G link shared by all the guests? Most ESX boxes used trunked connections. You need to get a summary of CPU and bandwidth stats at the Host level from the VM admins to troubleshoot. –  PowerApp101 May 18 '09 at 13:01
    
Is there any NAT'ing going? make sure your VMWare NIC is bridged not natted. What speed do the NIC's show on the virtual machine? –  Nick Kavadias May 18 '09 at 13:03

4 Answers 4

up vote 6 down vote accepted

The first thing to understand is that you're sharing the 1GB connection with other virtual machines. You're not the only person using the network. Ask how many other virtual servers are on that same host. If you're really nice, the sysadmins might let you look at the VMware Virtual Center Performance tab, which will show you the network use overall.

Next, windows drag & drop file copies aren't very efficient. One trick for speeding up file copies (if you really have to do this often) is to do it with the hosts via SCP/FTP instead. This isn't a quick setup, but here's how it works:

  • Set up an FTP server on the VMware host (the same one the guest is running on)
  • FTP (or SCP, if your sysadmins don't allow FTP) from the Windows guest to that VMware host (will run blazing fast since it doesn't actually go over the network - all file transfer is in the same host)
  • From your physical boxes running Windows, connect to the VMware host's FTP/SCP server and grab the file.

Like I said, not easy, but once you do it it's blazing fast. It will still be constrained by the network that you're sharing with other virtual machines, though. For best practices, VMware recommends using multiple network connections, and you might be able to sweet-talk your sysadmins into using a separate network adapter for the FTP/SCP server.

share|improve this answer
    
Thanks, your answer will be useful to anyone else who encounters this issue. I got my file copied by just FTPing out from the guest OS to an FTP server on the network. That did the copy in 15 minutes as opposed to 240. Your suggestion of hosting FTP on the VMWare host also sounds very good, will look into it. –  codeulike May 18 '09 at 14:16
    
If you need to copy several files, start several FTP sessions. Your VMware sysadmin will hate your guts, but you'll get even faster throughput. There's also a lot of tweaking you can do to the TCP parameters to make it utilize more of the network - check out the concepts of windowing and TCP chimneying - blogs.msdn.com/psssql/archive/2008/10/01/… –  Brent Ozar May 18 '09 at 14:26

FastSCP is slow as on esxi...just like everything else, just like VMware intended....

ESXi is capped to about 6meg max for file copies of SSH no matter what you do...if you want fast file copies you need to pay Vmware for the privilege

share|improve this answer
    
+1, you gotta be kidding me, have been pulling my hair out over this wondering why on earth transfers over ssh were maxing out at 6MB/sec; meanwhile, ftp transfers over a gigabit network are @100MB/sec. Big company policies like this completely blows, throttling performance to squeeze a few more $$ out of you. Anyway, @stavros, thanks for the tip, good to know why ssh transfers are dog slow... –  virtualeyes Nov 7 '11 at 8:14
    
Amidst the slight rant ;-), meant to say that throttling performance based on license type is effectively a tax on the poor (small business owner). I can understand if features are absent, but providing a barely usable option is evil marketing (through painful performance they force you to upgrade or come up with workarounds) –  virtualeyes Nov 7 '11 at 8:24

Do you mean ESX or Vmware Server (the freebie that runs on Windows)? I assume you mean ESX...in which case there are various options. How are you doing it now? Also what speed is the link? LAN or WAN? 100Mbps or 1Gbps etc? If you are copying from Windows using WinSCP or similar then that will be slow. SCP does encryption on the fly and WinSCP is particularly slow I find. Another alternative is to enable FTP server on the ESX host. Then you can just ftp from Windows, which will be hugely quicker. If you are going from ESX to ESX you can use command line scp which isn't too bad (compared to the Windows version).

share|improve this answer
    
Thanks, I have updated the question with more technical details –  codeulike May 18 '09 at 12:41

Have you tried using FastSCP from Veeam? It uses a less resource intensive encryption for SCP. It's free and fast.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.