Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I seem to have a strange issue with my VMware cluster, where I get inconsistent SCP transfer rates. I have Cluster1 and Cluster2, physically in different regions. I need to transfer large vmdk from Cluster1 to Cluster2. Here is what I get:

  • SCP from VMware host, directly to VMware host, no compression: 4MB/s
  • SCP from VMware host, directly to VMware host, with compression: 0.5MB/s
  • SCP from VMware host in Cluster1, to one of the virtual machines running in Cluster1: 30MB/s
  • SCP form virtual machine in Cluster1 to VMware host in Cluster2, with compression: 15MB/s
  • SCP form virtual machine in Cluster1 to VMware host in Cluster2, no compression: 5MB/s
  • SCP form virtual machine in Cluster1 to virtual machine in Cluster2, with compression: 20MB/s
  • SCP form virtual machine in Cluster1 to virtual machine in Cluster2, no compression: 7MB/s

Testing network bandwidth with iperf shows that I have 200-300Mbps between the locations consistently. Network connection in both clusters, as well as internet is gigabit.

What would cause VMware to limit CPU usage for compression, and what would limit it's transfer rates outside of the cluster?

PS: inside the cluster I am going through public IP, and guest VM is on a different host. So theoretically VMware shouldn't know that the transfer is somewhat local.

EDIT: Cluster1 is 4.1, Cluster2 is 5.0; Tried FastSCP, and getting the same result as I see with direct SCP with compression: about 0.5MB/s.

EDIT 2: Increased system resource allocation on VMware hosts to the levels that VMs are getting and beyond. Only change is SCP with compressing from host to host is not up from 0.5M to 4M, just like non-compressed transfer.

Still puzzling why would host-to-host transfer be slower though.

EDIT 3: after adding more resources, was able to achieve 4~10MB/s transfer speeds across data centers. Even though it is lower than 20~40 that the network should be capable of, I'll just have to live with that. Although if anyone has any other ideas - I'm eager to try them :)

share|improve this question
add comment

1 Answer

up vote 3 down vote accepted

This is why people use third-party products like FastSCP.

I'm assuming you're on ESXi version 4 (you did not specify). This was improved in ESXi version 5. The reason the copies were intentionally throttled was tied to console resource management.

share|improve this answer
    
Edited the question to provide more details :) Is there any way to increase "console resources?" –  Serge Apr 18 '12 at 17:53
    
I don't think you need compression. –  ewwhite Apr 18 '12 at 17:54
    
I don't, I just need faster transfer speeds :) –  Serge Apr 18 '12 at 17:56
2  
You can assign more resources to the Console under the node's Configuration -> System Resource Allocation -> Edit menu. –  pfo Apr 18 '12 at 18:05
    
Bump up the RAM and CPU MHz reservation... –  ewwhite Apr 18 '12 at 18:07
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.