4

What kind of performance can I expect from VMXNET3 adapter when running two Linux guests on the same VMware ESX private network defined between those guests? I use VMXNET3 adapter for communicating between these OSs and E1000 adapter to talk to external world.

When I run iperf TCP performance test I get 3.7 Gbits/sec on the VMXNET3 adapter and 1.6 Gbits/sec on the E1000 adapter. I would have expected VMXNET3 to be up in the 10+ Gbits/sec range.

OS guests are powered by Red Hat Enterprise Linux 6.5, 64 bit, kernel 2.6.32-431.el6.x86_64 under the control of VMware vCenter Server Hypervisor 5.0.0 build 913577. Host server used for benchmarks is IBM x3950 M2 server model 7233-AC1, 24 cores Intel Zeon X7440 2.66 GHz, 256 GB RAM with 1GB Ethernet NICs.

VMware tools installed in guest OSs are v8.6.10 build 913593 (seems to be the latest at the moment).

2 Answers 2

8

That's a really old version of VMware ESXi. The performance does seem to get better with newer revisions, but what you are seeing is probably accurate for that vintage.

Is there any reason your VMware environment hasn't been patched or updated? VMware tools are usually tied to the host version and patches, so you're running 2012-era VMware tools on a 2012-vintage ESXi installation. I have tools version 9.4.0.

You can update the tools independently of the ESXi installation by leveraging the VMware Operating System-Specific Packages (OSPs).

See: How do I make sure VMware Tools is installed and running?

On a modern ESXi 5.5 local copy (same host) between vmxnet3 VMs running EL6.5, I see:

Client:

[root@xt ~]# iperf -t 30 -c mccartney
------------------------------------------------------------
Client connecting to mccartney_mirror, TCP port 5001
TCP window size: 23.2 KByte (default)
------------------------------------------------------------
[  3] local 172.16.2.12 port 35543 connected with 172.16.2.159 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-30.0 sec  90.1 GBytes  25.8 Gbits/sec

Server:

[root@McCartney ~]# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 172.16.2.159 port 5001 connected with 172.16.2.12 port 35543
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-30.0 sec  90.1 GBytes  25.8 Gbits/sec
8
  • 2
    That's about what we get too btw
    – Chopper3
    Mar 7, 2014 at 15:06
  • Thanks for the reply. I have no control over the version of VMware being used. I was fiddling with all kinds of RHEL TCP settings in sysctl.conf, but none of it made positive difference (althought I was able to reduce performance by changing those settings). Mar 7, 2014 at 15:09
  • @romankhar You may not have control over the VMware being used, but it's not reasonable to use such an old version without any patches or improvements. If changing this is not possible, at least try to use the VMware OSP drivers and VMware tools inside of your guests.
    – ewwhite
    Mar 7, 2014 at 15:11
  • @ewwhite VMware docs are confusing as to what Tools version can be installed into which ESX server. Here it says I can only install tools of 5.1 into 5.0 server: pubs.vmware.com/vsphere-55/… Mar 7, 2014 at 17:36
  • 1
    @romankhar I'm sorry. I've retired all of my systems of that vintage. They've been eclipsed by: Nehalem, Westmere, Sandy Bridge and Ivy Bridge CPUs. So your older systems are a few generations behind what's current. I don't have anything of that age to repeat the test with.
    – ewwhite
    Mar 18, 2014 at 13:31
1

Try changing ethernet driver settings.

/sbin/ethtool --offload eth0 gso off tso off sg off gro off 

See

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/ch10s04.html

If you experience low performance with the para-virtualized network drivers, verify the setting for the GSO and TSO features on the host system. The para-virtualized network drivers require that the GSO and TSO options are disabled for optimal performance."

This is about virtio drivers, but I think the same might apply to vmx drivers.

You can also find a lot of similar articles about VMWare: https://info.pleasantsolutions.com/Blog/VMware_Networking_Speed_Issue

13
  • 1
    This should not be necessary. The VMware results posted above were without any modifications to sysctl.conf parameters.
    – ewwhite
    Mar 9, 2014 at 14:20
  • @ewwhite, you never know without testing. See kb.vmware.com/kb/1027511 kb.vmware.com/kb/2032709 Mar 9, 2014 at 15:20
  • kb.vmware.com/kb/2030927 Mar 9, 2014 at 16:05
  • 1
    I understand what you're saying about testing in general. However, those knowledge base articles don't apply to the original poster's environment.
    – ewwhite
    Mar 9, 2014 at 16:07
  • Agree, but that doesn't mean you shouldn't try testing with offload settings disabled. Mar 9, 2014 at 17:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .