2

I'm trying to get Bridged Networking to work on Ubuntu in VirtualBox, on a Windows Server 2016 host which is running as guest on an ESXI system (i.e. nested virtualization: ESXI -> Windows Server running Virtualbox -> Ubuntu).

My VMware ESXI installation:

  • ESXI 6.5.0 Update 1 (Build 5969303) running on an Intel Xeon
  • "Expose hardware assisted virtualization to the guest OS" enabled for my guests
  • All VLANs come in as 1 trunk over a physical network card (igbn driver)
  • Port groups are created per VLAN and are passed to my guests; in this case my guest is using 4x e1000 network cards with a different VLAN assigned per card
  • Pretty standard guest options (2 vCPUs, 8GB RAM, 50GB HDD, etc.)

My Windows Server 2016 install (as guest on ESXI):

  • Following config is used for Windows Server 2016 on ESXI:
ethernet4.pciSlotNumber 1216
featMask.vm.hv.capable  Min:1
ethernet3.pciSlotNumber 1184
tools.guest.desktop.autolock    FALSE
nvram   N.nvram
pciBridge0.present  TRUE
svga.present    TRUE
pciBridge4.present  TRUE
pciBridge4.virtualDev   pcieRootPort
pciBridge4.functions    8
pciBridge5.present  TRUE
pciBridge5.virtualDev   pcieRootPort
pciBridge5.functions    8
pciBridge6.present  TRUE
pciBridge6.virtualDev   pcieRootPort
pciBridge6.functions    8
pciBridge7.present  TRUE
pciBridge7.virtualDev   pcieRootPort
pciBridge7.functions    8
hpet0.present   TRUE
RemoteDisplay.maxConnections    -1
sched.cpu.latencySensitivity    normal
sata0:0.autodetect  TRUE
numa.autosize.vcpu.maxPerVirtualNode    2
numa.autosize.cookie    20001
sched.swap.derivedName  /vmfs/volumes/5b8555ab-94e03f86-bf5b-e0d55e5a526b/N/N-d4a539e0.vswp
pciBridge0.pciSlotNumber    17
pciBridge4.pciSlotNumber    21
pciBridge5.pciSlotNumber    22
pciBridge6.pciSlotNumber    23
pciBridge7.pciSlotNumber    24
scsi0.pciSlotNumber 160
usb.pciSlotNumber   32
ethernet0.pciSlotNumber 192
ethernet1.pciSlotNumber 224
ethernet2.pciSlotNumber 256
ehci.pciSlotNumber  33
vmci0.pciSlotNumber 34
sata0.pciSlotNumber 35
scsi0.sasWWID   50 05 05 64 2e 0d f7 a0
ethernet0.generatedAddressOffset    0
ethernet1.generatedAddressOffset    10
ethernet2.generatedAddressOffset    20
vm.genid    8985267772027956372
vm.genidX   7658496339281961829
monitor.phys_bits_used  43
vmotion.checkpointFBSize    4194304
vmotion.checkpointSVGAPrimarySize   4194304
softPowerOff    FALSE
usb:1.speed 2
usb:1.present   TRUE
usb:1.deviceType    hub
usb:1.port  1
usb:1.parent    -1
toolsInstallManager.lastInstallError    0
tools.remindInstall FALSE
toolsInstallManager.updateCounter   5
svga.guestBackedPrimaryAware    TRUE
ethernet3.generatedAddressOffset    30
usb:0.present   TRUE
usb:0.deviceType    hid
usb:0.port  0
usb:0.parent    -1
ethernet4.generatedAddressOffset    40
guestinfo.driver.vmci.version   9.8.16.0
guestinfo.driver.vmxnet3.version    1.8.16.0
vmware.tools.internalversion    10279
vmware.tools.requiredversion    10279
migrate.hostLogState    none
migrate.migrationId 0
migrate.hostLog ./N-d4a539e0.hlog
  • 4 Ethernet ports (passed from e1000 in ESXI), each identified as Intel 82574L. All these ports work perfectly fine (get an IP address, can communicate outside from the Windows server).
  • No specific firewalling software
  • Running VirtualBox 6.0.14 (up-to-date) but tried with older release as well earlier this year
  • Fully updated to the latest Windows patches

My Ubuntu install (as guest on VirtualBox):

  • Ubuntu Server 18.04 LTS x64 but have tried with other editions as well. I'm just trying to get an IP address through the Ubuntu installer for now, but have tried a full install, too. The Ubuntu guest does 'see' the adaptor but doesn't get an IP address while there's a DHCP server running on the VLAN the guest is provided with (the Windows 2016 host does get an IP address on the same interface).

While debugging with Wireshark on the Windows 2016 machine, I can see the Ubuntu guest is using the proper 'public internet' Ethernet adapter and sends DHCP requests on this interface. I'm sure it's the Ubuntu guest sending ARPs as the MAC address matches with what I set in the VirtualBox options (I tried with default random MAC as well). When setting a fixed IP address on Ubuntu, it broadcasts ARP requests to find the gateway to no avail.

I've tried:

  • Using each of the n=6 different adaptor options VirtualBox offers, including the Paravirtualization one
  • Uninstalling VirtualBox and re-installing (paying close attention to the 'Bridged Networking' option is enabled)
  • Adding a new adaptor to the Windows 2016 machine specifically for VirtualBox usage
  • Disabling all properties for the aforementioned adaptor, except for the NDIS6 Networking Driver
  • Changing the network adaptor to the Win 2016 machine from e1000 to vmxnet3
  • Adding a 'Host-Only Ethernet Adapter' to the Ubuntu guest (as suggested here)
  • Running VirtualBox in Windows 2008 compatibility mode (as suggested here)
  • Enabling Promiscuous mode ('Allow All') on VirtualBox
  • Verifying I'm running everything as Administrator (I am)
  • Verifying the Windows 2016 Firewall is turned off (it is)
  • Running everything in VMware Workstation (same problem)

Any clue what the issue can be due to or what to try next?

1 Answer 1

2

Both Promiscuous mode and Forged transmits need to be enabled in ESXI on the port configuration provided to the guest.

Port group config in ESXI

On the VirtualBox guest, Promiscious mode is not required.

More information on Forged transmits:

A Forged Transmit occurs when a network adapter starts sending out traffic that identifies itself as someone else. This security policy compares the Effective Address of the virtual network adapter and the source address inside of an 802.3 Ethernet frame generated by the virtual machine to make sure they match. If they don’t match, a Forged Transmit has occurred – sometimes this is known as MAC impersonation. If the security policy for Forged Transmits is set to Reject, the frame will be dropped.

And on Promiscuous mode:

In a Nested ESXi environment where you can have Nested Virtual Machines, the destination MAC Address for network packets destined to those Virtual Machines will differ from the Nested ESXi vmnic's MAC Address. Due to this, the physical ESXi host's virtual switch will drop the packet if Promiscuous Mode is not enabled. Promiscuous Mode allows the underlying Nested ESXi VM vmnic to monitor all traffic of the virtual switch it is connected to and thus providing connectivity to the underlying Nested Virtual Machines.

Note the potential performance hit when enabling Promiscuous mode.

1
  • Can the person who downvoted this answer explain why? Since I described exactly the solution to my problem. It is confirmed to work.
    – SaeX
    Dec 10, 2019 at 12:12

Not the answer you're looking for? Browse other questions tagged .