IP Alias/Virtual Interface for ESXi on KVM/libvirt [closed]

Question

I need to run two ESXi hosts on KVM and assign them distinct, external IPs visible to the external network. Desired topology:

  ESXI     ESXI
   ↓        ↓       
  KVM      KVM
   ↓        ↓
  vIF      vIF
(own ip)  (own ip)
   ↓        ↓
 Physical NIC 
      ↓
 External Net

It's a cinch with other hosts as two bridged Virtual Interfaces with vNIC model set to Virtio device would do the trick. But Virtio as NIC model is not supported by ESXi and it won't boot complaining about no network adapter detected.

Is there a way to make Virtual Interfaces work with ESXi on KVM? I tried to go along with E1000 as NIC but couldn't connect ESXi to external network.

Here's the config I tried. None worked:

<interface type='bridged'>
  <source bridge='br0'/>
  <model type='virtio'/>  #ESXi won't boot due to no adapter detected
</interface>

<interface type='bridged'>
  <source source='br0'  />
  <model type='e1000'/>  #ESXi boots but no network connectivity
</interface>

<interface type='direct'>
  <source dev='br0' mode='bridged' />
  <model type='e1000'/> #ESXi boots but no network connectivity
</interface>

<interface type='direct'>
  <source dev='br0' mode='passthrough' />
  <model type='e1000'/>  #ESXi boots but no network connectivity
</interface>

#Not applicable to my requirements as physical NIC would be passed to KVM
<interface type='direct'>
  <source dev='enp2s0:0' mode='passthrough' />
  <model type='e1000'/>  
</interface>

<interface type='direct'>
  <source dev='br0' mode='bridged' />
  <model type='e1000'/>  #ESXi boots but no network connectivity
</interface>

Here's my interfaces:

 br0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
    inet 192.168.1.10  netmask 255.255.255.0  broadcast 192.168.1.255
    inet6 fe80::b8ca:7bff:fe23:e847  prefixlen 64  scopeid 0x20<link>
    ether ba:ca:7b:23:e8:47  txqueuelen 1000  (Ethernet)
    RX packets 0  bytes 0 (0.0 B)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 4  bytes 300 (300.0 B)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

 enp0s26u1u6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.9  netmask 255.255.255.0  broadcast 192.168.1.255
    inet6 fe80::3be6:18f3:cdd7:d837  prefixlen 64  scopeid 0x20<link>
    ether a0:ce:c8:01:75:ef  txqueuelen 1000  (Ethernet)
    RX packets 25284  bytes 1737100 (1.6 MiB)
    RX errors 0  dropped 6860  overruns 0  frame 0
    TX packets 25208  bytes 29421441 (28.0 MiB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

  enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.6  netmask 255.255.255.0  broadcast 192.168.1.255
    inet6 fe80::ca2d:afad:d824:1205  prefixlen 64  scopeid 0x20<link>
    ether 90:2b:34:9a:bf:67  txqueuelen 1000  (Ethernet)
    RX packets 3375628  bytes 4511208522 (4.2 GiB)
    RX errors 0  dropped 3301  overruns 0  frame 0
    TX packets 1832807  bytes 847946426 (808.6 MiB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    device interrupt 18  

  enp2s0:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.10  netmask 255.255.255.0  broadcast 192.168.1.255
    ether 90:2b:34:9a:bf:67  txqueuelen 1000  (Ethernet)
    device interrupt 18 

Answer 1

It is understandable that virtio-net doesn't work with ESXi as they're unlikely to care about providing drivers for a NIC that's only relevant in nested-virt. The e1000 is a standard NIC type though, so I would fully expect that to work. There's always the possibility that some bug in QEMU's emulation makes it incompatible with ESXi.

The following blog post suggests trying the 'vmxnet3' NIC model with QEMU

https://xrsa.net/2014/08/25/running-vmware-esxi-under-ovirt/

This is the vmware paravirtualized NIC model, which QEMU also supports.

Answer 2

You started out with this configuration, which is otherwise correct but ESXi can't use a virtio NIC:

<interface type='bridged'>
  <source bridge='br0'/>
  <model type='virtio'/>  #ESXi won't boot due to no adapter detected
</interface>

But, for whatever reason, you don't appear to have ever simply changed the NIC model type to e1000 and made no other changes. All of the other settings you have tried have the correct NIC model, but other broken or inappropriate settings.

Of course, you can (and probably should) also just use the vmxnet3 NIC type, which has been supported in libvirt and qemu just fine for many years now.

So, change only the model type, and don't change anything else.