ESXi USB PCIe Passthrough Failing to Connect Connected Devices

Question

I have a fresh install of ESXi 6.7U2 up and running on which I can successfully start VMs. To this setup I've added a StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V). This has been configured for pass through following https://kb.vmware.com/s/article/1010789. This can be configured for passthrough to a VM:

The USB PCIe controllers are detected here as uPD720202 USB 3.0 Host Controller. A single PCIe adaptor has 4 independent controllers and I've chosen to pass then all through for simplicity while trying to debug this issue.

I have tried using both Amazon Linux 2 (on prem) and Ubuntu Server 18.04.2 for the VMs, what follows appears to be equivalent in both of them.

lspci output shows the expected pass through devices:

00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:01.0 SATA controller: VMware SATA AHCI controller
03:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
04:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
0b:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)
13:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)
1b:00.0 USB controller: Renesas Technology Corp. uPD720202 USB 3.0 Host Controller (rev 02)

lsusb output before connecting a device to the adaptor:

Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Viewing dmesg output when trying to connect a device, depending on device (USB hub, USB data drive etc.), I either get no additional output or:

[  818.213414] xhci_hcd 0000:04:00.0: Abort failed to stop command ring: -110
[  818.213435] xhci_hcd 0000:04:00.0: xHCI host controller not responding, assume dead
[  818.213453] xhci_hcd 0000:04:00.0: HC died; cleaning up
[  818.213467] xhci_hcd 0000:04:00.0: WARNING: Host System Error
[  818.213497] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[  818.213511] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[  818.213526] usb usb4-port1: couldn't allocate usb_device
[  818.213538] xhci_hcd 0000:04:00.0: Error while assigning device slot ID
[  818.213551] xhci_hcd 0000:04:00.0: Max number of devices this xHCI host supports is 32.
[  818.213565] usb usb3-port1: couldn't allocate usb_device

Clearly a device is being detected in the VM but something is preventing it from connecting successfully.

Things I tried (Ref https://bbs.archlinux.org/viewtopic.php?id=236536):

1. Setting the kernel parameters iommu=soft pci=nomsi
2. Setting the kernel parameter intel_iommu=off
3. Different VMs (as mentioned above)
4. Ensuring the USB devices are connected on VM boot
5. Setting the ESXi host power management to 'High performance'

To confirm I wasn't dealing with a hardware/power issue I ditched the hypervisor on the same hardware by performing a boot from USB (LinuxMint). Here the PCIe adaptor worked correctly with all connected devices so I think that can be ruled out.

At the moment my suspicions are:

1. This uPD720202 chipset isn't compatible with ESXi, even though many posts exist online suggesting it should work fine (which is why I bought it).
2. I'm missing some critical configuration within the ESIx host.

If anyone spots anything obvious that I could try, or would like additional output that would be greatly appreciated.

Answer 1

Looks like it was option 1, hardware compatibility.

I swapped the StarTech.com 4 Port USB 3.0 Card Adapter (PEXUSB3S44V) for a 4 Port HighPoint RocketU RU1144D which now works as expected.

Answer 2

VMware's guidance to customers is strongly in favor of USB over IP because of vMotion. Direct passthrough of USB host controllers is not encouraged because the VM is then tied to a physical host. Is there a reason you can't pass through the USB devices attached to the StarTech or RocketU USB ports? No driver for the USB device will be needed in ESXi, vMotion should "just work" and VMware will be much more likely to respond to a support request. Of course if your host needs to be taken down the VMs, wherever they are running, will temporarily lose the device(s), but that's probably better than taking the VMs down. JMHO and glad you got something to work.

It's not the case here but just to be clear to others, passthrough of motherboard host controllers is unsupported when ESXi boots from USB or uses USB as its local storage (e.g., for a coredump file, which configuration causes a file handle to be opened at boot). See https://kb.vmware.com/s/article/2068645.