Im trying to pass all gpus to different systems , my setup:

  • asus x399 gaming 6 mainboard
  • amd 1950x
  • 32G ram
  • 1 TB intel ssd m.2
  • gtx970
  • gtx750

this resources should be passed into 3 systems:

  • vm1 a windows 10 VM with the gtx 970
  • vm2 a Ubuntu VM with the gtx 750 (cannot run if vm3 is running)
  • vm3 a Ubuntu VM with the gtx 750 (cannot run if vm2 is running)

but booting with the grub cmd line:

GRUB_CMDLINE_LINUX_DEFAULT="nomodeset amd_iommu=on iommu=pt vfio-pci.ids=10de:13c2,10de:0fbb,10de:1381,10de:0fbc"

does result in a state where the gtx 750 is accessed by the ubuntu server and the guest system ive tried to disable a tty to prevent access but now the guest with the gtx 750 does not pass boot logo

did I miss a point where the host accesses the gtx750? ive searched a couple of times to find a setup matching mine, or matching my goal, but never found a complete solution. if you need any config ill provide them as soon as possible

EDIT#1 ive done some more work on this as well as trying to use an additional GPU(gtx650ti), sadly this card is a defect. I've done some rebinding and unbinding as well ive tried to unbind the EFI framebuffer. but the best effort I could get is that I can decide what to boot with wich GPU, but only one VM at a time.

if I boot more than one VM that uses a GPU the system freezes.

EDIT#2 After a bit of tinkering now the broken card serves as a dummy and i were aled to boot 2 vms with gpu passthrought, as mentiont in the comments ill try to disable gpu boot completly and try if i could get the kernel to boot with an serial vty

EDIT#3 im currently using a the gtx750 to show a menu build in go to allow easy start and stop vm's. im struggeling to get ubuntu to boot without an actual serial port. im trying to use an usb port as serial(shouldn't be that hard). but ubuntu refuses to boot. ill try to get an dummy like a rpi-zero as usb serial master and redirect all io to a screen or similar. if i try to boot with an gpu plugged in the system directly grabs the gpu and does a normal boot.

  • Which GPU did you intend the host to use? – Michael Hampton Jul 1 '20 at 20:14
  • ... i would like to reserve none of the for the host ... in the best case all gpus are passed to a vm – Basur Jul 1 '20 at 20:36
  • 1
    AFAIK you can't pass all GPUs to a VM. The host has to have something for its own display. – Michael Hampton Jul 1 '20 at 20:40
  • Linux itself can boot without a GPU, then the controlling vty is via serial port. If one can configure the kernel not to use GPU but only serial console when booting up, it might work. – Tero Kilkanen Jul 2 '20 at 7:07
  • But it has to be a real serial port on the motherboard or a PCIe card, not a USB serial port. – Michael Hampton Jul 3 '20 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.