Im trying to pass all gpus to different systems , my setup:
- asus x399 gaming 6 mainboard
- amd 1950x
- 32G ram
- 1 TB intel ssd m.2
this resources should be passed into 3 systems:
- vm1 a windows 10 VM with the gtx 970
- vm2 a Ubuntu VM with the gtx 750 (cannot run if vm3 is running)
- vm3 a Ubuntu VM with the gtx 750 (cannot run if vm2 is running)
but booting with the grub cmd line:
GRUB_CMDLINE_LINUX_DEFAULT="nomodeset amd_iommu=on iommu=pt vfio-pci.ids=10de:13c2,10de:0fbb,10de:1381,10de:0fbc"
does result in a state where the gtx 750 is accessed by the ubuntu server and the guest system ive tried to disable a tty to prevent access but now the guest with the gtx 750 does not pass boot logo
did I miss a point where the host accesses the gtx750? ive searched a couple of times to find a setup matching mine, or matching my goal, but never found a complete solution. if you need any config ill provide them as soon as possible
EDIT#1 ive done some more work on this as well as trying to use an additional GPU(gtx650ti), sadly this card is a defect. I've done some rebinding and unbinding as well ive tried to unbind the EFI framebuffer. but the best effort I could get is that I can decide what to boot with wich GPU, but only one VM at a time.
if I boot more than one VM that uses a GPU the system freezes.
EDIT#2 After a bit of tinkering now the broken card serves as a dummy and i were aled to boot 2 vms with gpu passthrought, as mentiont in the comments ill try to disable gpu boot completly and try if i could get the kernel to boot with an serial vty
EDIT#3 im currently using a the gtx750 to show a menu build in go to allow easy start and stop vm's. im struggeling to get ubuntu to boot without an actual serial port. im trying to use an usb port as serial(shouldn't be that hard). but ubuntu refuses to boot. ill try to get an dummy like a rpi-zero as usb serial master and redirect all io to a screen or similar. if i try to boot with an gpu plugged in the system directly grabs the gpu and does a normal boot.