We've got a server running Windows Server 2012 r2 with two guests: another Win Server 2012 and Ubuntu Server. We have recently purchased a couple of Nvidia Titan X GPUs to accelerate deep-learning software under Ubuntu Sever, but Hyper-V has no GPU-passthrough option, so I've decided to get rid of the Windows Server host and install Ubuntu. The problem is, I still need the guest Windows machine to work under the new host with RDP access. Any recommendations?
Update 1: I forgot to mention that our machine has 2 physical CPUs. That's important, I guess.
Update 2: After several hours of research, I believe KVM is the way to go. I know it's a bad idea to use the host Ubuntu as a physical machine and a host at the same time, but I see no other option (but purchasing another machine, which is not really an option). There is ESXi with GPU-passthrough, but it has issues with non-server Nvidia GPUs and very weird licensing.
P.S. I know I can install VM Ware ESXi, that supports GPU-passthrough, but people on the web report multiple problems with Nvidia cards.