On my HP Proliant microserver gen 8 box I have Ubuntu 18.04 running as host OS. Initially I was planning to setup one production VM (vm-prod) with multiple services inside lxd containers (vm-prod-lxd-01, vm-prod-lxd-02 etc.) inside. Another VM (vm-dev) would have same structure (vm-dev-lxd-01, 02, etc) but will be used as staging environment to test things before they were deployed into new prod containers. I also planned to spin some more VMs to test out completely new architectures.
When I deployed the production VM all went fine and I got acceptable performance. But when I setup second (dev) VM, performance hit was so severe, even the ssh sessions in terminal across all machines were lagging. I suspect it might be related to my particular network setup. Since all of the systems (host, VM, container) are Ubuntu 18.04, I paste netplan settings below:
network: version: 2 renderer: networkd ethernets: eno1: dhcp4: no dhcp6: no bridges: br0: interfaces: - eno1 dhcp4: no dhcp6: no addresses: [192.168.1.100/24] gateway4: 192.168.1.200 nameservers: addresses: [192.168.1.200]
network: version: 2 renderer: networkd ethernets: ens3: dhcp4: no dhcp6: no bridges: br0: interfaces: - ens3 dhcp4: no dhcp6: no addresses: [192.168.1.110/24] gateway4: 192.168.1.200 nameservers: addresses: [192.168.1.200]
network: ethernets: eth0: addresses: - 192.168.1.120/24 dhcp4: false gateway4: 192.168.1.200 nameservers: addresses: - 192.168.1.200 version: 2
I've chosen to use bridge networking to have separate IPs on same network subnet for all machines. Almost all services are inside prod VM lxd containers, and they are, with exception of one xmpp server, webapps.
The question is:
- Was the choice of nesting bridged interface a poor one in terms of I/O? If not, what may cause said performance issues? What tool should I use to measure it?
- If bridged interface is not good performance-wise, what type of configuration should I use instead? IP tables with port forwarding? Lxd port forwarding?
- The server has two NICs - is using another bridged interface for other VMs a solution?