We have 5 physical machines hosted at OVH. Each of them have Proxmox 5.2-10 installed Machines are inside of OVH vRack. we have separate LXC for database, separate LXC with loadbalancer and several separate instances with our web application stack (php and nginx).
When we put more LXCs with our web stack on a single physical machine after a while all of them except one just stuck. PHP nor nginx are not responding on the network. Load average on the physical server suddenly starts to grow. It is possible to enter the LXC via ssh or pct enter but impossible to shutdown from Proxmox web interface, LXC process has to be forcefully killed.
This problem started to appear when we upgraded PHP from 7.0 to 7.2. I don't think that it could be somehow related but I've never experienced simillar behaviour before. So far we weren't able to come up with any solution except keeping the web LXCs on separate machines. LXCs which do not have our web app stack can be normally together on one physical machine.
We believe that it has to do something with networking. Some files are shared via NFS on all LXCs. Could this be a hint? But again the problem is only on web LXCs. Other container with shared files can normaly work together.
Example configration of LXC network interfaces:
auto lo iface lo inet loopback iface lo inet6 loopback # interfaces(5) file used by ifup(8) and ifdown(8) # Include files from /etc/network/interfaces.d: source-directory /etc/network/interfaces.d auto eth0 iface eth0 inet static address 126.96.36.199 netmask 255.255.255.224 gateway 188.8.131.52 auto eth1 iface eth1 inet static address 172.16.32.172 netmask 255.240.0.0
Example configuration of host machine interfaces:
auto lo iface lo inet loopback iface eno1 inet manual auto eno2 iface eno2 inet manual auto vmbr0 iface vmbr0 inet static address 184.108.40.206 netmask 255.255.255.0 gateway 220.127.116.11 bridge-ports eno1 bridge-stp off bridge-fd 0 iface vmbr0 inet6 static address 2001:41d0:303:2dbb:: netmask 64 auto vmbr1 iface vmbr1 inet static address 172.16.35.1 netmask 255.240.0.0 broadcast 172.31.255.255 bridge-ports eno2 bridge-stp off bridge-fd 0 network 172.16.0.0