I have two containers running (A and B) inside a Virtual Machine (Ubuntu 18.04). For network simulation purposes I am routing packets from A to B (this part works). What I want to do is to allow A and B to communicate with the outside world. Thus I would have :
A ---- B ---- Host VM --- Host --- Internet
I've done this so far :
ip link add name br_routing type bridge ip link set dev br_routing up
Then add another virtual interface so container B has an interface to reach host vm network :
ip link add vm_side_iface type veth peer name containerb_side_iface
Add interface to the bridge
ip link set dev vm_side_iface master br_routing
Set ips for bridge and interface inside container
ip addr add 192.168.10.1/24 dev br_routing ip addr add 192.168.10.2/24 dev vm_side_iface
Activate interface on vm side
ip link set vm_side_iface up ip link set containerb_side_iface netns $container_pid
Then inside container B network namespace :
ip addr add 192.168.10.2/24 broadcast 192.168.10.255 dev containerb_side_iface ip route add default via 192.168.10.1 dev containerb_side_iface
At this point I'm able to ping A and B from the host virtual Machine
However I cannot ping beyond the bridge from A or B. The error seems to come from routing so here it is : B routing Table :
10.11.1.0/24 dev eth0 proto kernel scope link src 10.11.1.2 10.22.0.0/16 via 10.11.1.1 dev eth0 192.168.10.0/24 dev containerb_side_iface proto kernel scope link src 192.168.10.2
host VM routing Table :
default via 10.0.2.2 dev enp0s3 proto dhcp metric 100 10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15 metric 100 #Irrelevant here 10.11.0.0/16 via 192.168.10.2 dev br_routing #Route to reach B 10.22.0.0/16 via 192.168.10.2 dev br_routing #Route to reach A 192.168.10.0/24 dev br_routing proto kernel scope link src 192.168.10.1 #This seems odd 192.168.10.0/24 dev router_0 proto kernel scope link src 192.168.10.2 #This seems odd
My second guess is my routing tables, this is what I've tried :
iptables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE iptables -A FORWARD -i enp0s3 -o vm_side_iface, -m state --state RELATED,ESTABLISHED -j ACCEPT iptables -A FORWARD -i vm_side_iface, -o enp0s3 -j ACCEPT iptables -A FORWARD -i enp0s3 -o vm_side_iface, -j ACCEPT
I also tried replacing vm_side_iface by br_routing but it would not work either way. Moreover, the rules are not used (0 accepted packets, 0 dropped). These are the only rules that are configured in the VM. Regardless, the ping from the Host VM to A/B works without these rules.
Finally, modprobe br_netfilter and flushing files inside /etc/sys/net/bridge does not change anything.
What does not work :
ping from A or B to HOST VM ping from A or B to vm_side_iface ping from A or B to br_routing
What works :
ping from HOST VM to A or B
I've noticed that the messages get to the bridge (tcpdump -i br_routing) but are not forwarded to the enp0s3 interface.
Any guess ?