1

note: I still have been unable to get this config to work, but will post my solution as soon as I figure it out.

I am setting an Ubuntu host server (16.04) with a number of KVMs running on it. The host server has 4 physical NICs which are bonded in pairs to access a public facing network, and a private network. The host has a primary public subnet assigned to it (with only one usable external IP) and a primary private subnet (with one usable internal IP). Additionally I have acquired two additional 8 IP subnets, one public and one private for the virtual machines on this host. For example here is my IP layout:

FOR HOST:

  • Public Primary Subnet: 200.0.0.72/29 - Usable Public IP: 200.0.0.75
  • Private Primary Subnet: 10.0.0.128/26 - Usable Private IP: 10.0.0.138

FOR VMS

  • Public Portable Subnet: 200.0.1.240/29, 5 Usable Public IPs
  • Private Portable Subnet: 10.0.1.216/29, 5 Usable Private IPs

I am having a lot of trouble getting the host to have a separate subnet from my bridge interfaces for the VMs. It seems I can't get the host to network correctly when I have bridges specified, and bridges only work, when I do not specify a network in the host area on the bonded interface. My interfaces file is below. What you see here is my "desired" outcome, and doesn't actually work. I've been able to get pieces to work by commenting different portions and restarting networking. I also added routing commands (route add) per instructions from my ISP, but they have not been able to help with the bridging aspect and the VMs.

############################################
#For more information, see interfaces(5).
############################################
##Loopback
auto lo
iface lo inet loopback

## Setup bond0 interface - INTERNAL CONNECTIONS
auto bond0
iface bond0 inet static
bond-lacp-rate 1
#pre-up /sbin/ethtool -s bond0 speed 100 duplex full autoneg off
post-up ifenslave bond0 eth0 eth2
pre-down ifenslave -d bond0 eth0 eth2
address 10.0.0.138
netmask 255.255.255.192
gateway 10.0.0.129
bond-slaves none
bond-mode 4
bond-lacp-rate fast
bond-miimon 100
bond-downdelay 0
bond-updelay 0
bond-xmit_hash_policy 1
up route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.0.0.129

auto eth0
iface eth0 inet manual
bond-master bond0
#pre-up /sbin/ethtool -s eth0 speed 100 duplex full autoneg off

auto eth2
iface eth2 inet manual
bond-master bond0
#pre-up /sbin/ethtool -s eth2 speed 100 duplex full autoneg off

auto br0
iface br0 inet static
address 10.0.1.218
netmask 255.255.255.192
gateway 10.0.1.217
bridge_ports bond0
bridge_stp off
bridge_fd 9
bridge_hello 2
bridge_maxage 12

## Setup bond1 interface - EXTERNAL CONNECTIONS
auto bond1
iface bond1 inet static
bond-lacp-rate 1
post-up ifenslave bond1 eth1 eth3
pre-down ifenslave -d bond1 eth1 eth3
bond-slaves none
bond-mode 4
bond-lacp-rate fast
bond-miimon 100
bond-downdelay 0
bond-updelay 0
bond-xmit_hash_policy 1
address 200.0.0.75
netmask 255.255.255.248
gateway 200.0.0.73
up route add -net 0.0.0.0 netmask 0.0.0.0 gw 200.0.0.73

auto eth1
iface eth1 inet manual
bond-master bond1

auto eth3
iface eth3 inet manual
bond-master bond1

auto br1
iface br1 inet static
address 200.0.1.242
netmask 255.255.255.248
gateway 200.0.1.241
#up iptables-restore < /etc/iptables.rules
post-up echo 0 > /sys/devices/virtual/net/br1/bridge/multicast_snooping
bridge_ports bond1
bridge_stp off
bridge_fd 9
bridge_hello 2
bridge_maxage 12

If I already have a subnet on my host (on the bond interface), I may be wrong in specifying a specific address on the bridged interface, but then Im not sure how to specify a separate subnet for the VMS (on bridge) without doing that. Thanks in advance for any comments/suggestions as to what Im doing wrong here.

#Addition 1: VLANS

Below are the subnets for the VLANs given to me by my ISP. The "Primary IP for future server only" is for a primary IP given to another customer/host, and not accessible by me. Additionally there are two primary private IPs, one is accessible to my host OS and the other is a management interface for console access (via the bios it looks like).

There are two VLANs, one public and one private, and they each are numbered and have a router associated with them. I.e VLAN number 1200 router zzz02a.bbb03 for public network, and a similar one for private.

Public Primary Subnet

200.0.0.72  Reserved    Network
200.0.0.73  Reserved    Gateway
200.0.0.74  Reserved    Primary IP for future server only
200.0.0.75  In Use  host.mydomain.com
200.0.0.76  Reserved    Primary IP for future server only
200.0.0.77  Reserved    Primary IP for future server only
200.0.0.78  Reserved    Primary IP for future server only
200.0.0.79  Reserved    Broadcast

Public Portable Subnet

200.0.1.240 Reserved    Network
200.0.1.241 Reserved    Gateway
200.0.1.242 Available   vm1.mydomain.com
200.0.1.243 Available   <<available for vm>>
200.0.1.244 Available   <<available for vm>>
200.0.1.245 Available   <<available for vm>>
200.0.1.246 Available   <<available for vm>>
200.0.1.247 Reserved    Broadcast

Private Primary Subnet

10.0.0.128  Reserved    Network
10.0.0.129  Reserved    Gateway
10.0.0.130  Reserved    Primary IP for future server only
10.0.0.131  Reserved    Primary IP for future server only
10.0.0.132  Reserved    Primary IP for future server only
10.0.0.133  Reserved    Primary IP for future server only
10.0.0.134  Reserved    Primary IP for future server only
10.0.0.135  Reserved    Primary IP for future server only
10.0.0.136  Reserved    Primary IP for future server only
10.0.0.137  Reserved    Primary IP for future server only
10.0.0.138  In Use  host.mydomain.com
10.0.0.139  Reserved    Primary IP for future server only
10.0.0.140  Reserved    Primary IP for future server only
10.0.0.141  Reserved    Primary IP for future server only
10.0.0.142  Reserved    Primary IP for future server only
10.0.0.143  Reserved    Primary IP for future server only
10.0.0.144  Reserved    Primary IP for future server only
10.0.0.145  Reserved    Primary IP for future server only
10.0.0.146  Reserved    Primary IP for future server only
10.0.0.147  Reserved    Primary IP for future server only
10.0.0.148  Reserved    Primary IP for future server only
10.0.0.149  In Use  host.mydomain.com
10.0.0.150  Reserved    Primary IP for future server only
10.0.0.151  Reserved    Primary IP for future server only
10.0.0.152  Reserved    Primary IP for future server only

Private Portable Subnet

10.0.1.216  Reserved    Network
10.0.1.217  Reserved    Gateway
10.0.1.218  Available   vm1.mydomain.com
10.0.1.219  Available   <<available for vm>>
10.0.1.220  Available   <<available for vm>>
10.0.1.221  Available   <<available for vm>>
10.0.1.222  Available   <<available for vm>>
10.0.1.223  Reserved    Broadcast
  • How exactly are these single IP addresses and /29 subnets delivered to you by your ISP? Are they delivered in separate VLANs? – Tero Kilkanen Mar 25 '17 at 2:43
  • Yes, VLANs, primary VLANs are shared with other servers/customers, and I have one of the addresses, and the portable /29 VLANs are all mine. – hmorton Mar 25 '17 at 7:03
  • Are all the VLANs tagged or is the primary VLAN default/untagged VLAN? – Tero Kilkanen Mar 25 '17 at 12:38
  • Can you define tagged vs untagged VLANs? Im not sure what those are. – hmorton Mar 25 '17 at 21:18
  • A tagged VLAN is a VLAN with a VLAN header, which contains the VLAN ID, which is between 1 and 4095, while untagged VLAN has no VLAN header. – Tero Kilkanen Mar 26 '17 at 1:06
0

I think there are no vlans. If you want to share all 4 subnets​ to your virtual machines you should assign them on the bridges. The interfaces file should be:

############################################
#For more information, see interfaces(5).
############################################
##Loopback
auto lo
iface lo inet loopback

## Setup bond0 interface - INTERNAL CONNECTIONS
auto bond0
iface bond0 inet manual
bond-slaves eth0 eth2
bond-mode 4
bond-lacp-rate fast
bond-miimon 100
bond-downdelay 0
bond-updelay 0
bond-xmit_hash_policy 1

auto eth0
iface eth0 inet manual
bond-master bond0

auto eth2
iface eth2 inet manual
bond-master bond0

## Setup VLAN ID 1201 on bond0 interface
auto bond0.1201
iface bond0.1201 inet manual
vlan_raw_device bond0

auto br0
iface br0 inet static
address 10.0.0.138
netmask 255.255.255.192
bridge_ports bond0.1201
bridge_stp off
up route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.0.0.129

auto br0:1
iface br0:1 inet static
address 10.0.1.218
netmask 255.255.255.192

## Setup bond1 interface - EXTERNAL CONNECTIONS
auto bond1
iface bond1 inet manual
bond-slaves eth1 eth3
bond-mode 4
bond-lacp-rate fast
bond-miimon 100
bond-downdelay 0
bond-updelay 0
bond-xmit_hash_policy 1

auto eth1
iface eth1 inet manual
bond-master bond1

auto eth3
iface eth3 inet manual
bond-master bond1

## Setup VLAN ID 1200 on bond1 interface
auto bond1.1200
iface bond1.1200 inet manual
vlan_raw_device bond1

auto br1
iface br1 inet static
address 200.0.0.75
netmask 255.255.255.248
gateway 200.0.0.73
post-up echo 0 > /sys/devices/virtual/net/br1/bridge/multicast_snooping
bridge_ports bond1.1200
bridge_stp off

auto br1:1
iface br1:1 inet static
address 200.0.1.242
netmask 255.255.255.248
  • This is a great possible solution, and Im eager to try it shortly. I am not necessarily trying to bridge the primary subnet ... those single IP addresses (public and private) Im trying to keep for the host. The other subnet would be bridged. – hmorton Mar 25 '17 at 21:19
  • A bridge interface is the same as virtual unmanaged switch. – Mikhail Khirgiy Mar 26 '17 at 3:54
  • One thing about this solution, according to the ISP these subnets are VLANs, does that make a difference to your solution above? Also, is there a way to reserve the primary IPs (public and private) just for the host machine? It is my understanding that if I bridge those addresses, they would be accessible to the VMs as well. – hmorton Mar 26 '17 at 17:26
  • Yes, vlan usage gives you needing to change the configuration file. Your server has got all ip addresses (not all networks) which was written in the interfaces configuration file. Please update your question to describe what vlan number (tag) has each network. Also add all networks, which clients want to have access to your server from and what vlan will be used for it. – Mikhail Khirgiy Mar 26 '17 at 18:14
  • I posted my VLAN information above - let me know what you think – hmorton Mar 27 '17 at 7:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.