Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have 3 Ubuntu servers (each having only 1 NIC) for my openstack cluster configuration following the guide of http://docwiki.cisco.com/wiki/Cisco_OpenStack_Edition:_Folsom_Manual_Install

I also have a Cisco 3750 24 port ethernet switch which I am using as a L3 device in between the Ubuntu servers and another Windows (having 2 NIC) machine as an Internet gateway.

According to the guide, 2 sub-interfaces are created in the single physical NIC of the Ubuntu servers to separate traffics intended for different openstack purpose.

Network topology: http://docwiki.cisco.com/w/images/f/f3/Network-topology-v1.0.png

for example: I have the following vlans in the network

vlan10 -- to communicate between the physical ubuntu servers (needs to be routable towards the internet g/w) - already configured and working OK (switchports were configured as 'access' ports)

vlan101 -- internal data network for the virtual machines (only L2 inter-connectivity is reqiured)

vlan102 -- public network for the virtual machines (needs to be routable towards the internet g/w)

My question is: is it possible to configure the switchport as a trunk interface and allow all the vlans in the individual switch ports so that they can also communicate with each other properly.

Note that, I only have a cisco 3750 switch. No other option.

Thanks a lot ...


Here are configurations:

================================================================================ 01. controller1-01 (Ubuntu Server) /etc/network/interfaces (OpenStack Controller Node)

---------------------------------------

auto lo

iface lo inet loopback

---------------------------------------

auto eth0

iface eth0 inet static

address 192.168.0.29

netmask 255.255.255.224

network 192.168.0.0

broadcast 192.168.0.31

gateway 192.168.0.30

---------------------------------------

auto eth0.101

iface eth0.101 inet static

vlan-raw-device eth0

address 10.0.0.253

netmask 255.255.255.0

---------------------------------------

auto eth0.102

iface eth0.2 inet static

vlan-raw-device eth0

address 192.168.0.157

netmask 255.255.255.224

up route add -net 10.10.10.0/24 gw 192.168.0.131

================================================================================ 02. ns1-01 (Ubuntu Server) /etc/network/interfaces (Quantum Network Node)

---------------------------------------

auto lo

iface lo inet loopback

---------------------------------------

auto eth0

iface eth0 inet static

address 192.168.0.1

netmask 255.255.255.224

network 192.168.0.0

broadcast 192.168.0.31

gateway 192.168.0.30

---------------------------------------

auto eth0.101

iface eth0.101 inet static

vlan-raw-device eth0

address 10.0.0.1

netmask 255.255.255.0

---------------------------------------

auto eth0.102

iface eth0.102 inet manual

vlan-raw-device eth0

up ifconfig $IFACE 0.0.0.0 up

up ip link set $IFACE promisc on

down ifconfig $IFACE down

================================================================================ 03. ns1-02 (Ubuntu Server) /etc/network/interfaces (OpenStack Compute Node)

---------------------------------------

auto lo

iface lo inet loopback

---------------------------------------

auto eth0

iface eth0 inet static

address 192.168.0.2

netmask 255.255.255.224

network 192.168.0.0

broadcast 192.168.0.31

gateway 192.168.0.30

---------------------------------------

auto eth0.101

iface eth0.101 inet static

vlan-raw-device eth0

address 10.0.0.2

netmask 255.255.255.0

================================================================================ 04. LS1-01 (Ubuntu Server) /etc/network/interfaces (OpenStack Compute Node)

---------------------------------------

interface GigabitEthernet1/0/1

description //Connected to ns1-01 (Region One Network Node)

switchport trunk encapsulation dot1q

switchport trunk native vlan 10

switchport trunk allowed vlan 10,101,102

switchport mode trunk

---------------------------------------

interface GigabitEthernet1/0/2

description //Connected to ns1-02 (Region One Compute Node)

switchport trunk encapsulation dot1q

switchport trunk native vlan 10

switchport trunk allowed vlan 10,101,102

switchport mode trunk

---------------------------------------

interface GigabitEthernet1/0/11

description //Connected to controller1-01 (Region One Controller Node)

switchport trunk encapsulation dot1q

switchport trunk native vlan 10

switchport trunk allowed vlan 10,101,102

switchport mode trunk

---------------------------------------

interface GigabitEthernet1/0/12

description //Connected to Internet

no switchport

ip address 192.168.0.65 255.255.255.252

---------------------------------------

interface Vlan10

description //Management network for Physical Machines under Region One

ip address 192.168.0.30 255.255.255.224

---------------------------------------

interface Vlan101

description //Internal data network for VM instances under Region One

ip address 10.0.0.254 255.255.255.0

---------------------------------------

interface Vlan102

description //Public/API network for VM instances under Region One

ip address 192.168.0.158 255.255.255.224

---------------------------------------

ip route 0.0.0.0 0.0.0.0 192.168.0.66 # Towards Internet G/W

share|improve this question

1 Answer 1

up vote 3 down vote accepted

Absolutely you can. You're looking for 802.1q tagging, which is the underpinning of multiple VLANs on a switch.

Configure your server-facing ports like this...

interface GigabitEthernet0/1
 ! tell the switch to encapsulate frames with dot1q on this trunk port
 switchport trunk encapsulation dot1q
 ! make this a trunk port unconditionally
 switchport mode trunk
 ! allow VLANs 220, 221, and 223 to be sent on this port
 switchport trunk allowed vlan 220,221,223
 ! send vlan 221 frames untagged
 ! you could omit this command, but you'll need to move your eth0 config to a subinterface as well
 switchport trunk native vlan 221
 ! this is an end-device, and doesn't need to be treated like a switch for spanning-tree
 spanning-tree portfast trunk
 ! remove the (ignored) access vlan setting, as this is now a trunk
 no switchport access vlan

On your Ubuntu server you'll need to configure the subinterfaces to be associated to each appropriate VLAN. This can be done manually like this ...

# create a new subinterface tied to dot1q vlan 221
ip link add link eth0 name eth0.221 type vlan id 221
# change the MTU to allow for the inserted dot1q tag (4 bytes)
ifconfig eth0 mtu 1504
# bring the new interface up
ifconfig eth0.221 up

Now you can associate the correct IP address to new interface eth0.221, along with the appropriate default gateway and subnet mask.

If this doesn't work for you, please include a pastebin link to your switch config as well as the output of "ip -d link show" on your Ubuntu server.

Hope this helps!

share|improve this answer
    
Kudos, great answer. –  SpacemanSpiff Apr 7 '13 at 22:27
    
I believe he'll have to do "modprobe 8021q" to load vlan tagging support in Ubuntu, I don't think it's loaded by default. –  Lorin Hochstein Apr 7 '13 at 23:19
    
Shouldn't need to. I just tried the 'ip link add link' command on my Linux Mint 14 install that I did a couple weeks ago, and it worked fine. I believe it's loaded by default in Ubuntu. –  Keller G Apr 8 '13 at 0:02
    
Does the above answer I gave not work for you? Your server configs should work okay -- Linux automatically uses the subinterface number to assign it to a VLAN. I haven't used it exactly this way, but based on some quick research your hosts should be fine. Were you able to get the switch config for us? –  Keller G Apr 8 '13 at 0:40
    
Here is the pastebin link for the lan switch configuration. pastebin.com/VkZh3naS I am configuring the network node ns1-01 again. And can confirm the communication between the nodes through all these vlan afterward. Many thanks for your answer :) –  jmkam Apr 8 '13 at 1:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.