Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have Hardware Node (HN) which has 2 physical interfaces (eth0, eth1). I'm playing with OpenVZ and want to let my containers (CTs) have access to both of those interfaces. I'm using basic configuration - venet. CTs are fine to access eth0 (public interface). But I can't get CTs to get access to eth1 (private network). I tried:

# on HN
vzctl set 101 --ipadd 192.168.1.101 --save
vzctl enter 101
ping 192.168.1.2 # no response here
ifconfig # on CT returns lo (127.0.0.1), venet0 (127.0.0.1), venet0:0 (95.168.xxx.xxx), venet0:1 (192.168.1.101)

I believe that the main problem is that all packets flows through eth0 on HN (figured out using tcpdump). So the problem might be in routes on HN.

Or is my logic here all wrong? I just need access to both interfaces (networks) on HN from CTs. Nothing complicated.

share|improve this question

4 Answers 4

Same problem, but different solution. The two ports were not connected to the same network and needed to appear from the IP address of the virtual machine, so masquerading did not work.

The main issue here is that the openvz container sets the subnet of all of the ips on venet to 255.255.255.255. There is no preference of one interface. There is no preference on which router it should go through, so it sometimes uses eth0, and sometimes uses eth1. The result was random failures for certain IP addresses when the request goes out on the wrong interface.

One solution was to add a route that specified the source like so:

ip route add 10.20.0.0/16 dev venet0 src 10.20.0.xxx
ip route add a.b.c.241/24 dev venet0 src a.b.c.xxx

I found that the simplest solution for now was to set set the subnets just after they've been brought up (on an ubuntu/debian container in /etc/network/if-up.d):

#!/bin/sh
if [ "$IFACE" = "venet0:1" ]; then
        ifconfig venet0:1 netmask 255.255.0.0 up
fi
if [ "$IFACE" = "venet0:0" ]; then
        ifconfig venet0:0 netmask 255.255.255.0 up
fi
exit 0

Both solutions should have the same affect. Both solutions makes me a little concerned that when accessing the internet (to update or for DNS), it may unintentionally use the 10.x.x.x address that has no route to the internet. The default route is default via 192.0.2.1 dev venet0, so I'm not quite sure how it gets to there, but it appears to work as intended after several reboots of both the container and the host.

UPDATE For a more rubust solution: I used bash to check the IP and figure out what subnet to add it to.

Ubuntu/Debian (/etc/network/if-up.d):

#!/bin/bash
if [ "${IF_ADDRESS:0:6}" = "xx.yy." ]; then
        echo "AlReece45: $IFACE, IP Address $IF_ADDRESS marked as internal"
        ifconfig "$IFACE" netmask 255.255.0.0 up
fi
if [ "${IF_ADDRESS:0:11}" = "xxx.yy.zzz." ]; then
        echo "AlReece45: $IFACE, IP address $IF_ADDRESS marked as external"
        ifconfig "$IFACE" netmask 255.255.255.0 up
fi
exit 0

CentOS/Redhat (/sbin/ifup-local):

#!/bin/bash
IFACE="$1"
IF_ADDRESS=$(ifconfig $IFACE | grep "inet addr" | awk '{print $2}' | cut -d':' -f2);
if [ "${IF_ADDRESS:0:6}" = "xx.yy." ]; then
        echo "AlReece45: $1, IP Address $IF_ADDRESS marked as internal"
        ifconfig "$1" netmask 255.255.0.0 up
fi
if [ "${IF_ADDRESS:0:11}" = "xxx.yy.zzz." ]; then
        echo "AlReece45: $1, IP address $IF_ADDRESS marked as external"
        ifconfig "$1" netmask 255.255.255.0 up
fi
exit 0
share|improve this answer
    
Did you add the routes on the HN or on the CT ? –  Emmanuel Bourg Jun 26 '12 at 9:05
    
@EmmanuelBourg The routes (and the subsequent scripts) are executed in the CT –  Reece45 Jul 9 '12 at 19:43

The problem was in between chair and keyboard. I did not set masquerading on the other device. So for everyone having the same issue: Try to set masquerade on every interface on HN.

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE # I forgot this line

I figured this out thanks to: OpenVZ wiki

share|improve this answer
    
This fixed my issue. I have a lan for nfs trafic only that needed to be routed from the ct to the outside and was not getting past the host. masquerading for that specific ip/eth worked like a charm –  Kendrick Jan 5 '13 at 6:01

I recently setup an OpenVZ server with two Ethernet network adaptors each on their own subnet with masquerading on the HN.

Discovered the following: If a CT has two IPs on different subnets, the first IP to be set in the vzid.conf file must be the one that shares the default gateway with the HN. Switching the order of the IPs and restarting the CT fixed the routing issue for me.

share|improve this answer

I have the same configuration, with two different subnets in two different nic on my HN.

I've been observed that each first venet0 work fine, but if the second venet0:1 could recieve, they seem to have pain to find route himself to my second subnet.

The funny thing: I could ssh to my virtual host, from the second subnet (comming form desktop 10.24.14.21 to my virtual host: 10.24.14.24), but this won't work:

 # ping ${SSH_CONNECTION%% *} 
 PING 10.24.14.21 (10.24.14.21) 56(84) bytes of data.

Ok, I do need to do something about venet0:1, somethine like that seem make the job:

 # vzctl exec 777 ip address delete 10.24.14.24/32 dev venet0 label venet0:1
 # vzctl exec 777 ip address add    10.24.14.24/24 dev venet0 label venet0:1

Well from there I wrote this litte workaround-venet-netmask.sh:

#!/bin/bash

(
    export NEWMASK=24      # 255.255.255.0
    export IFACE=venet0:1
    export IP1
    while IP1=$(
        ip address show dev ${IFACE%*} label $IFACE |
            sed -ne "s/^.*inet \([0-9.]\+\)\/32 .*$IFACE/\1/p"
      ); ! [ "$IP1" ] ;do
        sleep 1
      done
    ip address delete $IP1/32 dev ${IFACE%*} label $IFACE
    ip address add $IP1/$NEWMASK dev ${IFACE%*} label $IFACE
) </dev/null >/dev/null 2>&1 &

Than i've made a symlink to this script, to each VE.start needed (for now, this script is located in /etc/vz/conf/workaround-venet-netmask.sh.):

# ln -s workaround-venet-netmask.sh 777.start
# ln -s workaround-venet-netmask.sh 10001.start
# ln -s workaround-venet-netmask.sh 10012.start

For now, this seem to work fine, for me. In the hope this could help you.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.