first of all, this is not an every-day routing issue. The setup is fairly complex, so let me state it before.
I got a router with, lets keep it simple, 3 interfaces. eth0, eth1, eth2. eth2 is used for pppoe. eth0 & eth1 have the clients.
Okay so far so good, all basic.. Now here comes the tricky thing: I create a bunch of macvlan-interfaces on top of eth0 and eth1, the name schema is:
g1eth0 : g1 for gate1, eth0 indicates on what physical interface its laying on
This I got for every uplink I provide, lets say 3, 1 pppoe and 2 VPNs. These are then merged into bridges named after the gate.
So far we got these interfaces:
<iface>:<description> eth0 : our 1st subnet is here eth1 : our 2nd subnet is here eth2 : our pppoe is hooked here ppp0 : our pppoe uplink tap0 : our vpn1 uplink tap1 : our vpn2 uplink g1eth0 : advertised gate over uplink1 on clients in eth0 g1eth1 : advertised gate over uplink1 on clients in eth1 g2eth0 : advertised gate over uplink2 on clients in eth0 g3eth1 : advertised gate over uplink3 on clients in eth1 gate1 : bridge containing g1eth0 and g1eth1 gate2 : bridge containing g2eth0 gate3 : bridge containing g3eth1
As I said, a bunch of interfaces... Notice that an uplink can be advertised over several physical interfaces, thats why we got the bridges.
Alright now lets take a look at the routing rules:
32763: from all fwmark 0x3 lookup 202 32764: from all fwmark 0x2 lookup 201 32765: from all fwmark 0x1 lookup 200
Okay this is not so spectacular, obviously, it only checks what FWMARK a pkg has and pushes it to the according table.
The routing tables:
default via 188.8.131.52 dev ppp0 src 184.108.40.206
default via 220.127.116.11 dev tap0 src 18.104.22.168
default via 22.214.171.124 dev tap1 src 126.96.36.199
Okay the IPs are just for to fill the gaps, you should be familiar with the syntax ;)
Right now we got the routing tables, routing rules and the interfaces - but we're missing out the pkg marking, so this is being done in iptables:
iptables -t mangle -A PREROUTING -i gate1 -s 10.0.0.0/16 -j MARK --set-xmark 0x1/0xffffffff iptables -t mangle -A PREROUTING -i gate2 -s 10.0.0.0/16 -j MARK --set-xmark 0x2/0xffffffff iptables -t mangle -A PREROUTING -i gate3 -s 10.0.0.0/16 -j MARK --set-xmark 0x3/0xffffffff
Okay for explanation, we mark all pkgs comming in our bridges with the right value for the routing rules.
Now I also had to do some tweaks in
arp_ignore so that the right MAC is being advertised for the
g*eth*-interfaces. This post is getting rather full, so I will skip describing it, both are set to
filter:FORWARD chain is empty for now, it just logs the pkgs it gets.
iptables -t nat -A POSTROUTING -s 10.0.0.0/16 -j MASQUERADE.
All default policies for iptables are
tcpdump shows that the incomming pkgs are directed to the right MAC according to the
mangle:PREROUTING counters for the rules increment as they should.
ip_forward verified to be
filter:FORWARD counters are NOT incrementing.
LOG rules in every chain, but the pkgs seem to vanish once passed the
Any ideas why?
Addition I: I placed a
TRACE rule in PREROUTING as the comment suggested me, ironically it doesn't show any of the pings my clients are running.
Addition II: After some playing around with the rules,tracing,promisc,... I noticed that I see the data getting in on
ethX but not on
gateX. Seems like the brigde-interface is just dropping it, no wonder the kernel cant get it into forward.
Why does my bridge-interface do this?
bridge name bridge id STP enabled interfaces gate1 8000.dead000200b5 no g1eth0 g1eth1