3

first of all, this is not an every-day routing issue. The setup is fairly complex, so let me state it before.

I got a router with, lets keep it simple, 3 interfaces. eth0, eth1, eth2. eth2 is used for pppoe. eth0 & eth1 have the clients.

Okay so far so good, all basic.. Now here comes the tricky thing: I create a bunch of macvlan-interfaces on top of eth0 and eth1, the name schema is:

g1eth0 : g1 for gate1, eth0 indicates on what physical interface its laying on

This I got for every uplink I provide, lets say 3, 1 pppoe and 2 VPNs. These are then merged into bridges named after the gate.

So far we got these interfaces:

<iface>:<description>
eth0   : our 1st subnet is here
eth1   : our 2nd subnet is here
eth2   : our pppoe is hooked here
ppp0   : our pppoe uplink
tap0   : our vpn1 uplink
tap1   : our vpn2 uplink
g1eth0 : advertised gate over uplink1 on clients in eth0
g1eth1 : advertised gate over uplink1 on clients in eth1
g2eth0 : advertised gate over uplink2 on clients in eth0
g3eth1 : advertised gate over uplink3 on clients in eth1
gate1  : bridge containing g1eth0 and g1eth1
gate2  : bridge containing g2eth0
gate3  : bridge containing g3eth1

As I said, a bunch of interfaces... Notice that an uplink can be advertised over several physical interfaces, thats why we got the bridges.

Alright now lets take a look at the routing rules:

32763:  from all fwmark 0x3 lookup 202
32764:  from all fwmark 0x2 lookup 201
32765:  from all fwmark 0x1 lookup 200

Okay this is not so spectacular, obviously, it only checks what FWMARK a pkg has and pushes it to the according table.

The routing tables:

200: default via 1.2.3.4 dev ppp0 src 4.3.2.1

201: default via 5.6.7.8 dev tap0 src 8.7.6.5

202: default via 9.10.11.12 dev tap1 src 12.11.10.9

Okay the IPs are just for to fill the gaps, you should be familiar with the syntax ;)

Right now we got the routing tables, routing rules and the interfaces - but we're missing out the pkg marking, so this is being done in iptables:

iptables -t mangle -A PREROUTING -i gate1 -s 10.0.0.0/16 -j MARK --set-xmark 0x1/0xffffffff
iptables -t mangle -A PREROUTING -i gate2 -s 10.0.0.0/16 -j MARK --set-xmark 0x2/0xffffffff
iptables -t mangle -A PREROUTING -i gate3 -s 10.0.0.0/16 -j MARK --set-xmark 0x3/0xffffffff

Okay for explanation, we mark all pkgs comming in our bridges with the right value for the routing rules.

Now I also had to do some tweaks in arp_announce and arp_ignore so that the right MAC is being advertised for the g*eth*-interfaces. This post is getting rather full, so I will skip describing it, both are set to 2.

The filter:FORWARD chain is empty for now, it just logs the pkgs it gets.

Now NAT'ing: iptables -t nat -A POSTROUTING -s 10.0.0.0/16 -j MASQUERADE.

All default policies for iptables are ACCEPT.

tcpdump shows that the incomming pkgs are directed to the right MAC according to the g*eth*-interfaces.

mangle:PREROUTING counters for the rules increment as they should.

ip_forward verified to be 1.

filter:FORWARD counters are NOT incrementing.

I got LOG rules in every chain, but the pkgs seem to vanish once passed the mangle:PREROUTING.

Any ideas why?

Addition I: I placed a TRACE rule in PREROUTING as the comment suggested me, ironically it doesn't show any of the pings my clients are running.

Addition II: After some playing around with the rules,tracing,promisc,... I noticed that I see the data getting in on ethX but not on gateX. Seems like the brigde-interface is just dropping it, no wonder the kernel cant get it into forward.

Why does my bridge-interface do this?

bridge name     bridge id               STP enabled     interfaces
gate1           8000.dead000200b5       no              g1eth0
                                                        g1eth1
4
  • # iptables -t raw -A OUTPUT -j TRACE # iptables -t raw -A PREROUTING -j TRACE will be useful for you (Logs from packet travel through iptables will be shown in /var/log/messages Apr 19, 2013 at 8:32
  • 1
    pastebin.com/ZgA5rtvJ This is odd, it doesnt show the ICMP dst as 8.8.8.8 - tcpdump shows: 10:39:26.395713 20:cf:30:22:c2:3a > de:ad:01:02:00:e0, ethertype IPv4 (0x0800), length 74: 10.1.10.50 > 8.8.8.8: ICMP echo request, id 1, seq 27370, length 40
    – f0o
    Apr 19, 2013 at 8:39
  • 3
    I'm moderately confused by your description, but it looks like the traffic you are concerned about is passing through the linux layer-2 ethernet bridges only, not being routed at layer-3 by the linux kernel. If I am correct then iptables won't see it go through its FORWARDING table, because it's not being routed. ebtables is probably what you want to look into.
    – Dan Pritts
    May 21, 2013 at 15:24
  • Hi Dan, I looked at ebtables and it turned out nothing... I'm really confused and basically gave up until 1st of June when I can actually have physical access on the network again. Because you are right, there seems to be no Layer3 interactions with the data being received. I will post any relevant achievements to the post and hopefully find cause & resolution.
    – f0o
    May 22, 2013 at 20:27

1 Answer 1

0

It could be blocked due to reverse path filtering.

Try turning it off: http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.kernel.rpf.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.