Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

On my firewall I've three interfaces :

eth0 : public IP (46.X.X.X.)
eth0:0 public IP (46.X.X.Y.)
eth1 : public IP (88.X.X.X.)
eth2 : private LAN (172.X.X.X)

I've setup a basic NAT which works great until I turn on the eth1 interface, I basically loose the connectivity. When I turn off the interface (ifconfig eth1 down) then the NAT re-work. I've added some policy routing via iproute, which makes my three public IP's available.

I don't understand why turning on eth1 on makes the LAN unavailable.

PS : weirder ; when I turn on eth1 BUT remove the NAT, then the firewall is accessible by using the public IPS. So to me it's exclusively a NAT issue, since without the NAT the network works while with the NAT without the second public interface, the NAT does work.

Regards

EDIT : I've been able to make it work by using iproute2 rules. That was definitely a routing issue. Here is what I did :

ip rule add prio 50 table main
ip rule add prio 201 from ip1/netmask table 201
ip rule add prio 202 from ip2/netmask table 202

ip route add default via gateway1 dev interface1 src ip1 proto static table 201
ip route append prohibit default table 201 metric 1 proto static

ip route add default via gateway2 dev interface2 src ip2 proto static table 202
ip route append prohibit default table 202 metric 1 proto static

# mutipath
ip rule add prio 221 table 221

ip route add default table 221 proto static \
            nexthop via gateway1 dev interface1 weight 2\
            nexthop via gateway2 dev interface2 weight 3
share|improve this question
    
Can you show the output of iptables-save and netstat -rn? –  bahamat Jul 28 '11 at 16:11
    
Hi bahamat, here is the netstat -rn ouput pastie.org/2288568 see below for the extra infos and the whole network config. Thanks :) –  Razique Jul 29 '11 at 7:55

3 Answers 3

As others have said, there is an issue with traffic going out eth0 and MASQUERADING. Could you try adding a rule with

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

when you're in the it-doesn't-work scenario, and we can see if that fixes things?

Edit 1 (after many comments):

Ah, OK, that changes everything. I didn't expect to see the client on eth1 (I assumed it would be on eth2), and secondly, the IP address you're trying to ping from the client belongs to the firewall itself. I've just tested that on a similar (but working) setup, and you shouldn't see that ICMP traffic on the eth0 interface at all, so something's screwy.

A couple of questions occur: is the subnet on eth2 really privately addressed, but using non-RFC1918 addresses, or are you really connecting two public subnets together here? Is there a custom route on ubuntu? (Could we get the output of netstat -rn on both the firewall and the client?)

I notice the hostname of the client is VALRAISO-VM4; is this a VM, and if so, is it on a different physical host from the ubuntu box?

And - and this is the kicker, I think - could you clarify what you're actually trying to achieve here? I understand this ping is just a prelude to see if the general architecture is working, but what do you actually want to achieve with this setup?

Edit 2: thanks for that. Now I'm even more confused, because the client's default route is 91.194.96.1, which isn't any of the interfaces on your firewall. So I can only assume that it has some other route to the outside world, and the PINGs we saw on your external interface are exactly that: traffic coming in from the outside world, having made it out via another route. That would certainly explain why they were not being NATted by the firewall.

You say that "the client doesn't use eth1 as main interface, but eth0" which I completely accept, but I don't believe I've ever said otherwise. My comment below about packets going out eth0 badged with eth1's address referred to firewall interfaces, not client interfaces (and was in any case wrong). If the client is multi-homed then your earlier pastie showing its ifconfig output is wrong.

The VM issue substantially confuses things, too, as it means there's another silent layer altering traffic between the hosts we see and the network hardware. I'm really sorry, but this is rapidly turning into a very slow method of me mapping out your entire network topology, and I don't have either the time or the inclination to do this unpaid.

share|improve this answer
    
Hi MadHatter, this masquerading rule doesn't do anything at all... –  Razique Aug 1 '11 at 21:49
    
How do you know? Could you try a tcpdump -n -n -i eth0, send some traffic out that interface from an interior box, and see what you see? That could shed much useful light on whether it's a NAT failure, an internal routing failure, or an external routing failure - or something else altogether. –  MadHatter Aug 2 '11 at 7:31
    
I'm still unable to reach the .84 ip. Here is the output : pastie.org/2310953 the server is unable to send a "reply" packet. When I remove the nat, the server send "reply" packets ; when I turn down the second iface, the server doesn't send "reply" packets, but the NAT works (the server behind the NAT sends the "reply" packets), thanks :) –  Razique Aug 2 '11 at 21:10
    
What I can say with certainty is that you didn't have my NAT rule in place when you ran that, because you printed out your NAT rules and it wasn't there. I am certain there is a rational explanation for what you're seeing, but it's hard to determine what's going on because of the redacted information that you post (eg, as others have noted, the IP address on eth1 of system ubuntu doesn't seem to match the broadcast address - but we can't say for sure). Before I try anything else, are you prepared to (a) stop redacting ip addresses, and (b) actually try what you're asked to try? –  MadHatter Aug 3 '11 at 8:47
    
Hi MadHatter, you right, and I'm sorry, it's not helping at all. Please find here a- the ping from a server b- the tcpdump trace : pastie.org/2315959 I've also tried directly the public IP's ,which should not change the behaviour (eg 46.227.16.81 instead of 172.16.40.81) –  Razique Aug 3 '11 at 19:41

I'd suspect some problem with either routing or NAT definition. Possible problem areas:

  1. NAT definition rules;
  2. routing rules (if some packets leave via eth0, and some via eth1 things may get strange);

Show us the rules (both iptables and iproute).

Edit: You have 2 problems:

  1. You MASQUERADE only packets going out of eth1. Your default route specifies both eth0 and eth1. This means, that outgoing packets can get out of any of the interfaces. If they go out of eth1, NAT works, if they go out of eth0, it doesn't.

  2. You have 2 default routes with NATting. This creates a problem of source IP address for outgoing packets -- they may end up having any of the 2 addresses, depending on the route they go. Therefore, for the destination host, they appear as connection from 2 different sources. The easiest thing would be to specify SNAT with just one address. This will cause all outgoing packets to have a well-defined source IP address, and come back just via 1 interface. The down side is, if you loose that interface, you'll loose outside connectivity as well.

share|improve this answer
    
Hi Pawel, here are all the network-related settings : pastie.org/2288555 BTW : I've tried to ping google using the -I INTERFACE and the three public interfaces are able to reach the external network. –  Razique Jul 29 '11 at 7:53
    
@Kronick I think you may have bad definition of eth1: you have IP 88.X.X.X/24 but broadcast address and gateway in 91.X.X.X, which is outside 88.X.X.X network. I'd also try replacing MASQUERADE with SNAT. Second thing -- after eth1 is up you have 2 default routes. I don't see anything that would prevent the client's session to send alternate packets via alternate interfaces (eth0 & eth1), resulting in different source addresses at destination. –  Paweł Brodacki Jul 30 '11 at 5:48
    
HI Pawel,I've removed the MASQ and replaced it by a SNAT, but it didn't work at all either :/ Thanks –  Razique Aug 1 '11 at 21:55

The route for eth0 has the highest priority. Your iptables rules only MASQ for eth1. Therefore, when your traffic is coming from the internal network it's using eth0's default gateway, and not using the NAT.

You have a few options.

  1. Adjust your route priority to make eth1 preferred
  2. Add an iptables rule to MASQ for eth0
  3. Add an ip rule for traffic from source 172.16.40.0/24 to use table T2.
share|improve this answer
    
Hi bahamat, I've tried to add an explicit rule : $ ip rule add 172.16.40.0/24 table T2 and the iptables rule which MASQ eth0, but both commands don't work, I'm still unable to ping eth0:0's ip –  Razique Aug 1 '11 at 21:53
    
@kronick Can you show me the exact output? –  bahamat Aug 1 '11 at 22:51
    
Hi bahamat, here is the output of the ping :pastie.org/2310953 thanks :) –  Razique Aug 2 '11 at 21:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.