Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I am running two Ubuntu 12.04 instances, both of which cannot access each other.

Machine 1:

eth0 - 10.0.0.20
eth1 - 10.0.0.30

netstat -nr:

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG        0 0          0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 eth1
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 eth0

Machine 2:

eth0 - 10.0.0.100
eth1 - 10.0.0.40

netstat -nr:

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG        0 0          0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0 eth1

Both machines have access to each other only through internet, but not through VPC (LAN, in other words). Disabling firewall does not help. Machine 1 can ping only 10.0.0.100. Machine 2 can ping only 10.0.0.30. I am unable to connect to any of the machines in TCP/UDP protocols through LAN. There are no ACLs and the daemons on the machines are listening on all IPs, all interfaces. What could be the problem?

Edit 1:

It seems that the problem was in routing tables. I have solved the issue on Machine 1 after deleting 1 NIC on Machine 2 and doing some random actions. This resulted in DHCP obtaining these rules:

~$ ip rule
0:      from all lookup local 
32765:  from 10.0.0.30 lookup 2 
32766:  from all lookup main 
32767:  from all lookup default 
~$ 

I cannot reproduce this on Machine 2, but I can add the rules and routing tables manually with:

ip route add default via 10.0.0.1 dev eth0
ip route add default via 10.0.0.1 dev eth1 table 2
ip rule add from 10.0.0.167 lookup 2 prio 1000

How could I avoid doing it manually?

share|improve this question
2  
can i ask: why do you have two ip's from same subnet on different interfaces? you can have multiple ip addresses assigned to the same interface. –  GioMac Sep 6 '13 at 8:22
    
As I understand, Amazon does not allow binding more than 2 external IP (Elastic IPs) to one network interface. Am I right? Or should I bind it to local network address and then it would work? But still - shouldn't it work the way I'm trying to make it work? –  Ernestas Sep 6 '13 at 11:29
    
Could you please show a traceroute from machine1 to machine2 and vice-versa? But as GioMac already said, your routing tables look messed up because you use the same subnet on multiple interfaces. Doing that may produce the side-effects you encounter. –  Marki Sep 8 '13 at 12:27
    
If you need that many IPv4 addresses, something is probably wrong with your architecture. (Or you're trying to send spam....) –  Michael Hampton Sep 8 '13 at 13:10
    
I need at least 2 IP addresses because I am running several HTTP servers. If I were sending spam, Amazon wouldn't work (they limit the amount of traffic on 25 port and they are very cautious, meaning that they check bounces and etc. from time to time). By the way - sometimes the ping as well as traffic from machine 2 to 10.0.0.30 goes through and sometimes it doesn't. The traceroutes are in case they don't: pastebin.com/CGjmSAR9 –  Ernestas Sep 9 '13 at 10:28

1 Answer 1

Try adding the routes more specifically. Routing will prefer the first matching route ordered by specificity, metric, then order of addition.

For example, on machine 1, add the following route:

ip route add 10.0.0.40/32 dev eth1 src 10.0.0.30

And on machine 2:

ip route add 10.0.0.30/32 dev eth1 src 10.0.0.40

you may want to delete the more general routes for eth1 on both machines:

ip route del 10.0.0.0/24 dev eth1

Since they are in /24s, you should try to get them in different subnets, for example, if both eth1s were in 10.0.1.0/24, you wouldn't have this problem at all.

share|improve this answer
    
Now I get this: pastebin.com/NVu36Hiq –  Ernestas Sep 9 '13 at 10:19
    
hmm, it's not particularly clear which two are your internal interface pair and which are your external; that should work. –  Andrew Domaszek Sep 10 '13 at 14:54
    
Both of them are internal and external. –  Ernestas Sep 10 '13 at 18:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.