I'm troubleshooting an issue on Centos 6.4. I have a few systems setting up for greenplum. Each one has two "interconnect" interfaces connecting to an isolated subnet. It is like a cluster interconnect. So each has eth2 and eth3 both in 10.1.1.0/24 network. Each interface has its own IP address.
when I ping the two IPs of host2 from host1, the traffic always come from eth2 with the same MAC address. This can be seen from the ARP table and tcpdump. Looks like the arp table on host1 has both IP on the Mac address of eth2 on host2
I read something about ARP FLux. I tried all of the followings but nothing seem to work. Any suggestion?
echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_filter
echo 1 > /proc/sys/net/ipv4/conf/eth3/arp_filter
echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/eth3/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/eth2/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/eth3/arp_announce