Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We have a Linux (debian) server that has two NICs that are connected to the same switch. (We used to have a switch dedicated for traffic between our servers but when most of our servers moved to a colo facility that switch moved as well.) These NICs have different static IP addresses but about once a month arpwatch will send out a pair of flip flop messages as one of the IPs bounces from one interface to the other and back. What could be causing this?

share|improve this question
1  
You don't say anything about the OS or the hardware. It might be helpful if you did. –  Dennis Williamson Aug 24 '09 at 21:28
    
On DHCP? I honestly can't think of a reason this would ever happen on fixed IPs. –  Ernie Aug 24 '09 at 21:36
1  
Is the flip-flop timed around a reboot, by chance? –  Bill Weiss Aug 24 '09 at 21:37
    
I've edited the question to reflect that the server is running linux and uses static IP addresses. –  David Locke Aug 24 '09 at 22:53
    
@Bill, We reboot this server about once a year, so no it's not related to a reboot. –  David Locke Aug 24 '09 at 23:02

3 Answers 3

This might be related to Linux ARP behavior with multiple NICs on the same network and has been discussed in this Server Fault question. Basically you need to set some sysctl parameters:

net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

These will cause ARP replies to only go out on the network interface that actually owns the IP address being requested in the ARP request.

As mentioned in the above-referenced question this is because by default in Linux the host owns an IP, and not a particular interface, so the replies will go out on whatever interface it chooses. This can cause problems when you have multiple interfaces on the same network and is exactly what the above sysctl parameters modify.

share|improve this answer

if it is on linux AND you are using udev (probably are, it's been "standard" for a few years now), you can force the assignment of eth0 or eth1 etc to specific MAC addresses.

e.g. i have the following in /etc/udev/rules.d/70-persistent-net.rules on an IBM x3655 fileserver at work. it has 4 NICs, three of them plugged in to 3 different VLANs, and the final one plugged into the "internal Gb backbone" (a dedicated Gigabit switch in the rack for all inter-server communications - NFS, ntp, dns, rsync, etc). Obviously, it is crucial that each interface gets the same name and the same IP address when it reboots:

# This file was automatically generated by the /lib/udev/write_net_rules
# program run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single line.

# built-in broadcom extreme NICs
## PCI device 0x14e4:0x164a (bnx2)
SUBSYSTEM=="net", ACTION=="add",DRIVERS=="?*", ATTR{address}=="00:14:5e:5a:18:ac", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add",DRIVERS=="?*", ATTR{address}=="00:14:5e:5a:18:ae", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI Intel e1000 NICs
## PCI device 0x8086:0x105e (e1000)
SUBSYSTEM=="net", DRIVERS=="?*", ATTR{address}=="00:15:17:2e:e0:c3", NAME="eth2"
SUBSYSTEM=="net", DRIVERS=="?*", ATTR{address}=="00:15:17:2e:e0:c2", NAME="eth3"
share|improve this answer
    
I looked at the equivalent file on this server and I noticed that two of the interfaces (but not the two that are flip floping) have the name "eth0". What could go wrong with that? –  David Locke Aug 24 '09 at 23:00
    
well, lots could go wrong with that. or nothing. depends. are the two "eth0" entries both for NICs that are still currently installed in the machine? or is one or both of them for old NICs that have been replaced? either way, it would be a good idea to clean up that file so that it accurately reflects current reality. –  cas Aug 24 '09 at 23:43

I don't have an answer, but maybe a hint. A while ago a colleague of mine mentioned that there had been a bug in the linux driver for a particular network card. This bug would apparently lead to the MAC(!) addresses bouncing between interfaces if the server had more than one. I never checked this, but I seem to remember that this bug was supposedly fixed with 2.6.17 or thereabouts.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.