Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Note: An update as been added below

I'm having a very strange issue with a Linux server (Debian 6.0.6) using 802.3ad NIC bonding. Occasionally all of the NICs in the bond get marked as down and the events process runs away with 100% CPU utilization on a single thread. Nothing can be done to bring the bond back up other than restarting the server; then everything goes back to normal.

The NIC in use is an Intel VT Quad Port Server Adapter and all four ports are bonded. 802.3ad has been correctly enabled on the switch (a Netgear Smart Switch) and functions properly.

Nothing is appearing in the logs after a reboot, although when examining the terminal after the crash/failure has occurred, error messages about not being able to reset the NIC have been printed. I cannot remember the exact message; I expected them to be in one of the log files so I just rebooted to bring the server back up.

Here is how the bond is configured:

############################
# modprobe.d configuration #
############################
alias bond0 bonding
options bond0 mode=4 miimon=100 arp_interval=100 arp_ip_target=10.1.1.1 max_bonds=2
# Another bonding rule for two other NICs...

############################
# relevent lines in        #
# /etc/network/interfaces  #
############################
auto bond0
iface bond0 inet static
        address 10.1.1.100
        netmask 255.255.254.0
        broadcast 10.1.1.255
        network 1.1.0.0
        gateway 10.1.0.1
        slaves eth2 eth3 eth4 eth5
        bond-mode 802.3ad
        bond-miimon 100
        bond-downdelay 200
        bond-updelay 200
        dns-nameservers 10.1.0.1

In the above configuration, 10.1.1.1, the arp_ip_target, is the IP address of the Switch, which is always on.

Anyone seen this before or have any suggestions? It has happened about twice in the past two months; it is pretty hard to pinpoint exactly what is causing the problem, but it needs to be fixed. I've tried everything I can think of to reproduce the problem.

Update: One of the interfaces on NIC started only negotiating a 10mbit speed with the switch. I confirmed that cable wasn't the problem, nor is the switch by plugging a laptop with a 1gbit NIC directly into the interface on the server that is experiencing problems; it first negotiates a 1000gbit speed, a few seconds later dies, and re-negotiates a 100mbit speed. I also tried plugging another interface on the card in question into the same port on the switch the failing interface was plugged into and it works perfectly.

Assume for a second that only this one interface on the card is failing, not the entire card itself. Why would the entire bond fail if one interface dropped out? Is there something wrong with my configuration (above)? If not, I'm assuming that the entire card is slowly starting to fail and will need to be replaced.

Either way I plan on replacing the entire card, I just want to make sure that my configuration is correct for both future failures and educational purposes.

share|improve this question
    
have you considered the option that Netgear is unable to implement the IP stack properly and the ARP monitor you've configured is breaking because of this? Other than that, my experience with trunking on Netgear switches is limited, but the only time I tried to configure link trunking with a FS750T it was failing intermittently for no obvious reason, where I simply gave up and replaced the switch. This is also what I would advise to do in your situation. –  the-wabbit Feb 23 '13 at 13:20
    
The other bond rule (bond1) is through another Netgear switch (the same model as the one bond0 is connected to) and it has not caused any problems. At this point the switches cannot be replaced, although I will change the ARP target and not use the Netgear switch address. The problem is that this has happened so infrequently that only time will tell, I guess. I kind of doubt that that is the issue because if the ARP monitor was failing, would events really run away with 100% of a core? –  Rain Feb 23 '13 at 23:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.