Tell me more ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.
up vote 1 down vote favorite

I'm going to start by admitting I don't really know much about networking beyond "Ethernet plug goes here, switch connects to other switch(es) here, switch connects to internet". Most of my experience with switches has been with their default configuration and we just leave each other alone.

In our datacenter, we have a network of servers served by a single 24 port PLANET SGSW-24040, which is apparently a 1000 Mbps Level 2 managed switch. When we bought it, it replaced an older Cisco/Linksys 10/100 switch that wasn't up to the task of switching VoIP data. Replacing the old switch solved our voice quality problems.

That was all well and good, and we didn't appear to have any further problems with it, until I found a nifty feature that lets me use Munin to measure bandwidth through SNMP. It also measures network errors, which I considered to be a good thing. After setting Munin up to collect these statistics, I started getting paged frequently about individual interface errors.

Reading up on the problem, I found that I could resolve the network errors by explicitly configuring the speed of the port that was generating the error. And that's where I started running into real problems. Whenever I rebooted a server, it wouldn't renegotiate the ethernet connection, and the server would be offline until I set its port on the switch to "autodetect".

So now I'm stuck between a rock and a hard place: I can either turn off the reporting of network errors and set all the ports on the switch to autodetect, or I can eliminate the errors in the first place at the cost of being forced to remember to reconfigure the switch anytime a server is rebooted for a kernel upgrade. Is this a problem with this particular switch? Is there some way to manually configure the ethernet on the servers (they're all Debian Linux)? Should I have to do any of this at all in the first place?

share|improve this question
What errors are you seeing exactly? I don't know munin, but it sure seems like it should be possible to be more discriminating about what errors you ignore. – Zoredache Dec 11 '12 at 19:06
Oh, diagnostic information like that would be gold wouldn't it? No, instead this switch gives me a table with a count of send errors and receive errors. Even the system log doesn't have any actual error messages. – Ernie Dec 12 '12 at 16:30
OK, I found the statistics detail somewhat illuminating. One port has seen 34K Rx drops since rebooting 7 days ago, for example. – Ernie Dec 12 '12 at 16:50

2 Answers

up vote 2 down vote

You can manually configure the speed / duplex on the server computers and the switch ports, but it's not the preferred solution. Auto-negotiation is part of the gigabit Ethernet standard and all gigabit Ethernet chipsets should support it. Assuming you're using servers that all have gigabit Ethernet interfaces it sounds like you're experiencing either hardware implementation faults or driver defects.

I'd start by doing some searches on the gigabit Ethernet chipsets in your servers and the Linux kernel versions you're running to see if there are known issues with auto-negotiation. A kernel upgrade might do the trick.

If you're still running some 10/100 devices then you may have to bite the bullet and configure the interfaces on the switch and servers to disable negotiation. Some older NICs were unable to handle auto-negotiation very well, but that hasn't been the norm for 10+ years.

share|improve this answer
We have 3 APC Masterswitches that run at 10Mbps half duplex, but they're not the ones generating errors. Most (7 out of 15 devices) are Gigabit ethernet adaptors on Dell Poweredge servers, and the rest (5) are 100Mbps full duplex. – Ernie Dec 11 '12 at 22:23
up vote 1 down vote

It sounds like you have one side of the link set to Autonegotiate and the other side hard coded for speed and duplex. you need both sides configured the same, either to Autonegotiate or to the same speed and duplex. I've linked to this article before, have a look at figure 12 which illustrates the results of various speed and duplex settings between a switch and a host NIC:

http://www.ethermanage.com/ethernet/pdf/dell-auto-neg.pdf

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.