Why are we having problems with multiple nics in office and production networks going to 100mbps mode?

Question

updates and additional info

2 environments, office and production at datacenter

Both environments have Cat-5e cables, a cable was replaced when it tested as bad with Broadcom's cable testing, and some others cables have been switch out during trial and error troubleshooting

Both environments have HP ProCurve gigabit switches

Office environment has Cisco 861w router which spec sheets say has 4 100mbps ports

Production environment has Cisco ASA 5510 router which has a mixture of gigabit and 100mbps ports. I am currently looking into what ports are in use.... Ok, the switch is reporting gigabit connection to router.

note: I do not know how a 100mbps router port affects nic modes if there is a gigabit switch between them

This has been a recurring issue for us both on our office network and production. It rarely matters since normal traffic rarely comes close to utilizing 100mbps but when doing vmware conversions it is particularly annoying for it to take 8 hours instead of maybe 2 due to this limitation. My laptop and one of our office servers are showing as gigabit on our switch and we are seeing such speeds, however another server and NAS we are testing goes down to 100mbps mode when going through our primary office gigabit switch by way of wall drops (same wall plate as laptop). When we hook up the NAS to a local "desktop" gigabit switch it switches to 1000mbps mode and tests as such speeds to the laptop. In production only half of the nics report as 1000mbps though they all should. Any insight into why this might happen or what should be taken into account when troubleshooting would be appreciated.

Answer 1

It sounds like you are using auto-negotiation because you plugged in the NAS box and the line speed was reported differently. Wikipedia actually has a very good article on auto-negotiation if you are interested in the minituae ( http://en.wikipedia.org/wiki/Autonegotiation ). In an ideal world, everyone would follow the spec exactly and all the manufacturers would play nice and it would always work correctly. That is not always the case. That would be the last-ditch, I've checked everything else kind of thing.

The reason that the other commenters are asking is that during this auto-negotiation process, the NICs will listen to the messages and chose the best possible. If the signal to noise ratio is too low, the NICs move to a lower speed until they can both agree. Long runs reduce the signal strength. Runs near ceiling lighting or any A/C can induce noise. The reason we use twisted pairs is to reduce that induced noise.

All that being said, the cheap-o cable testers do not test signal to noise ratio usually just insuring that the pin-outs are correct. If you have a noise issue, there are higher end testers. Typically, this is consultant work because the testers are expensive. Check for crimps in the line or bends greater than 90 degrees. Check to see if A/C cable are wrapped with or near the affected NICs. Check to see if the runs are near overhead lighting. Draw a map of the office and geographically figure out if the low NICs are in a certain area.

Alternatively, you can "nail" down speeds on most NICs i.e. using a manual assignment of 1G instead of a negotiated 100 though this opening a different box of worms. HTH