Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a main router with several other switches connected to it. The network went down and when I consoled the main router, it was sluggish. Rebooted it, but once the interfaces where the other switches were connected came up, the router went back to being sluggish. Bounced the switches one by one and everything was back to normal. Now I need to figure out which of the switches was the source of the issue, and what the issue was.

What logs should I be looking at? What might I be looking for?

share|improve this question

2 Answers 2

up vote 2 down vote accepted

Are you logging to a syslog server? If not, the logging buffers on Cisco devices don't survive a reboot by default.

If you do have the logs, look at the whole network's worth of messages around the time the problem started. Any interfaces bounce? MAC flapping? Strange errors from any devices? Do you have a monitoring platform that can show you which device spiked CPU first? Route table size? BGP prefixes received?

With the information you've provided, this is a bit like taking your car to the mechanic and saying, "it doesn't work right, and you can't open the hood. How much to fix it?". This is a really complicated topic that could have thousands of answers based on your topology, hardware, configuration, and karma.

share|improve this answer
    
I totally get that. The syslog is now actually going somewhere. I was more looking for a list of log files and what they log so I can start digging. –  MathewC May 3 '13 at 14:56

Rebooting devices to solve a problem and finding root causes for those problems after the reboot are many times exclusive.

With external syslog, netflow and SNMP polled stats (cpu, memory, and interface stats including bandwidth, errors and packets per second) you do have a chance. However often these fail to function properly in the case of a problem that brings the "down."

Another option would be to gather the output from the "show tech-support" or equivalent commands prior to the reboots.

However, without sources of information to look at and following a reboot, you are only really guessing at the cause.

I would suspect some sort of L2 loop, but proving that would require stats from the interfaces (specifically packets/second).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.