Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Let me preface this with the fact that I'm just a web developer at my company with little networking knowledge.

Earlier today there was a department that lost all of their network connections so I popped open Wireshark and observed the influx of packets to my machine.

There was normal traffic (ARP requests, etc.) coming in at ~50 packets every second. Then all of a sudden the log was flooded with packets arriving ~5000 a second. It looks as though they all contain the same data, just a looped sequence.

We have someone here looking at it but I thought I would ask if anyone had seen anything like this before.

Here's a selection from one of the captures in Wireshark (click on the image to open the Cloudshark capture): capture hosted at Cloudshark

share|improve this question
6  
Looks like someone connected a switch to another switch in the same broadcast domain. –  David Schwartz Nov 9 '12 at 19:37
3  
Those aren't packets, they're Ethernet frames. Specifically they're LLC frames. What they are exactly I don't know but they're coming from a Xerox MAC address so I would look at whatever Xerox devices you have. –  joeqwerty Nov 9 '12 at 20:21
    
@DavidSchwartz no way - the destination address of the frames is not a broadcast address (and there surely would be a broadcast storm in case of a loop). And the frames are not identical - they differ slightly. –  the-wabbit Nov 9 '12 at 21:00
    
Adam, did you mangle the MAC addresses within your capture or are those truly the ones from the wire? –  the-wabbit Nov 9 '12 at 21:03
6  
I have never heard of CloudShark before - thank you. –  the-wabbit Nov 9 '12 at 21:36

2 Answers 2

First, neither the number of frames nor the amount of data should as such significantly impact network connections even with just Fast Ethernet in place - 5,000 frames of 500 bytes amount to a bit less than 2,5 MB / second of data. They might trigger broadcast storm detection mechanisms on your swithes though, leading to broadcast frame drops of legitimate traffic - especially ARP requests - which could adversely impact IP connectivity (although typically not interrupt it completely, you are likely to see packet losses due to untimely ARP resolution).

The LLC frames in your submitted capture look weird. Neither the source address nor the multicast destination address look like valid, real-world addresses. Also, the LLC frame format violates the standard - NULL addresses are used in conjunction with a UI frame type - which should never happen:

The null address is only valid for use in the address fields of XID and TEST PDUs. The use of the null address (DSAP and SSAP) is specified in ISO/IEC 8802-2.

(Source: IEEE LLC tutorial)

I would suspect some device (presumably not a Xerox albeit the source address resolves to Xerox's MAC address space - I would expect them to know about and abide by basic rules) is violating the protocol. Try hunting it down by looking into your managed switches' FDB / address tables: start at an arbitrary managed switch, find the 00:00:03:20:00:00 address in the table which presumably would be associated with an upstream port to another switch, follow to the next switch repeating the procedure unless you find the address associated with an edge port (i.e. a port with a single connected host).

share|improve this answer
    
All this assumes the frames aren't just junk transmitted by broken hardware or buggy software; see the other answer to this question, which notes that the frame appears to consist of 0x03 0x20 0x00 0x00 repeated over and over again. –  Guy Harris May 18 at 15:14
    
@GuyHarris it does not matter much whether the device transmitting the frames is broken. My answer is primarily meant to state that a) the traffic is unlikely the reason for the network failure and b) there is a way to systematically identify the component if your switching infrastructure consists of managed devices. –  the-wabbit May 20 at 13:05

When I look at the hexdump I notice that it is just the same sequence of four bytes repeating over and over. That sequence of four bytes is not valid data, and trying to decode it will not yield any useful information.

That repeating sequence covers both source and destination MAC address as well as the Ether type. This means neither source nor destination MAC address means anything, they have been corrupted. This is also not an LLC frame, it just happens that this sequence of bytes got decoded as LLC.

If you see something like that again, I'd first try to capture the traffic using more than one brand of Ethernet adapter to be sure those corrupt frames are truly being sent on the wire, and is not just an artifact introduced on the machine doing the capture.

If you have established this traffic is really being sent on the wire, then you'll have to start looking for what device is producing it. This looks like a low level problem at the physical layer beneath the MAC layer. You won't have MAC addresses to go by, so the only option for finding the source may be to look at flashing link status, and unplug wires until you find the one producing those packets.

It is not unlikely that the root cause was defective hardware.

share|improve this answer
    
Or defective firmware or software running on that hardware. This answer is probably correct. –  Guy Harris May 18 at 15:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.