Background: I'm a sysadmin at a relatively large university (about 15,000 users in residential space/ResNet). While my group is responsible for administering ResNet space (authentication, DHCP, etc.) the actual network devices are owned and run by another department. Most dorm buildings are on their own router and, by policy, VLANs cannot cross routers.
We're experiencing some strange DHCP problems, most likely related to viruses that run DHCP servers. Our DHCP failover plan is based on our DHCP servers not being authoritative, so when a box with a DHCP server (virus) pops up on a network, everyone starts taking its offers instead of the ones from the actual DHCP box.
Troubleshooting is greatly complicated by the fact that, since we don't have any devices in the subnet or on the VLAN, our only method of identifying the infected user is to physically drive out to the site, hook up to a port, and run a traffic capture (then associate a MAC with a switch port and turn off the port).
So, my idea is to make a few traffic sniffers/probes to install on problem networks (with one interface in the affected network and one in a known good subnet, to communicate back to us).
I've got two theories on how to do this:
- Setup the device with vtund, and (as needed) tunnel all traffic from the interface in the affected network back to one of our boxes.
- Install tcpdump and SShd on the device, write a little script that phones home with the device's known good IP, and then SSH into the device and do a packet capture as needed.
Any alternate theories? Has anyone had to handle a problem like this (specifically capturing traffic on a network that you can get a route to but can't actually hop on). Unfortunately, this needs to be a zero-budget solution (though I have a few Soekris boards with 3 interfaces each lying around).