Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We currently have a T3 line for about 28 people and it gets deadly slow during the day so I need something to help track down why. I'm assuming someone is downloading something that they may not be aware of.

share|improve this question

migrated from superuser.com Jun 2 '10 at 18:45

This question came from our site for computer enthusiasts and power users.

2  
Voting to move this question to Server Fault, where it's a duplicate of Network Traffic Monitoring and How Can I monitor internet usage within my lan. –  Arjan Jun 2 '10 at 18:27
1  
Knowing it was a duplicate would it not have made more sense to simply close it as off topic? –  John Gardeniers Jun 2 '10 at 19:51
    
@John - always good to add a different title and description to be found by search engines, I think. –  Gnoupi Jun 2 '10 at 21:13
    
Announce that you'll be taking away internet access for P2P app users. –  duffbeer703 Jun 3 '10 at 0:51
add comment

8 Answers 8

up vote 6 down vote accepted

I would recommend against using wireshark to monitor traffic. You'll just get too much data, but you have a hard time analyzing the data. If you need to look at/troubleshoot the interaction between a couple machines, wireshark is great. As a monitoring tool, IMHO, wireshark is not quite the tool you need.

  1. Profile the network traffic. Try out some actual monitoring tools: http://sectools.org/traffic-monitors.html. You're looking for Top Type of traffic (likely HTTP, but who knows), Top Talkers (should be your servers, but who knows), and potentially Malformed Traffic (large amount of TCP retransmissions, malformed packets, high rates of very small packets. Probably won't see, but who knows)

  2. At the same time, work with your management to develop a network resource usage policy. In general, business terms, what business needs does the computer network exist to meet, and what are appropriate uses of the resource. This thing is costing money, so there has to be a business justification for its very existence. Your company has policies for handling the "petty cash" drawer, and I would bet your network infrastructure costs a lot more that. The key thing to focus on is not catching people doing bad things but rather watching for potential malicious activity that is degrading network functionality (i.e., the employees' ability to get their work done). Southern Fried Security Podcast and PaulDotCom Security Weekly cover information about creating appropriate security policies.

  3. @John_Rabotnik idea for a proxy server was great. Implement a proxy server for web traffic. Compared to traditional firewalls, proxy servers give you much better visibility into what is going on as well as more granular control over what traffic to allow (for example, real web sites) and what traffic to block (URLs made up of [20 random characters].com)

  4. Let people know - the network is having a problem. You are monitoring the network traffic. Give them a mechanism to register network slowdowns, and capture enough meta-data about the report so that in aggregate, you might be able to analyze network performance. Communicate with your coworkers. They want you to do a good job so that they can do a good job. You are on the same team.

  5. As a general rule, block everything, and then allow what should be allowed. Your monitoring from step one should let you know what needs to be allowed, as filtered through your network usage/security policy. Your policy should also include a mechanism by which a manager can request new kinds of access be granted.

In summary, step one, the traffic monitoring (Nagios seems to be a standard tool) helps you figure out, in general, what is going on to stop the immediate pain. Steps 2 - 5 help prevent the problem in the future.

share|improve this answer
    
+1; all great tips. Proper monitoring is a must. –  Jimmy Shelter Jun 2 '10 at 20:57
add comment

I can't believe no one mentioned iptraf.

share|improve this answer
add comment

Tell us some more about the type of traffic you would normally expect over the circuit. Are you filesharing across it? Accessing mailboxes across it? Accessing PST files across it? Any Access databases? Local servers or remote servers for the users? Anything else we need to know?

share|improve this answer
add comment

If you have a spare machine you could set it up to be an internet proxy server. Instead of the machines accessing the internet via the router, they access it via the proxy server (which accesses the internet using the router for them). This will log all internet traffic and which machine it came from. You can even block certain websites or filetypes and lots of other cool things.

The proxy server will also cache frequently used webpages so the users visit the same websites, the images, downloads, etc will already be on the proxy server so they won't need to be re-downloaded again. This might save you some bandwidth too.

This could take some setting up but if you have the time and patience then it's definitely worth going for. Setting up the proxy server is probably beyond the scope of this question, but here's a few pointers to get you started:

  1. Install the Ubuntu operating system on a spare machine (get the server version if you're comfortable with Linux):

    http://www.ubuntu.com/desktop/get-ubuntu/download

  2. Install the squid proxy server on the machine by opening a terminal/console window and typing the following command:

    sudo apt-get install squid

  3. Configure squid the way you like, here's a guide for setting it up on Ubuntu. You can also check the squid website for more documentation and setup help.:

    https://help.ubuntu.com/9.04/serverguide/C/squid.html

  4. Configure your client machines to use the Ubuntu server as their proxy server to access the internet:

    http://support.microsoft.com/kb/135982

  5. You might want to block internet access on the router to all machines except the proxy server, to stop cunning users from accessing the internet from the router and bypassing the proxy server.

There is plenty of help out there on setting the Squid proxy server up on Ubuntu.

All the best, I hope you get to the bottom of it.

share|improve this answer
    
Is that only for the "web" data? Are you serious that you want to set up a proxy for all kinds of protocols and data? –  d-_-b Jun 3 '10 at 0:46
add comment

28 people saturating a T3? Doesn't seem likely (Everyone could use streaming media all day long, and it wouldn't come close.) You might want to check for routing loops and other types of network mis-configuration. You should also check for viruses. If you've got a little botnet running on your local network, that would easily explain the traffic.

What sort of switching/firewall do you use? You may already have some capability to monitor packet traffic.

Edit: I'm also a big fan of Wireshark (though I'm old, so I still think "Ethereal" in my head). If you're going to use it, the best way is to put a machine in-line so all traffic has to pass through it. That'll allow you to run exhaustive logging without having to switch your equipment into promiscuous mode.

And if it turns out you're in need of some traffic shaping, you'll be in a good position to set up a Snort proxy...I wouldn't start out with the intention of installing one, however. I really doubt your problem is bandwidth.

share|improve this answer
    
Have to agree with this. We can talk about using various software monitoring tools all day but 28 users killing a T3 through any type of "normal, but excessive" use sounds just plain wrong to me... This is more than a few heavy users and "someone is downloading something they may not be aware of". –  RobM Jun 2 '10 at 18:48
add comment

Wireshark will create a packet capture and you can analyze the network traffic with it http://www.wireshark.org/

If you need to visualize the traffic more you can use filters to show you only specific traffic based on size, type, etc.

share|improve this answer
add comment

See Daisetsu's answer for a software solution.

For obvious reasons, most/some countries' laws require you to inform the employees that the traffic will be monitored, though. But I assume you already know that.

A more low-tech but less invasive technique would be to visually check the physical switches for blinking lights: when the network slows, someone is probably using a lot of bandwidth, so the LED indicator for their cable will blink furiously in comparison to everybody else's. With 28 computers weeding out the "innocent" ones shouldn't take long and the user in question can be informed that their computer is misbehaving and will be checked by you shortly.

If you don't care about your employees' privacy (they might be abusing your bandwidth wilfully after all) and they either signed an agreement or local jurisdiction allows you to, you can just ignore that step and check what they are doing without advance notice, of course. But unless you think someone might me actively harming the company (e.g. violating laws, leaking information), this might result in an awkward situation (ultra-high broadband is tempting and there are lots of things on the web you could download en masse on a daily basis, most of which you shouldn't at work but might be tempted to).

share|improve this answer
    
Not for network analysis, if it is for maintaining the network then they don't need to inform employees. –  Daisetsu Jun 2 '10 at 18:35
    
They probably don't need to inform them if they're just checking the surface, but if they're actually looking at the exact nature of the data (i.e. packet sniffing), which could contain passwords or personal data (however inappropriate that kind of use of the company network may be), it'd be at least good karma (if not a legal requirement) to tell them about it in advance. –  Alan Jun 2 '10 at 18:40
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.