What's the best way to monitor Internet traffic for the entire office?

Question

We currently have a T3 line for about 28 people and it gets deadly slow during the day so I need something to help track down why. I'm assuming someone is downloading something that they may not be aware of.

Answer 1

I would recommend against using wireshark to monitor traffic. You'll just get too much data, but you have a hard time analyzing the data. If you need to look at/troubleshoot the interaction between a couple machines, wireshark is great. As a monitoring tool, IMHO, wireshark is not quite the tool you need.

1. Profile the network traffic. Try out some actual monitoring tools: http://sectools.org/traffic-monitors.html. You're looking for Top Type of traffic (likely HTTP, but who knows), Top Talkers (should be your servers, but who knows), and potentially Malformed Traffic (large amount of TCP retransmissions, malformed packets, high rates of very small packets. Probably won't see, but who knows)

2. At the same time, work with your management to develop a network resource usage policy. In general, business terms, what business needs does the computer network exist to meet, and what are appropriate uses of the resource. This thing is costing money, so there has to be a business justification for its very existence. Your company has policies for handling the "petty cash" drawer, and I would bet your network infrastructure costs a lot more that. The key thing to focus on is not catching people doing bad things but rather watching for potential malicious activity that is degrading network functionality (i.e., the employees' ability to get their work done). Southern Fried Security Podcast and PaulDotCom Security Weekly cover information about creating appropriate security policies.

3. @John_Rabotnik idea for a proxy server was great. Implement a proxy server for web traffic. Compared to traditional firewalls, proxy servers give you much better visibility into what is going on as well as more granular control over what traffic to allow (for example, real web sites) and what traffic to block (URLs made up of [20 random characters].com)

4. Let people know - the network is having a problem. You are monitoring the network traffic. Give them a mechanism to register network slowdowns, and capture enough meta-data about the report so that in aggregate, you might be able to analyze network performance. Communicate with your coworkers. They want you to do a good job so that they can do a good job. You are on the same team.

5. As a general rule, block everything, and then allow what should be allowed. Your monitoring from step one should let you know what needs to be allowed, as filtered through your network usage/security policy. Your policy should also include a mechanism by which a manager can request new kinds of access be granted.

In summary, step one, the traffic monitoring (Nagios seems to be a standard tool) helps you figure out, in general, what is going on to stop the immediate pain. Steps 2 - 5 help prevent the problem in the future.

Answer 2

28 people saturating a T3? Doesn't seem likely (Everyone could use streaming media all day long, and it wouldn't come close.) You might want to check for routing loops and other types of network mis-configuration. You should also check for viruses. If you've got a little botnet running on your local network, that would easily explain the traffic.

What sort of switching/firewall do you use? You may already have some capability to monitor packet traffic.

Edit: I'm also a big fan of Wireshark (though I'm old, so I still think "Ethereal" in my head). If you're going to use it, the best way is to put a machine in-line so all traffic has to pass through it. That'll allow you to run exhaustive logging without having to switch your equipment into promiscuous mode.

And if it turns out you're in need of some traffic shaping, you'll be in a good position to set up a Snort proxy...I wouldn't start out with the intention of installing one, however. I really doubt your problem is bandwidth.

Answer 3

Wireshark will create a packet capture and you can analyze the network traffic with it http://www.wireshark.org/

If you need to visualize the traffic more you can use filters to show you only specific traffic based on size, type, etc.

Answer 4

See Daisetsu's answer for a software solution.

For obvious reasons, most/some countries' laws require you to inform the employees that the traffic will be monitored, though. But I assume you already know that.

A more low-tech but less invasive technique would be to visually check the physical switches for blinking lights: when the network slows, someone is probably using a lot of bandwidth, so the LED indicator for their cable will blink furiously in comparison to everybody else's. With 28 computers weeding out the "innocent" ones shouldn't take long and the user in question can be informed that their computer is misbehaving and will be checked by you shortly.

If you don't care about your employees' privacy (they might be abusing your bandwidth wilfully after all) and they either signed an agreement or local jurisdiction allows you to, you can just ignore that step and check what they are doing without advance notice, of course. But unless you think someone might me actively harming the company (e.g. violating laws, leaking information), this might result in an awkward situation (ultra-high broadband is tempting and there are lots of things on the web you could download en masse on a daily basis, most of which you shouldn't at work but might be tempted to).

Answer 5

http://www.clearfoundation.com/ or http://sourceforge.net/apps/trac/ipcop/wiki

Clear OS specifically notes that ability on their site: http://www.clearfoundation.com/docs/howtos/bandwidth_reporting_with_ntop

Answer 6

If you have a spare machine you could set it up to be an internet proxy server. Instead of the machines accessing the internet via the router, they access it via the proxy server (which accesses the internet using the router for them). This will log all internet traffic and which machine it came from. You can even block certain websites or filetypes and lots of other cool things.

The proxy server will also cache frequently used webpages so the users visit the same websites, the images, downloads, etc will already be on the proxy server so they won't need to be re-downloaded again. This might save you some bandwidth too.

This could take some setting up but if you have the time and patience then it's definitely worth going for. Setting up the proxy server is probably beyond the scope of this question, but here's a few pointers to get you started:

1. Install the Ubuntu operating system on a spare machine (get the server version if you're comfortable with Linux):

   http://www.ubuntu.com/desktop/get-ubuntu/download

2. Install the squid proxy server on the machine by opening a terminal/console window and typing the following command:

   sudo apt-get install squid

3. Configure squid the way you like, here's a guide for setting it up on Ubuntu. You can also check the squid website for more documentation and setup help.:

   https://help.ubuntu.com/9.04/serverguide/C/squid.html

4. Configure your client machines to use the Ubuntu server as their proxy server to access the internet:

   http://support.microsoft.com/kb/135982

5. You might want to block internet access on the router to all machines except the proxy server, to stop cunning users from accessing the internet from the router and bypassing the proxy server.

There is plenty of help out there on setting the Squid proxy server up on Ubuntu.

All the best, I hope you get to the bottom of it.

Answer 7

Tell us some more about the type of traffic you would normally expect over the circuit. Are you filesharing across it? Accessing mailboxes across it? Accessing PST files across it? Any Access databases? Local servers or remote servers for the users? Anything else we need to know?

Answer 8

I can't believe no one mentioned iptraf.