How can i monitor internet traffic in my LAN?

Question

hey dudes, i read a few related questions but i dont think they're related.

I have a centOS box that ive set up as a gateway with iptables (among many things such as proxy cache using squid). my lan consists of some 30 machines, all of which connect to the internet through the gateway's eth0 (which is the only computer connected to the dsl modem via its eth1). Facing a limited download monthly quota, i installed bandwidthd on the gateway to monitor my lan pcs' bandwidth usage and had it set to monitor eth0. all of the network traffic coming to the gateway is forwarded to port 3128 for squid to handle, but on bandwidthd, it doesnt seem to be accurate. for instance, a 3.3 MB download on one of the computers after starting bandwidthd states 4.8 M for that computer under HTTP. i understand that by monitoring eth0 im monitoring all network activity and not just internet usage (port 80 etc..) but isnt the http tab in bandwidthd relevant to packets on port 80 or internet usage? thing is i just need to know how much of the internet download quota each ip used on a daily basis. what to do :) ?

Answer 1

The problem I find with counter applications is that once you have usage counts, or graphs, the inevitable question becomes "what is that" (when pointing at a total or peak on a graph).

I find the best way to investigate these things is through netflows and nfsen. A netflow is a record of a conversation: source, destination, ports, bytecounts, time. Think a wire capture where you don't care about the actual bytes transferred, just the aggregate information. By using something like nfsen to do analysis on a netflow collection, you can see who is talking to who, and by looking at ports, can usually make good guesses as to what they were doing. And best of all, you can go back in time to look at old conversations.

Here are my notes for installing nfsen on CentOS.

Answer 2

Also an old but trusty tool would be iptraf. It comes with every distribution,small and quick to install. It's nice that it is ncursed based so you can run it from the command line quickly to get realtime info about the flow of traffic on that box.

Answer 3

You could probably set a specific iptables allow rule for port 3128, since that is all of your outbound traffic and then use "/sbin/iptables -L -v" and it shows you the bandwidth and amount of packets that have came through each rule.

Answer 4

You could use a pair of very simple iptables rules to count all IP network traffic in/out of your CentOS gateway box. If eth0 is your internet-facing network interface, the rules could be thus:

iptables -I 1 INPUT -i eth0
iptables -I 1 OUTPUT -o eth0

The first rule matches all traffic coming in; the second, all traffic going out. Neither rule takes any action, so network traffic will not be modified in any way. Also note that the -I 1 option has put the rules at the top of their respective chains, ensuring that all packets will hit them.

iptables keeps a packet- and byte-count of all packets matching all rules. You can query the rules and their counters as follows:

iptables -L -n -v

You can also view the counters and reset them to zero as follows:

iptables -L -n -v -Z

Answer 5

Trust bandwidthd. A 3.3MB file COULD conceivably use 4.8MB of bandwidth. I don't think you are accounting for overhead. Each packet, or bit of data from that file has information attached to it that helps it get to it's destination.

I don't know how to calculate the amount of overhead you should see. Hopefully someone can chime in on that and add to this answer.

Answer 6

i decided on using ntop, its rather simple to install with a very rich web interface. it will most probably do the job :)

edit: my problem's solved. under HTTP with ntop, while monitoring the eth0 to which all traffic is routed, i can see accurately the size of everything i download to every ip.