1

I need to collect certain monitoring data every minute and send it to a monitoring server. The source of that data must be parseable using standard Linux tools. For example, to monitor number of current connections every minute, I use netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n, and then I extract the necessary numbers using, again, awk.

Now I also need to monitor:

  • network traffic (data send / received per minute)
  • number of newly established connection in last minute

Please give me hints to tools which would produce grep-able and awk-able output.

Note: Measurements are done on a dedicated machine, so it doesn't matter if I get data for one interface (eth0) or for the whole host.

Note: I need just TCP connections.

2
  • To confirm post processing of the raw data needs to also be done with grep/awk/sed type tools? Splunk comes to mine but it does not meet that critera. Dec 6, 2011 at 20:41
  • I assume just tcp, right?
    – Rilindo
    Dec 6, 2011 at 21:49

3 Answers 3

2

iptables rules that are set up to ACCEPT all traffic and all SYN packets (new connections) could be used as counters

iptables -A INPUT -j ACCEPT 
iptables -A INPUT --protocol tcp --syn -j ACCEPT

then run

iptables --list -v -n -Z

and pipe through as many cut cat sort grep awk sed perl ruby and similar commands as needed to get your numbers. The -Z will atomically zero out the counters every time you run this so there is no race-condition where you lose a few packets during the counting process.

0

For the network-traffic: netstat -in or have a deeper look at the statistic counters that are provided by ethtool -S IFACE.

In both cases you have to build the delta between two measurements.

Newly established... seems to be related to "current connections" delta or diff to the previous value.

BTW - you mentioned a monitoring server. The monitoring server might be able to do all that stuff on its own just by looking at the SNMP-data. Active the snmpd and enjoy...

2
  • Well, that's the problem with "newly established connections count": you can't just diff two "current connections count". Some or even all connections might be "old", that is, not newly established. Dec 7, 2011 at 8:23
  • With "diff" I mean the diff command. Not just the substraction of the plain numbers. Only the "new" established connections should count (should be below the listing of the previous ones). Old, terminated connections should be missing "above" the still existing ones.
    – Nils
    Dec 8, 2011 at 21:59
0

With the help of others, mixed with own ideas, I now do the following to monitor current number of connections and traffic periodically. (Sadly, still not implemented number of newly established connections.)

Connections (current)

OUTPUT=`netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n`

ESTABLISHED=`echo "${OUTPUT}" | grep ESTABLISHED | awk '{print $1}'`                                                      
LISTEN=`echo "${OUTPUT}" | grep LISTEN | awk '{print $1}'`

Traffic (per minute)

OUTPUT=`(LANG='' && ifstat -i eth0 60 1 | sed '3!d')`

IN=`echo "${OUTPUT}" | awk '{print $1}'`
OUT=`echo "${OUTPUT}" | awk '{print $2}'`

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .