Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I need to collect certain monitoring data every minute and send it to a monitoring server. The source of that data must be parseable using standard Linux tools. For example, to monitor number of current connections every minute, I use netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n, and then I extract the necessary numbers using, again, awk.

Now I also need to monitor:

  • network traffic (data send / received per minute)
  • number of newly established connection in last minute

Please give me hints to tools which would produce grep-able and awk-able output.

Note: Measurements are done on a dedicated machine, so it doesn't matter if I get data for one interface (eth0) or for the whole host.

Note: I need just TCP connections.

share|improve this question
    
To confirm post processing of the raw data needs to also be done with grep/awk/sed type tools? Splunk comes to mine but it does not meet that critera. –  jeffatrackaid Dec 6 '11 at 20:41
    
I assume just tcp, right? –  Rilindo Dec 6 '11 at 21:49

3 Answers 3

iptables rules that are set up to ACCEPT all traffic and all SYN packets (new connections) could be used as counters

iptables -A INPUT -j ACCEPT 
iptables -A INPUT --protocol tcp --syn -j ACCEPT

then run

iptables --list -v -n -Z

and pipe through as many cut cat sort grep awk sed perl ruby and similar commands as needed to get your numbers. The -Z will atomically zero out the counters every time you run this so there is no race-condition where you lose a few packets during the counting process.

share|improve this answer

For the network-traffic: netstat -in or have a deeper look at the statistic counters that are provided by ethtool -S IFACE.

In both cases you have to build the delta between two measurements.

Newly established... seems to be related to "current connections" delta or diff to the previous value.

BTW - you mentioned a monitoring server. The monitoring server might be able to do all that stuff on its own just by looking at the SNMP-data. Active the snmpd and enjoy...

share|improve this answer
    
Well, that's the problem with "newly established connections count": you can't just diff two "current connections count". Some or even all connections might be "old", that is, not newly established. –  Ivivi Vavava Dec 7 '11 at 8:23
    
With "diff" I mean the diff command. Not just the substraction of the plain numbers. Only the "new" established connections should count (should be below the listing of the previous ones). Old, terminated connections should be missing "above" the still existing ones. –  Nils Dec 8 '11 at 21:59

With the help of others, mixed with own ideas, I now do the following to monitor current number of connections and traffic periodically. (Sadly, still not implemented number of newly established connections.)

Connections (current)

OUTPUT=`netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n`

ESTABLISHED=`echo "${OUTPUT}" | grep ESTABLISHED | awk '{print $1}'`                                                      
LISTEN=`echo "${OUTPUT}" | grep LISTEN | awk '{print $1}'`

Traffic (per minute)

OUTPUT=`(LANG='' && ifstat -i eth0 60 1 | sed '3!d')`

IN=`echo "${OUTPUT}" | awk '{print $1}'`
OUT=`echo "${OUTPUT}" | awk '{print $2}'`
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.