Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We have a backup application that runs once every 12 hours.

Multiple servers, desktops, and laptops connect to an EC2-instance and push a backup via SFTP using their own log-in credentials.

If they need to recover a file, they can browse files via a simple SFTP-Client and restore their files. This has been running very well for the last 8 months.

I would like to know how many data is transferred on a monthly basis, per user via SSH. I don't need the logs for the past 8 months, but something that would record it starting now would be great.

Is there anything that allows me to do this?

OS: Ubuntu 10.10

share|improve this question

4 Answers 4

up vote 6 down vote accepted

It took some awk-magic, this is what my colleague and I where able to put together.

#!/bin/bash

main() {
  if [ -e $1 ] ; then
    MONTH=$(date | awk '{ print $2 }')
  elif [ $1 -ge 1 -a $1 -le 12 ] ; then
    month $1
  else
    exit 1
  fi

  echo
  echo "Usage statistics for month $MONTH"
  echo

  USERS=(`awk '/^'$MONTH'.*session opened for local user.*$/ { print $(NF-2) } ' /var/log/auth.log* | sort | uniq`)
  for i in "${USERS[@]}"
  do :
    echo "################################"
    echo "Usage for user: $i"
    READ=0
    WRITTEN=0
    #processes for this user  
    PROCS=(`awk '/^'$MONTH'.*session opened for local user '$i'.*$/ { gsub("\\[|]|sftp-server|:","", $(NF-8)); print $(NF-8) } ' /var/log/auth.log* | sort | uniq`)
    for j in "${PROCS[@]}"
    do :

      TEMP_READ=$(awk '/^'$MONTH'.*\['$j'\].*\ read\ [0-9]+\ written\ [0-9]+$/ { sum+=$(NF-2)}END{ print sum}' /var/log/auth.log*)
      READ=$(($TEMP_READ+$READ))
      TEMP_WRITTEN=$(awk '/^'$MONTH'.*\['$j'\].*\ read\ [0-9]+\ written\ [0-9]+$/ { sum+=$(NF)}END{ print sum}' /var/log/auth.log*)
      WRITTEN=$(($TEMP_WRITTEN+$WRITTEN))
    done
    echo "Read     $(($READ/(1024*1024))) MiB"
    echo "Written  $(($WRITTEN/(1024*1024))) MiB"
    echo "################################"
    echo
  done
}

month() {
case "$1" in
  1)  MONTH='Jan'
    ;;
  2)  MONTH='Feb'
    ;;
  3)  MONTH='Mar'
    ;;
  4)  MONTH='Apr'
    ;;
  5)  MONTH='May'
    ;;
  6)  MONTH='Jun'
    ;;
  7)  MONTH='Jul'
    ;;
  8)  MONTH='Aug'
    ;;
  9)  MONTH='Sep'
    ;;
  10)  MONTH='Oct'
    ;;
  11)  MONTH='Nov'
    ;;
  12)  MONTH='Dec'
    ;;
  *) echo 'Crash and Burn!'
     exit 1
   ;;
esac
}

main $1
exit 0

In sshd_config I put this:

 Subsystem sftp /usr/lib/openssh/sftp-server -l VERBOSE

Warning: This script hogs memory! If you have large logfiles, it could take up to 10 min for the script to finish (tested on EC2 Micro).

share|improve this answer
    
shame you can't receive your own bounty... –  warren Mar 28 '11 at 14:16
2  
I'm glad I got something that works :-). It even allowed me to get the statistics for the past moths. I'm a happy man ;-). –  Bart De Vos Mar 28 '11 at 15:26

You could track user logons by IP (available in the log files), and then use nearly any traffic monitoring solution (Netflow and IPtraf come to mind) to keep track of the port 22 traffic by that IP.

Unfortunately, historical usage for the previous few months is not available unless you already had something to this effect set up.

share|improve this answer

You might want to check out this post on sftp logging. I think it will get you what you want, with a bit of additional work to parse the logs.

share|improve this answer

auditd is a good utility to know what have been done to every file. It might not be what you need tough.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.