0

I've been doing a lot of searching around for a tool to monitor a website's "bandwidth" usage, and I'm not talking about how many MB/data is being loaded but the servers actual output speed, to clarify what I mean I'll use an' example,

My server has a Internet connection at 100/MBs, I want to be able to monitor how much of that 100/MBs is being used, and have it being logged, (As a background process).

I've tried different kind of tools such as Zabbix, Nagios and ntopng(ntop), all useful tools but none of them seem cable of logging the actual server speed / usage of the speed. Currently I still have Ntop(ng) installed but I am unable to get it to do what I want it to. (it is vital that the data is logged 24/7)

The reason for this is that due to me hosting files that people can download I am with the low internet connection sometimes hitting that cap, and I want to be able to monitor when / how offen I hit that cap.

My server is a Ubuntu 12.04 running with Apache 2.2, I don't know whether it's needed but in case: My domains are setup with Virtual hosts for each domain.

I appreciate any help and hints I can get.

2
  • Just out of curiosity: what's wrong with for example Zabbix default network graphs + triggers? – Janne Pikkarainen Nov 7 '14 at 8:14
  • What I was able to get Zabbix to do was not what I needed, it seems like it didn't log / show me the actual server(network) speed being used. It is highly possible that I haven't been able to configure it correctly. I'm not looking for a "trigger" since I don't just want to be notified when it hits the cap, but I want to be able to log the entire server's "network" speed(as explained in the question) – Epodax Nov 7 '14 at 8:18
1

It sounds like you are trying to monitor full network bandwidth by measuring web server output as noted in its logs - this would not be sufficient as that not include any TCP/IP protocol overheads or other traffic (any SSH, FTP, database, or other traffic that the server might take part in).

What you need to be watching is interface statistics. Any server monitoring tool such as Zabbix and Nagios will have a function for that, for instance with collectd you would use either the interfaces module (https://collectd.org/wiki/index.php/Plugin:Interface) to monitor all traffic on a given interface or the iptables module (https://collectd.org/wiki/index.php/Plugin:IPTables) to monitor traffic that matches certain filtering rules (so you can analyse the bandwidth used in more detail).

1
  • I'm not trying to fetch the data from the log, but I want the data I find / get to be logged, atleast that's how I understood your reply? None the less I will try out the two links you tried and come back. – Epodax Nov 10 '14 at 7:29
0

I would use a script (my favorite would be python with "re" module) to parse Apache access log

For an example:

127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326

The last element is the bytes transferred to the client. You need to keep adding this number and print or log at a regular frequency to produce the total bytes transferred to the client from the last run till that moment.

For more information on Apache access log, see here: http://httpd.apache.org/docs/2.2/logs.html

#!/bin/env python
import time, sys, re

regex = re.compile(r'\s+([0-9]+)$')


def fetch_bytes(file, bytes):
        for line in file:
                match = regex.search(line)
                if match:
                        bytes = int(match.group(1))
                        yield bytes


def main():
        Access_File = "apache_access_log"
        try:
                with open(Access_File) as file:
                        gen = None
                        while True:
                                for i in range(1001):
                                        gen = fetch_bytes(file, bytes)
                                #bytes += sum(gen)
                                #prints how much KB has been transferred in the time duration
                                print "%s\t%dKB" % (time.strftime("%H:%M:%S"), (sum(gen)/1024))
                                time.sleep(120)

        except IOError:
                print "Could not locate the apache log file"
                sys.exit(1)

if __name__ == '__main__':
        main()
1
  • That would only monitor web server input+output, not any other sort of traffic (SSH, database calls if the web server accesses a database on another host, FTP connections, and so forth). Also it won't take into account TCP/IP overheads. – David Spillett Nov 7 '14 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.