0

I have a web application which will be powering a wide range of websites. I want to implement some form of bandwidth monitoring on a per domain basis for billing and excessive use purposes. I am using Apache as the web server and am just using the default vhost for all sites (with the site level routing and control handled in the application instead of the server). Is there a way to, and if so how do I, extract bandwidth usage on a per domain name basis in Apache instead of a per vhost basis.

Just to clarify by domain based monitoring I am talking about the HOST request header field.

As an example imagine I have 4 sites names www.example.com, www.example.org, www.site1.com, and site2.example.com. Now all four of these sites will be handled by the exact same PHP code and same vHost file (the default catch all file in Apache), will run under the same server user, etc. The sites however are separate sites and are differentiated on the server by getting the host request header in the application. I need to monitor the bandwidth used by each of those 4 sites separately as each one will have a different amount of bandwidth they are allowed.

I am not concerned about if this is something that can be changed by users on the server as the users will not have access to the servers at all and will manage their sites entirely through a web panel (think something akin to Weebly where a single application powers all sites load balanced across multiple servers). As such the method would be used across multiple servers and fed into a single database.

I envision something that either...

  • At scheduled intervals (eg: every hour, every 6 hours, etc) it will create a bandwidth usage file recording the site name, and bandwidth used in that period which I can then pull from each server regularly and process on another system myself.
  • At scheduled intervals it will take the bandwidth used and push the data to a remote file system or database server.
  • At scheduled intervals I will have a bespoke internal application poll the server for the bandwidth used per site and drop it into a database myself.

Any method that achieves the result of getting bandwidth usage accurate enough for billing purposes would be good. I would prefer some method that works with a daemon system on the server itself that my code can interact with or that produces a parsable file that I can work with rather than needing to code a system to perform terminal commands and process the results.

As a sample I am wanting to achieve something similar to the way Wix works out the bandwidth used by various sites hosted on their system as it is effectively an application service not a shared hosting service.

1 Answer 1

1

A fairly common approach is to generate usage statistics from the access logs.

Create a CustomLog in a LogFormat that includes the domain name from host header.

Then depending on which web analytics package you select either use the recorded Host to split the log file on separate ones for each domain and generate usage statistics from those, or maybe the software already has support to segregate multiple domains.

5
  • And the log files would show bandwidth and not just the page request details? Apr 13, 2017 at 10:37
  • Requests are logged with their size, that is not quite the bandwidth but often close enough
    – HBruijn
    Apr 13, 2017 at 10:42
  • Would using one of these options impart a substantial load on the server, or would it be thrown out by the use of a dynamic language like PHP. I took a look at AWStat and it seems, unless I am misunderstanding, that it calculates the bandwidth used based on file size rather than the actual html payload that is sent to the browser. Apr 13, 2017 at 14:26
  • I would set your hosts up as separate name based hosts. They can all share the same document root, but segregating them into separate hosts allows you more flexibility - customer provided SSL certs, custom error pages, different webmaster contact emails in server signature (if turned on), etc and of course this would allow separate log files.
    – ivanivan
    Apr 13, 2017 at 14:54
  • The log file stores the number of bytes sent to the client for the request. So, it counts all the bytes for dynamically generated requests, error messages, redirections etc. And yes, you should definitely use separate virtual hosts. Apr 13, 2017 at 16:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .