10

I want to measure the justify stress-test result on production env.

How to measure req/sec by analyzing apache logs?

apache2.2

LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %D" combined

Can I do with %t and %D parameters?

Improve this question

5 Answers 5

Reset to default
2

In real time you could use mod_status. You could also count the lines in your access.log over a given period and work out the rate from that. Something like this 

#!/bin/bash
LOGFILE=/var/log/apache2/access.log
STATFILE=/var/tmp/apachestats
START=$(wc -l "$LOGFILE" | awk '{print $1}')
PERIOD=10
PRECISION=2
sleep "$PERIOD"
while true
do
    HITSPERSECOND=0
    HITS=$(wc -l "$LOGFILE" | awk '{print $1}')
    NEWHITS=$(( HITS - START ))
    if [[ "$NEWHITS" > 0 ]]
    then
        START=$HITS
        HITSPERSECOND=$(echo -e "scale=$PRECISION\n$NEWHITS / $PERIOD" | bc -l )
    fi
    echo "$(date) rate was $HITSPERSECOND" >>"$STATFILE"
    sleep "$PERIOD"
done
Improve this answer
25

This great article helped me a lot...

http://www.inmotionhosting.com/support/website/server-usage/view-level-of-traffic-with-apache-access-log

I had created a set of prepered commands that I am using to analyze apache log:

request per hour
cat access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

request per hour by date
grep "23/Jan" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

request per hour by IP
grep "XX.XX.XX.XX" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

requests per minute:
cat access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

requests per minute for date:
grep "02/Nov/2017" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

requests per minute for url:
grep "[url]" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

per IP per minute
grep "XX.XX.XX.XX" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

hopes it will help anyone who's looking for it...

Improve this answer
4
  • My system required quoting the bracket delimiters to make these work ( '[' and ']' ) but works fine -- example: (request per hour) cat access.log | cut -d'[' -f2 | cut -d']' -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c
    – Jeff Clayton
    Sep 18, 2018 at 14:35
  • @JeffClayton in my case i just had to add a space between the "d" and the "[": cut -d [ -f2 | cut -d] -f1
    – Daviz
    Mar 5, 2021 at 10:44
  • @Daviz thank you, you never know what slight differences occur between systems. I am certain someone else will find your adjustment handy as well.
    – Jeff Clayton
    Mar 5, 2021 at 14:35
  • Minor nit pick... This is "requests grouped by clock hour" and similar, not exactly "requests per hour". Requests/minute won't show 100 when 50 come in ten seconds before the minute change and 50 come in ten seconds after, it will show two separate minutes with 50 connections/minute each. That's a subtle but important distinction to be aware of.
    – dannysauer
    Feb 28, 2022 at 19:43
1

How about using a tool like awstat.

To do manually, you can count the log entries (requests) and then divide them by the number of seconds between the first and last request. So, you get the average req/sec.

Improve this answer
1

If you want to run relatively short performance tests, you can keep your eye on reqs or bytes/sec in real-time with apachetop. It's like top command in Linux and Unix, but provides a view to your Apache. It works by tailing the access log file.

I don't know if it's accurate enough for your purposes, but it definitely gives you some nice ballpark figures.

Improve this answer
1

I did it this way with gawk:

#!/usr/bin/awk -f
{
        date_hour=gensub(/^\[([^:]*:[0-2][0-9]):.*/,"\\1",1,$4)
        count[date_hour]++
}
END {
        for(i in count) {
                printf("%s %.2f%s\n",i":00",count[i]/3600,"qps")
        }
}
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.