Is there any way to show the most requested URL from apache over a timeframe, e.g. the most requested URL over the past 2 hours.
Is this type of thing possible with mod_status or could I aggregate the access logs?
Did you try awstat? It is log file analyzer tool. I am not sure if it can provide you with the stats within two hours as you requested.
You can also use a command like this to see the visited pages and its count within your log file.
$ awk {'print $7'} /var/log/apache2/access.log | sort | uniq -c
The timeframe will depend on the contents of the log file. You can combine this with other commands like head
and tail
.
goaccess is my favorite tool for this task https://goaccess.io/.
WORKING WITH DATES
Another useful pipe would be filtering dates out of the web log
The following will get all HTTP requests starting on 05/Dec/2010 until the end of the file.
# sed -n '/05Dec2010/,$ p' access.log | goaccess -a -
or using relative dates such as yesterdays or tomorrows day:
# sed -n '/'$(date '+%d%b%Y' -d '1 week ago')'/,$ p' access.log | goaccess -a -
If we want to parse only a certain time-frame from DATE a to DATE b, we can do:
# sed -n '/5Nov2010/,/5Dec2010/ p' access.log | goaccess -a -