How to measure req/sec by analyzing apache logs

Solution 1:

This great article helped me a lot...

http://www.inmotionhosting.com/support/website/server-usage/view-level-of-traffic-with-apache-access-log

I had created a set of prepered commands that I am using to analyze apache log:

request per hour
cat access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

request per hour by date
grep "23/Jan" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

request per hour by IP
grep "XX.XX.XX.XX" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

requests per minute:
cat access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

requests per minute for date:
grep "02/Nov/2017" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

requests per minute for url:
grep "[url]" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

per IP per minute
grep "XX.XX.XX.XX" access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c

hopes it will help anyone who's looking for it...

Solution 2:

In real time you could use mod_status. You could also count the lines in your access.log over a given period and work out the rate from that. Something like this

#!/bin/bash
LOGFILE=/var/log/apache2/access.log
STATFILE=/var/tmp/apachestats
START=$(wc -l "$LOGFILE" | awk '{print $1}')
PERIOD=10
PRECISION=2
sleep "$PERIOD"
while true
do
    HITSPERSECOND=0
    HITS=$(wc -l "$LOGFILE" | awk '{print $1}')
    NEWHITS=$(( HITS - START ))
    if [[ "$NEWHITS" > 0 ]]
    then
        START=$HITS
        HITSPERSECOND=$(echo -e "scale=$PRECISION\n$NEWHITS / $PERIOD" | bc -l )
    fi
    echo "$(date) rate was $HITSPERSECOND" >>"$STATFILE"
    sleep "$PERIOD"
done