extract last 10 minutes from logfile

You can match the date range using simple string comparison, for example:

d1=$(date --date="-10 min" "+%b %_d %H:%M")
d2=$(date "+%b %_d %H:%M")
while read line; do
    [[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $line
done

For example if d1='Dec 18 10:19' and d2='Dec 18 10:27' then the output will be:

Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32

Or using awk if you wish:

awk -v d1="$d1" -v d2="$d2" '$0 > d1 && $0 < d2 || $0 ~ d2'

Introduction

This answer is something long, because there is 3 different way on thinking: 1) perl quick or exact, 2) pure bash and 3) perl script in bash function.

That's a (common) job for perl!:

Simple and efficient:

perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time($1)>time-600' /path/log

This version print last 10 minutes event, upto now, by using time function.

You could test this with:

sudo cat /var/log/syslog |
  perl -MDate::Parse -ne '
    print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time($1) > time-600'

Note that first representation use only firsts 15 chars from each lines, while second construct use more detailed regexp.

As a perl script: last10m.pl

#!/usr/bin/perl -wn

use strict;
use Date::Parse;
print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time($1) > time-600

Strictly: extract last 10 minutes from logfile

Meaning not relative to current time, but to last entry in logfile:

There is two way for retrieving end of period:

date -r logfile +%s
tail -n1 logfile | perl -MDate::Parse -nE 'say str2time($1) if /^(.{15})/'

Where logically, last modification time of the logfile must be the time of the last entry.

So the command could become:

perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time($1)>'$(
    date -r logfile +%s)

or you could take the last entry as reference:

perl -MDate::Parse -E 'open IN,"<".$ARGV[0];seek IN,-200,2;while (<IN>) {
    $ref=str2time($1) if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;};seek IN,0,0;
    while (<IN>) {print if /^(.{15})\s/&&str2time($1)>$ref-600}' logfile

Second version seem stronger, but access to file only once.

As a perl script, this could look like:

#!/usr/bin/perl -w

use strict;
use Date::Parse;
my $ref;                 # The only variable I will use in this.

open IN,"<".$ARGV[0];    # Open (READ) file submited as 1st argument
seek IN,-200,2;          # Jump to 200 character before end of logfile. (This
                         # could not suffice if log file hold very log lines! )
while (<IN>) {           # Until end of logfile...
    $ref=str2time($1) if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;
};                       # store time into $ref variable.
seek IN,0,0;             # Jump back to the begin of file
while (<IN>) {
    print if /^(.{15})\s/&&str2time($1)>$ref-600;
}

But if you really wanna use bash

There is a very quick pure bash script:

Warning: This use recent bashisms, require $BASH_VERSION 4.2 or higher.

#!/bin/bash

declare -A month

for i in {1..12};do
    LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400))
    month[$var]=$i
  done

printf -v now "%(%s)T" -1
printf -v ref "%(%m%d%H%M%S)T" $((now-600))

while read line;do
    printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \
        $((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \
        $((10#${line:13:2}))
    # echo " $crt < $ref ??"   # Uncomment this line to print each test
    [ $crt -gt $ref ] && break
done
cat

Store this script and run:

cat >last10min.sh
chmod +x last10min.sh
sudo cat /var/log/syslog | ./last10min.sh

Strictly: extract last 10 minutes from logfile

Simply replace line 10, but you have to place filename in the script and not use it as a filter:

#!/bin/bash

declare -A month

for i in {1..12};do
    LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400))
    month[$var]=$i
  done

read now < <(date -d "$(tail -n1 $1|head -c 15)" +%s)
printf -v ref "%(%m%d%H%M%S)T" $((now-600))

export -A month

{
    while read line;do
        printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \
            $((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \
            $((10#${line:13:2}))
        [ $crt -gt $ref ] && break
    done
    cat
} <$1

A perl script into a bash function

As commented by ajcg, this could be nice to put efficient perl script into a bash function:

recentLog(){ 
    perl -MDate::Parse -ne '
        print if/^(.{'${3:-15}'})\s/ &&
            str2time($1)>time-'$((
                60*${2:-10}
            )) ${1:-/var/log/daemon.log}
}

Usage:

recentLog [filename] [minutes] [time sting length]

  • filename of log file
  • minutes max before now of lines to show
  • time sting length from begin of lines (default 15).

In bash, you can use the date command to parse the timestamps. The "%s" format specifier converts the given date to the number of seconds since 1970-01-01 00:00:00 UTC. This simple integer is easy and accurate to do basic arithmetic on.

If you want the log messages from the last 10 minutes of actual time:

now10=$(($(date +%s) - (10 * 60)))

while read line; do
    [ $(date -d "${line:0:15}" +%s) -gt $now10 ] && printf "$line\n"
done < logfile

Note the ${line:0:15} expression is a bash parameter expansion which gives the first 15 characters of the line, i.e. the timestamp itself.

If you want the log messages from the last 10 minutes relative to the end of the log:

$ lastline=$(tail -n1 logfile)
$ last10=$(($(date -d "$lastline" +%s) - (10 * 60)))
$ while read line; do
> [ $(date -d "${line:0:15}" +%s) -gt $last10 ] && printf "$line\n"
> done < logfile
Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32
$ 

Here's a mild performance enhancement over the above:

$ { while read line; do
> [ $(date -d "${line:0:15}" +%s) -gt $last10 ] && printf "$line\n" && break
> done ; cat ; }  < logfile
Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32
$ 

This assumes the log entries are in strict chronological order. Once we match the timestamp in question, we exit the for loop, and then just use cat to dump the remaining entries.


Here is nice tool range is any you wish from -10 till now

sed -n "/^$(date --date='10 minutes ago' '+%b %_d %H:%M')/,\$p" /var/log/blaaaa