How do I append/prepend a timestamp to grep output?

a script to log […] every 5 seconds […] I need timestamps […] a loop […] the file gets to a certain size

You want a tool that takes as its input the log output of your main program, and that writes it to a size-capped log file, and adds timestamps. Tools exist that do this and more. The "more" that they do is automatic rotation of the log file that is also triggerable on demand, maintaining a size-capped log directory of current and old log files.

You have a choice of tools:

  • Dan Bernstein's multilog from daemontools
  • Bruce Guenter's multilog from daemontools-encore
  • Laurent Bercot's s6-log from s6
  • Gerrit Pape's svlogd from runit
  • Wayne Marshall's tinylog from perp
  • My cyclog from nosh

Assume a long-running utility that prints your desired log output every few seconds, something like:

#!/bin/sh
# monitor-sensors
exec 2>&1
while true
do
    sensors
    sleep 5
done | grep --line-buffered -- '^Core'

Usage is as simple as:

  • monitor-sensors | cyclog ./temps
  • monitor-sensors | multilog t ./temps

and can be easily tweaked:

  • monitor-sensors | cyclog --max-file-size 32768 --max-total-size 1048576 ./temps
  • monitor-sensors | s6-log t s32768 S1048576 ./temps
  • monitor-sensors | multilog t s32768 n5 ./temps

From here, using toolsets like daemontools-encore/runit/perp/s6/nosh, it is but a small step to moving the left and right hand sides of this pipe into run programs and running this as a linked pair of actual dæmons.

But they can all, also, handle being spun up for one-off output to existing log directories, if you just want to (say) perform one-off invocations from a command line.

Some of these tools can do other forms of timestamps, but they can all do TAI64N timestamps. TAI64N timestamps are designed to be capable of nanosecond precision, albeit that some of the aforementioned do not quite implement this; are designed to quite happily cope with things like leap seconds, as long as your TZ database knows about them; and are trivial to sort, or even sort -m.

Convert from TAI64N timestamps to your current timezone's local time (or, given that it's just the TZ environment variable, an arbitrary timezone of your choosing) using tools such as:

  • Dan Bernstein's tai64nlocal from daemontools
  • Bruce Guenter's tai64nlocal from daemontools-encore
  • My tai64nlocal from nosh

Watch such logs as they are written with:

  • tail -F, albeit that tail has known problems when there are very speedy rotations. (This is one of several known problems with tail. Other known problems including handling in-place truncation of log files that are rapidly followed by more log data. This problem can be triggered by inferior systems such as logrotate. Fortunately, the aforementioned tools do not truncate files once they are fully written and do not risk such additional tail problems.)
  • My follow-log-directories from nosh, which "knows" this kind of log directory and uses a "cursor" (persistently maintained on disc) to reliably track the position in the log directory to continue from even if several rotations happen when the log follower isn't looking.

Other sorts of processing can be done with tools such as:

  • Russ Allbery's multilog-watch
  • logrange
  • Paul Kremer's multilog-stamptail
  • My export-to-rsyslog from nosh

Further reading

  • Jonathan de Boyne Pollard (2015). "Logging". The daemontools family. Frequently Given Answers.
  • Bryan Cantrill (2013). How a pleasure cruise became an odyssey. OmniIT Surge 2013. YouTube.
  • Bryan Cantrill (2012-07-29). Behavior of tail -f wrt truncation illumos-developers.
  • https://unix.stackexchange.com/a/294206/5132

@JdeBP already answered what you should probably be doing. But that wasn't the question you made. So to anyone arriving here from Google trying to know how to append/prepend things to a multi-line output, here it is:

First, ^Core* probably doesn't do what you think it does. It is a regular expression to match lines that start with "Cor" immediately followed by any number of "e"s: "Cor", "Core", "Coreeeee", etc.

Second, xargs is a great tool, especially for one-liners and dirty-quick scripts. You can have xargs to act on each line of input and issue a command for every one of them with the parameter -I. So you can easily do something like this:

sensors | grep '^Core*' | xargs -I{} echo "${stamp}: {}" >> temps.log

Put '^Core*' withing quotes because you don't want unexpected (though highly unlikely) shell expansions there. The {} in -I tells xargs to replace {} with each line of input in the arguments of the command it will be executing. You can choose another pattern, of course; I just like to mimic the syntax of find -exec. The ${stamp} assumes you saved the date/time in stamp.

You can also raise a subshell, run the date command (specifying whatever format you want) in it and have xargs use its standard output as part of your arguments to echo:

sensors | grep '^Core*' | xargs -I{} echo "$(date): {}" >> temps.log

Again, what you should probably be doing is using a tested and mantained tool for logging. But I'm answering here how to append/prepend a time stamp to grep output, not how to properly make your logs.