Rewriting logs received via syslog from another machine
The question is a bit vague but I'll try to propose a possible solution.
To rewrite messages rsyslog
provides a number of modules one of which is mmfields
.
It splits an incoming message at a certain character (just one character) into fields and
then allows to access these fields. For a message like
a=1 b=two c=3 d=four
the separator would be a blank and the fields are then accessible as
$!f2
, $!f3
, $!f4
, and $!f5
.
Unfortunately, the very first field ($!f1
) is always empty because the message is preceeded by
a space and that would be the first field. Thus, for the above message we get $!f1==""
,
$!f2=="a=1"
, $!f3=="b=two"
, $!f4=="c=3"
, and $!f5=="d=four"
.
rsyslog
ships with other message modification modules as well but in lack of further
details I chose this one. Store the following file as /etc/rsyslog.d/10-so.conf
.
Change the name according to the desired order of execution but keep the .conf
extension.
# Load the "Message Modification" module "mmfields" to split
# incoming messages into fields at a certain separator:
module(load="mmfields")
# Format used for messages that are forwarded to another host.
# The fields are re-ordered and field #3 is omitted.
template(name="rewrite_forward" type="list") {
constant(value="<")
property(name="pri")
constant(value=">")
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="syslogtag")
constant(value=" ")
property(name="$!f4")
constant(value=" ")
property(name="$!f2")
constant(value=" ")
property(name="$!f5")
}
# Format used for messages that are written to a local logfile.
# The format is almost the same as above, but lacks the priority,
# uses a different timestamp format, and ends with a "\n" as this
# is suitable for messages printed to a logfile.
template(name="rewrite_file" type="list") {
property(name="timestamp")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="syslogtag")
constant(value=" ")
property(name="$!f4")
constant(value=" ")
property(name="$!f2")
constant(value=" ")
property(name="$!f5")
constant(value="\n")
}
if ( $programname == "so" ) then {
# split message into fields at (exactly) one space character.
# The "fields" can then be referred to as "$!f1", "$!f2", ..., "$!fN".
# Note that "$!f1" will always be an empty string because the message
# usually starts with a blank and that is considered to be the first
# field.
action( type="mmfields"
separator=" "
)
# If the second field is the string "b=2", then go ahead and log and
# forward the message. Change the condition to your liking.
if ($!f3 == "b=2") then {
# write rewritten logmessage to local file
action( type = "omfile"
file = "/var/log/so-rewrite-fields.log"
template = "rewrite_file"
)
# just for reference: write the unmodified message to the
# very same logfile. Drop this for actual processing. It
# serves just as a debugging aid.
action( type = "omfile"
file = "/var/log/so-rewrite-fields.log"
)
# forward rewritten message to another host
action( type = "omfwd"
target = "127.0.0.1" # change to actual destination
port = "514" # change to actual destination
protocol = "udp" # change to actual destination
template = "rewrite_forward"
)
}
# no further processing:
stop
}
Restart rsyslog
(via sudo systemctl restart rsyslog.service
) and try it out:
# A message that won't be logged because `$!f3 != "b=2"`:
logger -t so --id=$$ "a=1 b=3 txt=Hello number=$RANDOM"
# A message that will be logged because `$!f3 == "b=2"`:
logger -t so --id=$$ "a=1 b=2 txt=Hello number=$RANDOM"
The output of the second logger
statement will be:
Mar 14 21:40:34 stratum9 so[3533]: txt=Hello a=1 number=6484 # rewritten msg
Mar 14 21:40:34 stratum9 so[3533]: a=1 b=2 txt=Hello number=6484 # original msg
To change the hostname, simply replace
constant(value=" ")
property(name="hostname")
constant(value=" ")
in the templates with
constant(value=" fromElsewhere ")
To change the syslogtag (what you called service), replace
constant(value=" ")
property(name="syslogtag")
constant(value=" ")
with
constant(value=" otherService: ")
The output will then be:
Mar 14 22:05:51 fromElsewhere otherService: txt=Hello a=1 number=11763 # rewritten
Mar 14 22:05:51 stratum9 so[3533]: a=1 b=2 txt=Hello number=11763 # original
See here for further message properties.
Note that my approach (with mmfields
) relies on the fields to always have the same order
and does not easily allow for rewriting messages like a=1 b=2
to b=1 a=2
(reorder and change
key-value-pairs). For that, another module might be more appropriate.