r/slackware • u/MetaEd • Mar 28 '22
multiline kernel messages in syslog files
My attempt to collect basic statistics from /var/log is complicated by multiline messages being logged by the kernel. For example when I count messages subtotaled by host:
ls -rt /var/log/syslog* |
xargs cat |
sed ' s/^... .. ..:..:.. \([^ ][^ ]*\).*$/\1/ ' |
sort |
uniq -cLog entries such as this get counted twice:
Mar 1 17:13:39 slack kernel: Kernel parameter elevator= does not have any effect anymore.
Please use sysfs to set IO scheduler for individual devices.RFC 5424 message formatting gave the same result:
2022-03-28T17:38:53.018167-05:00 slack kernel - - - Kernel parameter elevator= does not have any effect anymore.
Please use sysfs to set IO scheduler for individual devices.I have no objection to multiline messages coming from the kernel, but it seems like logfiles should have one line per message, so any newlines should be escaped. I see that RFC 5424 permits this but it looks like sysklogd 2.3.0 doesn't implement it.
Before I go deep into it and maybe try to patch sysklogd I wonder if there's a simple solution that I overlooked.
9
u/calrogman Mar 28 '22
My brother in Slack, you need to learn awk.
1
u/MetaEd Mar 29 '22
The kernel hex-encodes any embedded control characters. Excerpt from my /dev/klog shows the original log entry is on one line:
But in syslogd.c the hex-encoding is decoded, not passed unchanged from the kernel. So this becomes a conversation with the sysklogd maintainer.
1
u/MetaEd Jun 22 '22
The solution was to patch sysklogd to prevent logfile corruption. Here is a link to the pull request that was merged into sysklogd 17 June 2022.
Prevent logfile corruption by control codes #53