one common mistake that's made when using name-value pairs in log messages is
using the same key multiple times. Please try to add some protection against
that
Personally, I don't mind JSON, and the ability to have multi-level structures
can signficiantly clarify logs, but defining a way to encode a hierarcy in the
key field (for example, use . as a separator, so source.name source.ip
source.interface are related and clearly different thatn dest.name dest.ip
dest.interface)
David Lang
On Mon, 19 Jun 2023, Ricardo Signes via Devel wrote:
Devs,
For about three years, Cyrus has been moving to use the new xsyslog function for
syslogging, meant to have a more uniform format for easier parsing or scanning.
Meanwhile, inside Fastmail, we've begun moving to logfmt
<https://brandur.org/logfmt>, which is similar but not the same. This format
is pretty easy for a human to read, most of the time, but also easy for a machine to
read. I think Cyrus should start moving from the existing xsyslog format to logfmt.
We (Fastmail) are primarily using our own Log::Fmt Perl library
<https://metacpan.org/dist/Log-Dispatchouli/source/lib/Log/Fmt.pm>. You can read the logfmt
"spec <https://pkg.go.dev/github.com/kr/logfmt>", from the Go source. Our behavior
is not exactly the same, but I suspect the difference is largely theoretical as regards Cyrus. Our
behavior is:
• every logfmt line is a sequence of k=v pairs separated by spaces
• every key must only contain octets in [\x21\x23-\x3C\x3E-\x7E]
• any value that does not match the same rule as a key must be quoted and
escaped
• values are quoted by being placed inside "..."
• escaping…
• replaces \ with \\
• replaces " with \"
• replaces \x0a and \x0d with \n and \r respectively
• replaces any control character or vertical whitespace with \x{...} where
... is the hex value of the character
I'm sending this email after some discussion about using logfmt on our weekly call that
ended with me saying, "I'll send notes on just what we do in the Perl code."
------------------------------------------
Cyrus: Devel
Permalink:
https://cyrus.topicbox.com/groups/devel/T4f31bdf24bafdf26-Me85800e50c316c91171ebdac
Delivery options: https://cyrus.topicbox.com/groups/devel/subscription
