Indeed ColumnMapping is an alternative approach. I believe due to its
origin, i.e., RDBMSes, it facilitates a 1-to-1 mapping particularly
targeted at table column names, e.g.,
timestamp -> instant
message -> summary
One can borrow the idea and extend it a little bit to support the
structured form of a JSON:
timestamp.millis -> instant
throwable -> error.message
That said, I believe current JSON-based templating in LogstashLayout
communicates the intent more concisely (the template is WYSIWYG) and
allows extra (JSON-specific) flexibility, e.g.,
"user": ["${json:mdc:user}", "${json:mdc:fallbackUser}"]
On Tue, Dec 24, 2019 at 3:54 PM Matt Sicker <boa...@gmail.com> wrote:
>
> Perhaps the ColumnMapping plugin from the database appenders could be
> useful here?
>
> On Tue, Dec 24, 2019 at 03:09 Volkan Yazıcı <volkan.yaz...@gmail.com> wrote:
>
> > This is easier said than done. Customizing the schema and injected
> > content is the raison d'être of LogstashLayout. Would you people
> > accept a PR re-branding LogstashLayout as the new JsonLayout where the
> > default JSON schema conforms with the JsonLayout of Log4j 2.0? This
> > would provide 100% backward-compatibility with the existing JsonLayout
> > and introduce all the benefits (i.e., configuration knobs, zero TLA)
> > of LogstashLayout.
> >
> > @Ralph, do you have any plans on your mind for the API to expose to
> > the user for customizing the JSON schema? In LogstashLayout, I use an
> > ad-hoc JSON templating -- see Log4j2JsonLayout.json[1], for instance.
> > I am not proud of this way of templating, you can't express every kind
> > of transformation, e.g., "message": "foo bar baz ${json:message}" will
> > not yield the output what one would expect it to. That said, it is
> > simple and powerful enough to empower thousands of users so far.
> > Further, I have never received any single complaints about its
> > shortcomings. (If there are bright minds in the room with some
> > JsonT[2] and/or fn:xml-to-json[3], please chime in.)
> >
> > For XML layout, we can add a configuration knob for XSLT
> > transformation that kicks in at rendering time. How to perform this
> > "efficiently" (e.g., can we compile the transformation once and reuse
> > it as in LogstashLayout) is an open question. We can work that detail
> > out in later releases since it will be a user-invisible enhancement.
> > (Given both XML and HTML are SGML derivatives, can't we employ the
> > same XSLT transformation magic in HtmlLayout design too?)
> >
> > For YAML layout... Is anybody using it at all? If so, I am really
> > curious about the use cases. If there is a demand, we can introduce
> > custom schema templating (ala LogstashLayout) to YamlLayout too.
> > Though judging from my experience in LogstashLayout, this is quite
> > some work. The absence of any YAML schema transformation tools in the
> > F/OSS market is also a strong indication that this is not trivial and
> > implies many subtle corner cases that needs to be brushed.
> >
> > I would like to share some remarks about introducing dependencies
> > (e.g., Jackson) here.
> >
> > - For JSON rendering, we can do without Jackson. For
> > instance log4j2-ecs-layout[4] renders JSON using its own
> > simple single-class-file tooling and *optionally* falls back
> > to Jackson for serializing custom Message objects.
> >
> > - Ever single extra dependency on top of Log4j core
> > increments the exponent of dev(ops) nightmare.I find
> > myself explaining how to properly package
> > log4j2-logstash-layout to dev(ops) people a couple
> > of times every year.
> >
> > In case you wonder, my intention is to collect feedback, conclude on a
> > blueprint blessed by existing contributors, break the blueprint down
> > into structured JIRA stories, and start working on them.
> >
> > [1]
> > https://github.com/vy/log4j2-logstash-layout/blob/master/layout/src/test/resources/Log4j2JsonLayout.json
> > [2] https://goessner.net/articles/jsont/
> > [3] https://www.w3.org/TR/xslt-30/#func-xml-to-json
> > [4]
> > https://github.com/elastic/java-ecs-logging/tree/master/log4j2-ecs-layout
> >
> > On Tue, Dec 24, 2019 at 12:25 AM Ralph Goers <ralph.go...@dslextreme.com>
> > wrote:
> > > What I was thinking was that I would just like a more
> > > flexible way to specify what should be included in the
> > > JSON. In its current form the JSON Layout just creates
> > > a serialized version of a LogEvent. I would prefer it if
> > > users could define the individual fields they want and
> > > what they should contain.
> >
> --
> Matt Sicker <boa...@gmail.com>
