Making compression extensible

[Volkan Yazıcı]

Recently, somebody asked for zstd compression
. I indicated that
adding zstd would be nice, but ideally we should make the compression
extensible [by means of plugins]. As usual, even though I offered PR
assistance, the user has individually submitted a PR only solving their
problem: adding zstd to the rolling appender. I am not inclined to accept
these kinds of "add button X to dialer Y" contributions, where I think
Log4j suffers a lot from.

AFAIC, there should be plugins implementing a certain compressor interface,
a registry of these, and the use-site (`RollingAppender`, `GelfLayout`,
etc.) accessing these compression schemes via registry. This will not only
enable users to add their custom compression scheme, but also other
components to take advantage of this. As a particular example, more or less
the only thing `JsonTemplateLayout` falls short compared to `GelfLayout` is
compression support; though this could have perfectly been solved via a
[wrapper] `CompressingLayout` I suggested in LOG4J2-3023.

I would appreciate some feedback on my thinking process here. When it comes
to Log4j, I am really allergic to specialized solutions ignoring the rest
of the code base.

Re: Making compression extensible

[Gary Gregory]

I agree with Volkan, we should not accept a one-off. Instead, I would
prefer we generically support whatever Apache Commons Compress supports. I
can help beef up Compress if we need anything in there to make Log4j better.

Gary

On Sun, Jun 18, 2023, 14:52 Volkan Yazıcı  wrote:

> Recently, somebody asked for zstd compression
> . I indicated that
> adding zstd would be nice, but ideally we should make the compression
> extensible [by means of plugins]. As usual, even though I offered PR
> assistance, the user has individually submitted a PR only solving their
> problem: adding zstd to the rolling appender. I am not inclined to accept
> these kinds of "add button X to dialer Y" contributions, where I think
> Log4j suffers a lot from.
>
> AFAIC, there should be plugins implementing a certain compressor interface,
> a registry of these, and the use-site (`RollingAppender`, `GelfLayout`,
> etc.) accessing these compression schemes via registry. This will not only
> enable users to add their custom compression scheme, but also other
> components to take advantage of this. As a particular example, more or less
> the only thing `JsonTemplateLayout` falls short compared to `GelfLayout` is
> compression support; though this could have perfectly been solved via a
> [wrapper] `CompressingLayout` I suggested in LOG4J2-3023.
>
> I would appreciate some feedback on my thinking process here. When it comes
> to Log4j, I am really allergic to specialized solutions ignoring the rest
> of the code base.
>

Re: Making compression extensible

[Ralph Goers]

First, and foremost, we don’t want Log4j to have required dependencies. So 
whatever the JDK supports out of the box we need to take advantage of. Whatever 
it doesn’t, I agree we should delegate that to Apache Commons.

Ralph

> On Jun 18, 2023, at 12:00 PM, Gary Gregory  wrote:
> 
> I agree with Volkan, we should not accept a one-off. Instead, I would
> prefer we generically support whatever Apache Commons Compress supports. I
> can help beef up Compress if we need anything in there to make Log4j better.
> 
> Gary
> 
> On Sun, Jun 18, 2023, 14:52 Volkan Yazıcı  wrote:
> 
>> Recently, somebody asked for zstd compression
>> . I indicated that
>> adding zstd would be nice, but ideally we should make the compression
>> extensible [by means of plugins]. As usual, even though I offered PR
>> assistance, the user has individually submitted a PR only solving their
>> problem: adding zstd to the rolling appender. I am not inclined to accept
>> these kinds of "add button X to dialer Y" contributions, where I think
>> Log4j suffers a lot from.
>> 
>> AFAIC, there should be plugins implementing a certain compressor interface,
>> a registry of these, and the use-site (`RollingAppender`, `GelfLayout`,
>> etc.) accessing these compression schemes via registry. This will not only
>> enable users to add their custom compression scheme, but also other
>> components to take advantage of this. As a particular example, more or less
>> the only thing `JsonTemplateLayout` falls short compared to `GelfLayout` is
>> compression support; though this could have perfectly been solved via a
>> [wrapper] `CompressingLayout` I suggested in LOG4J2-3023.
>> 
>> I would appreciate some feedback on my thinking process here. When it comes
>> to Log4j, I am really allergic to specialized solutions ignoring the rest
>> of the code base.
>>