Re: Monitoring discards from async logging

[Thomas, Adam]

> What version of Java are you using? If it is Java 17

Unfortunately, I need to support Java 8 and 11.

> you can still configure Log4j Core to use your own implementation of 
> `AsyncQueueFullPolicy` by setting the `log4j2.asyncQueueFullPolicy` system 
> property

I would do this if I had control of the end user's logging setup. 
Unfortunately, I vend an internal library that is used by a lot of different 
teams. By the time my code is invoked, AsyncLoggerDisruptor has already read 
the system property and initialized the queue full policy. For other log4j 
telemetry I just programmatically inspect the LoggerContext when my code 
initializes, and install a property change listener to listen for configuration 
changes. Even if I vend my own custom AsyncQueueFullPolicy, practically 
speaking I will never get very good coverage, because I would have to go 
convince thousands of developers to change their system properties.

-Adam

On 4/9/24, 4:02 PM, "Piotr P. Karwasz" mailto:piotr.karw...@gmail.com>> wrote:

Hi Adam,


On Wed, 10 Apr 2024 at 00:11, Thomas, Adam mailto:adamt...@amazon.com.inva>lid> wrote:
> I created a pull request[1] late last year for this, and was encouraged to 
> start a discussion on the dev list at the time.


Be reassured that we didn't forget about your PR, we just have a lot
of changes going on to publish Log4j Core 3.x.


> We run log4j2 on a lot of hosts that are operated by different teams. By 
> default, our users use async logging and use the Discard 
> asyncQueueFullPolicy. As far as I can tell, the status logger gets one WARN 
> message the first time an event is discarded, and during shutdown a TRACE 
> message is emitted if any discards happened. This leaves us with essentially 
> no insight as to whether or not events are actually being discarded, when 
> they are being discarded, or how many we are discarding.


What version of Java are you using? If it is Java 17 or later, you
could use Log4j Core 3.x that is based on a small dependency injection
implementation. While this technical detail is of no importance to
most users, in your case you could use an `InstancePostProcessor`[1]
to intercept the creation of each service class (AsyncQueueFullPolicy
in your case) and instrument it.


If you are running Java 8 or Java 11, you can still configure Log4j
Core to use your own implementation of `AsyncQueueFullPolicy` by
setting the `log4j2.asyncQueueFullPolicy` system property.


Piotr


[1] 
https://github.com/apache/logging-log4j2/blob/main/log4j-plugins/src/main/java/org/apache/logging/log4j/plugins/di/spi/InstancePostProcessor.java
 

Re: Monitoring discards from async logging

[Piotr P. Karwasz]

Hi Adam,

On Wed, 10 Apr 2024 at 19:03, Thomas, Adam  wrote:
> I would do this if I had control of the end user's logging setup. 
> Unfortunately, I vend an internal library that is used by a lot of different 
> teams. By the time my code is invoked, AsyncLoggerDisruptor has already read 
> the system property and initialized the queue full policy. For other log4j 
> telemetry I just programmatically inspect the LoggerContext when my code 
> initializes, and install a property change listener to listen for 
> configuration changes. Even if I vend my own custom AsyncQueueFullPolicy, 
> practically speaking I will never get very good coverage, because I would 
> have to go convince thousands of developers to change their system properties.

Log4j supports multiple property sources, not only Java system properties:

https://logging.apache.org/log4j/2.x/manual/configuration#SystemProperties

All you need is a `log4j2.component.properties` file in your library.

Regarding your original PR, we are working hard to reach a definitive
Log4j Core 3.x version. Once that is achieved, we can propose a way to
get metrics that will work in both 2.x and 3.x. Until then I am afraid
that all libraries that propose metrics and traces in Log4j Core will
need to have specific version for each release.

Piotr

Re: Monitoring discards from async logging

[Thomas, Adam]

> All you need is a `log4j2.component.properties` file in your library.

It is very common practice here to explicitly specify the discard policy as a 
system property, as it was recommended to teams moving from log4j1 to log4j2. 
Adding this would have very little impact.

> Until then I am afraid that all libraries that propose metrics and traces in 
> Log4j Core will need to have specific version for each release.

I can cope with different log4j versions having different ways to discover 
metrics, I understand a major version increase is likely to be disruptive. I 
also completely understand not wanting to add this as a first-class property of 
`LoggerContext`. Would publishing this value via JMX be more palatable? I 
believe you've removed JMX from 3.x, so this hopefully would not collide with 
whatever instrumentation you decided to backport to 2.x.

-Adam

On 4/10/24, 10:42 AM, "Piotr P. Karwasz" mailto:piotr.karw...@gmail.com>> wrote:

Hi Adam,


On Wed, 10 Apr 2024 at 19:03, Thomas, Adam mailto:adamt...@amazon.com.inva>lid> wrote:
> I would do this if I had control of the end user's logging setup. 
> Unfortunately, I vend an internal library that is used by a lot of different 
> teams. By the time my code is invoked, AsyncLoggerDisruptor has already read 
> the system property and initialized the queue full policy. For other log4j 
> telemetry I just programmatically inspect the LoggerContext when my code 
> initializes, and install a property change listener to listen for 
> configuration changes. Even if I vend my own custom AsyncQueueFullPolicy, 
> practically speaking I will never get very good coverage, because I would 
> have to go convince thousands of developers to change their system properties.


Log4j supports multiple property sources, not only Java system properties:


https://logging.apache.org/log4j/2.x/manual/configuration#SystemProperties 



All you need is a `log4j2.component.properties` file in your library.


Regarding your original PR, we are working hard to reach a definitive
Log4j Core 3.x version. Once that is achieved, we can propose a way to
get metrics that will work in both 2.x and 3.x. Until then I am afraid
that all libraries that propose metrics and traces in Log4j Core will
need to have specific version for each release.


Piotr