Re: Dynamically updating filters across many instances

2022-04-10 Thread Ralph Goers
Well, you are correct that if a bug creeps in then it effects everyone. That said, currently our apps do not dynamically reconfigure logging. They only pick up changes on restart. That said, something is restarting many times a day, so any bug like that gets caught pretty quickly. And since our

Re: Dynamically updating filters across many instances

2022-04-10 Thread Dominik Psenner
Ralph, I do not believe that there exists a "one size fits it all" solution. Yes, running in debug log has drawbacks and should not be necessary. And yet I learned that only debug logs may provide enough insights. The cause for this is simple: developers test code at "debug" level and when time is

Re: Dynamically updating filters across many instances

2022-04-09 Thread Ralph Goers
What you propose below is exactly what I described in my previous post. Whether it makes its way into Log4J will depend on how generally useful it ends up being. As for distributed tracing, yes, of course it is, it is the collection of logs for a single user or account across all the components

Re: Dynamically updating filters across many instances

2022-04-08 Thread Volkan Yazıcı
What you are describing sounds to me more fitting for distributed tracing rather than logging, but I will skip that discussion for now. Rather than "hacking" a Log4j component to deliver this functionality, I'd simply implement a custom Filter keeping a cache of user IDs to be inspected. This cach

Re: Dynamically updating filters across many instances

2022-04-07 Thread Ralph Goers
Dominik, What you are suggesting is NOT a best practice. Logging at debug or trace in production can impact application performance and can be prohibitively expensive. If that were the recommended practice then Logging frameworks wouldn’t even bother providing the ability to filter log messages

Re: Dynamically updating filters across many instances

2022-04-07 Thread Dominik Psenner
Hi Ralph I do not necessarily see this as a problem of the logging configuration. Yes, it can be solved with the logging configuration but you could do otherwise. You could log at debug and roll the logs quickly. Further poll and aggregate the logs in regular and short intervals with a platform li

Re: Dynamically updating filters across many instances

2022-04-06 Thread Ralph Goers
Thanks Robert. Yes. We want to log everything that happens for a single user or a single account across all the services that may get called. Viewing the logs is not a problem. We route logs for everything to Elasticsearch and can view them with Kibana. The issue is simply in getting Log4j to

Re: Dynamically updating filters across many instances

2022-04-06 Thread Robert Middleton
So if I'm understanding you correctly, you want to do live debugging of one user's information, but you can't do that because that information is not going to the logs. In order to get that information, some part of log4j would need to be reconfigured in order to send that information out to the l

Dynamically updating filters across many instances

2022-04-06 Thread Ralph Goers
I’m looking for some inspiration. At work we use Spring Cloud Config and host our logging configuration there. It is shared by something like 150 services, which comes out to hundreds of service instances. We have a standard of including the user’s id and the customer’s account number in the Th