Re: Log4j2 replacement of PropertyConfigurator APIs configureAndWatch and configure
Just a follow up on this, do we have replacement for
"LogManager.resetConfiguration" in log4j2?
On Sat, Mar 18, 2023 at 11:27 AM Viraj Jasani wrote:
> Ah looks like monitorinterval is supported as per the doc[1]:
> Properties configuration files support the advertiser, monitorInterval,
> name, packages, shutdownHook, shutdownTimeout, status, verbose, and dest
> attributes.
>
> 1.
> https://logging.apache.org/log4j/2.x/manual/configuration.html#ConfigurationSyntax
>
> On Sat, Mar 18, 2023 at 11:14 AM Viraj Jasani wrote:
>
>> Thanks Ralph for the recommendations!
>>
>> > First, I strongly recommend you switch from properties to either XML,
>> Yaml, or JSON.
>>
>> The reason why we would still like to stick to properties (at least for
>> now during the migration) is that LOG4J2-3341 allows setting level and
>> appender at once in the properties file. After migration, if there is any
>> better way, we would consider migrating to xml or yaml for sure. Here is
>> the old thread for the reference[1] after which both hbase and hadoop have
>> decided to stick to properties for now.
>>
>> > Note that there are several variations of the initialize method.
>>
>> Sure I think we might be able to use, will explore:
>> public static LoggerContext initialize(final String name, final
>> ClassLoader loader, final URI configLocation) {
>> return initialize(name, loader, configLocation, null);
>> }
>>
>> Even with properties file, is it still possible to configure monitor
>> interval?
>>
>>
>> 1. https://lists.apache.org/thread/gvfb3jkg6t11cyds4jmpo7lrswmx28w3
>>
>> On Fri, Mar 17, 2023 at 8:59 PM Viraj Jasani wrote:
>>
>>> Hi,
>>>
>>> Could you please help with log4j2 replacement for PropertyConfigurator
>>> APIs configureAndWatch and configure?
>>>
>>> For instance, this is the logic we have that we need to migrate to
>>> log4j2:
>>>
>>>
>>>
>>> protected void initLog() throws ServerException {
>>> verifyDir(logDir);
>>> LogManager.resetConfiguration();
>>> File log4jFile = new File(configDir, name + "-log4j.properties");
>>> if (log4jFile.exists()) {
>>> PropertyConfigurator.configureAndWatch(log4jFile.toString(), 10 *
>>> 1000); //every 10 secs
>>> log = LoggerFactory.getLogger(Server.class);
>>> } else {
>>> Properties props = new Properties();
>>> try {
>>> InputStream is = getResource(DEFAULT_LOG4J_PROPERTIES);
>>> try {
>>> props.load(is);
>>> } finally {
>>> is.close();
>>> }
>>> } catch (IOException ex) {
>>> throw new ServerException(ServerException.ERROR.S03,
>>> DEFAULT_LOG4J_PROPERTIES, ex.getMessage(), ex);
>>> }
>>> PropertyConfigurator.configure(props);
>>> log = LoggerFactory.getLogger(Server.class);
>>> log.warn("Log4j [{}] configuration file not found, using default
>>> configuration from classpath", log4jFile);
>>> }
>>> }
>>>
>>>
>>>
Re: Log4j2 replacement of PropertyConfigurator APIs configureAndWatch and configure
I believe we can use "Configurator.reconfigure(uri)", however just wanted
to make sure if this is the only right way (stable API).
On Tue, Mar 21, 2023 at 4:57 PM Viraj Jasani wrote:
> Just a follow up on this, do we have replacement for
> "LogManager.resetConfiguration" in log4j2?
>
>
> On Sat, Mar 18, 2023 at 11:27 AM Viraj Jasani wrote:
>
>> Ah looks like monitorinterval is supported as per the doc[1]:
>> Properties configuration files support the advertiser, monitorInterval,
>> name, packages, shutdownHook, shutdownTimeout, status, verbose, and dest
>> attributes.
>>
>> 1.
>> https://logging.apache.org/log4j/2.x/manual/configuration.html#ConfigurationSyntax
>>
>> On Sat, Mar 18, 2023 at 11:14 AM Viraj Jasani wrote:
>>
>>> Thanks Ralph for the recommendations!
>>>
>>> > First, I strongly recommend you switch from properties to either XML,
>>> Yaml, or JSON.
>>>
>>> The reason why we would still like to stick to properties (at least for
>>> now during the migration) is that LOG4J2-3341 allows setting level and
>>> appender at once in the properties file. After migration, if there is any
>>> better way, we would consider migrating to xml or yaml for sure. Here is
>>> the old thread for the reference[1] after which both hbase and hadoop have
>>> decided to stick to properties for now.
>>>
>>> > Note that there are several variations of the initialize method.
>>>
>>> Sure I think we might be able to use, will explore:
>>> public static LoggerContext initialize(final String name, final
>>> ClassLoader loader, final URI configLocation) {
>>> return initialize(name, loader, configLocation, null);
>>> }
>>>
>>> Even with properties file, is it still possible to configure monitor
>>> interval?
>>>
>>>
>>> 1. https://lists.apache.org/thread/gvfb3jkg6t11cyds4jmpo7lrswmx28w3
>>>
>>> On Fri, Mar 17, 2023 at 8:59 PM Viraj Jasani wrote:
>>>
Hi,
Could you please help with log4j2 replacement for PropertyConfigurator
APIs configureAndWatch and configure?
For instance, this is the logic we have that we need to migrate to
log4j2:
protected void initLog() throws ServerException {
verifyDir(logDir);
LogManager.resetConfiguration();
File log4jFile = new File(configDir, name + "-log4j.properties");
if (log4jFile.exists()) {
PropertyConfigurator.configureAndWatch(log4jFile.toString(), 10 *
1000); //every 10 secs
log = LoggerFactory.getLogger(Server.class);
} else {
Properties props = new Properties();
try {
InputStream is = getResource(DEFAULT_LOG4J_PROPERTIES);
try {
props.load(is);
} finally {
is.close();
}
} catch (IOException ex) {
throw new ServerException(ServerException.ERROR.S03,
DEFAULT_LOG4J_PROPERTIES, ex.getMessage(), ex);
}
PropertyConfigurator.configure(props);
log = LoggerFactory.getLogger(Server.class);
log.warn("Log4j [{}] configuration file not found, using default
configuration from classpath", log4jFile);
}
}
Re: Log4j2 replacement of PropertyConfigurator APIs configureAndWatch and configure
Actually, Configurator.reconfigure() will cause the existing configuration
(file or uri) to be reloaded. You would use it with the uri or configuration
parameter if you want to load a new configuration.
Ralph
> On Mar 21, 2023, at 5:48 PM, Viraj Jasani wrote:
>
> I believe we can use "Configurator.reconfigure(uri)", however just wanted
> to make sure if this is the only right way (stable API).
>
>
> On Tue, Mar 21, 2023 at 4:57 PM Viraj Jasani wrote:
>
>> Just a follow up on this, do we have replacement for
>> "LogManager.resetConfiguration" in log4j2?
>>
>>
>> On Sat, Mar 18, 2023 at 11:27 AM Viraj Jasani wrote:
>>
>>> Ah looks like monitorinterval is supported as per the doc[1]:
>>> Properties configuration files support the advertiser, monitorInterval,
>>> name, packages, shutdownHook, shutdownTimeout, status, verbose, and dest
>>> attributes.
>>>
>>> 1.
>>> https://logging.apache.org/log4j/2.x/manual/configuration.html#ConfigurationSyntax
>>>
>>> On Sat, Mar 18, 2023 at 11:14 AM Viraj Jasani wrote:
>>>
Thanks Ralph for the recommendations!
> First, I strongly recommend you switch from properties to either XML,
Yaml, or JSON.
The reason why we would still like to stick to properties (at least for
now during the migration) is that LOG4J2-3341 allows setting level and
appender at once in the properties file. After migration, if there is any
better way, we would consider migrating to xml or yaml for sure. Here is
the old thread for the reference[1] after which both hbase and hadoop have
decided to stick to properties for now.
> Note that there are several variations of the initialize method.
Sure I think we might be able to use, will explore:
public static LoggerContext initialize(final String name, final
ClassLoader loader, final URI configLocation) {
return initialize(name, loader, configLocation, null);
}
Even with properties file, is it still possible to configure monitor
interval?
1. https://lists.apache.org/thread/gvfb3jkg6t11cyds4jmpo7lrswmx28w3
On Fri, Mar 17, 2023 at 8:59 PM Viraj Jasani wrote:
> Hi,
>
> Could you please help with log4j2 replacement for PropertyConfigurator
> APIs configureAndWatch and configure?
>
> For instance, this is the logic we have that we need to migrate to
> log4j2:
>
>
>
> protected void initLog() throws ServerException {
>verifyDir(logDir);
>LogManager.resetConfiguration();
>File log4jFile = new File(configDir, name + "-log4j.properties");
>if (log4jFile.exists()) {
> PropertyConfigurator.configureAndWatch(log4jFile.toString(), 10 *
> 1000); //every 10 secs
> log = LoggerFactory.getLogger(Server.class);
>} else {
> Properties props = new Properties();
> try {
>InputStream is = getResource(DEFAULT_LOG4J_PROPERTIES);
>try {
> props.load(is);
>} finally {
> is.close();
>}
> } catch (IOException ex) {
>throw new ServerException(ServerException.ERROR.S03,
> DEFAULT_LOG4J_PROPERTIES, ex.getMessage(), ex);
> }
> PropertyConfigurator.configure(props);
> log = LoggerFactory.getLogger(Server.class);
> log.warn("Log4j [{}] configuration file not found, using default
> configuration from classpath", log4jFile);
>}
> }
>
>
>
Re: Log4j2 replacement of PropertyConfigurator APIs configureAndWatch and configure
Thanks a lot, Ralph!
Let me ask one more question. For any further questions, I would create a
new mail thread.
We also have a utility in hadoop that tries to add FileAppender at runtime,
and also change the threshold of all appenders from root level. Is this
doable with log4j2? I could not find Appender or AbstractAppender method to
set filter/threshold to the given appender at runtime.
IIUC, we can use something like
"*setFilter(ThresholdFilter.**createFilter(Level.TRACE,
null, null))*" to the builder class while creating the appender instance,
but I am not sure if we can update the threshold for the existing appender.
Here is the code reference (log4j1):
https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/AdHocLogDumper.java#L71-L103
On Tue, Mar 21, 2023 at 5:48 PM Viraj Jasani wrote:
> I believe we can use "Configurator.reconfigure(uri)", however just wanted
> to make sure if this is the only right way (stable API).
>
>
> On Tue, Mar 21, 2023 at 4:57 PM Viraj Jasani wrote:
>
>> Just a follow up on this, do we have replacement for
>> "LogManager.resetConfiguration" in log4j2?
>>
>>
>> On Sat, Mar 18, 2023 at 11:27 AM Viraj Jasani wrote:
>>
>>> Ah looks like monitorinterval is supported as per the doc[1]:
>>> Properties configuration files support the advertiser, monitorInterval,
>>> name, packages, shutdownHook, shutdownTimeout, status, verbose, and dest
>>> attributes.
>>>
>>> 1.
>>> https://logging.apache.org/log4j/2.x/manual/configuration.html#ConfigurationSyntax
>>>
>>> On Sat, Mar 18, 2023 at 11:14 AM Viraj Jasani
>>> wrote:
>>>
Thanks Ralph for the recommendations!
> First, I strongly recommend you switch from properties to either XML,
Yaml, or JSON.
The reason why we would still like to stick to properties (at least for
now during the migration) is that LOG4J2-3341 allows setting level and
appender at once in the properties file. After migration, if there is any
better way, we would consider migrating to xml or yaml for sure. Here is
the old thread for the reference[1] after which both hbase and hadoop have
decided to stick to properties for now.
> Note that there are several variations of the initialize method.
Sure I think we might be able to use, will explore:
public static LoggerContext initialize(final String name, final
ClassLoader loader, final URI configLocation) {
return initialize(name, loader, configLocation, null);
}
Even with properties file, is it still possible to configure monitor
interval?
1. https://lists.apache.org/thread/gvfb3jkg6t11cyds4jmpo7lrswmx28w3
On Fri, Mar 17, 2023 at 8:59 PM Viraj Jasani
wrote:
> Hi,
>
> Could you please help with log4j2 replacement for PropertyConfigurator
> APIs configureAndWatch and configure?
>
> For instance, this is the logic we have that we need to migrate to
> log4j2:
>
>
>
> protected void initLog() throws ServerException {
> verifyDir(logDir);
> LogManager.resetConfiguration();
> File log4jFile = new File(configDir, name + "-log4j.properties");
> if (log4jFile.exists()) {
> PropertyConfigurator.configureAndWatch(log4jFile.toString(), 10
> * 1000); //every 10 secs
> log = LoggerFactory.getLogger(Server.class);
> } else {
> Properties props = new Properties();
> try {
> InputStream is = getResource(DEFAULT_LOG4J_PROPERTIES);
> try {
> props.load(is);
> } finally {
> is.close();
> }
> } catch (IOException ex) {
> throw new ServerException(ServerException.ERROR.S03,
> DEFAULT_LOG4J_PROPERTIES, ex.getMessage(), ex);
> }
> PropertyConfigurator.configure(props);
> log = LoggerFactory.getLogger(Server.class);
> log.warn("Log4j [{}] configuration file not found, using default
> configuration from classpath", log4jFile);
> }
> }
>
>
>
Re: Log4j2 replacement of PropertyConfigurator APIs configureAndWatch and configure
Yes, it is doable. That said, there are almost always better ways to implement
whatever your “real” requirement is.
Looking at what that code is doing it is just enabling logging to a file at a
level for some time period. Instead, you can just create the Appender in your
configuration and configure it with a dynamic filter similar to the way
MutableThreadContextMapFilter works. See
https://logging.apache.org/log4j/2.x/manual/filters.html#MutableThreadContextMapFilter.
My employer uses this and has a service backed by Redis where our operations
folks can add keys to filter on to enable debug logging for specific customers
and the data in Redis is automatically removed after a predetermined amount of
time.
In addition, you can use Arbiters to enable/disable sections of the
configuration during reconfiguration. See
xhttps://logging.apache.org/log4j/2.x/manual/configuration.html#arbiters So,
for example, you could create a custom Arbiter that enables a section of the
configuration based on the result of some REST call it makes or the value in a
database, or some variable set by a call to the application.
It is extremely common for users migrating from Log4j 1 to try to port their
logic as is to Log4j 2. This is almost always the wrong approach.
Ralph
> On Mar 21, 2023, at 11:05 PM, Viraj Jasani wrote:
>
> Thanks a lot, Ralph!
> Let me ask one more question. For any further questions, I would create a
> new mail thread.
>
> We also have a utility in hadoop that tries to add FileAppender at runtime,
> and also change the threshold of all appenders from root level. Is this
> doable with log4j2? I could not find Appender or AbstractAppender method to
> set filter/threshold to the given appender at runtime.
>
> IIUC, we can use something like
> "*setFilter(ThresholdFilter.**createFilter(Level.TRACE,
> null, null))*" to the builder class while creating the appender instance,
> but I am not sure if we can update the threshold for the existing appender.
>
> Here is the code reference (log4j1):
> https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/AdHocLogDumper.java#L71-L103
>
>
> On Tue, Mar 21, 2023 at 5:48 PM Viraj Jasani wrote:
>
>> I believe we can use "Configurator.reconfigure(uri)", however just wanted
>> to make sure if this is the only right way (stable API).
>>
>>
>> On Tue, Mar 21, 2023 at 4:57 PM Viraj Jasani wrote:
>>
>>> Just a follow up on this, do we have replacement for
>>> "LogManager.resetConfiguration" in log4j2?
>>>
>>>
>>> On Sat, Mar 18, 2023 at 11:27 AM Viraj Jasani wrote:
>>>
Ah looks like monitorinterval is supported as per the doc[1]:
Properties configuration files support the advertiser, monitorInterval,
name, packages, shutdownHook, shutdownTimeout, status, verbose, and dest
attributes.
1.
https://logging.apache.org/log4j/2.x/manual/configuration.html#ConfigurationSyntax
On Sat, Mar 18, 2023 at 11:14 AM Viraj Jasani
wrote:
> Thanks Ralph for the recommendations!
>
>> First, I strongly recommend you switch from properties to either XML,
> Yaml, or JSON.
>
> The reason why we would still like to stick to properties (at least for
> now during the migration) is that LOG4J2-3341 allows setting level and
> appender at once in the properties file. After migration, if there is any
> better way, we would consider migrating to xml or yaml for sure. Here is
> the old thread for the reference[1] after which both hbase and hadoop have
> decided to stick to properties for now.
>
>> Note that there are several variations of the initialize method.
>
> Sure I think we might be able to use, will explore:
>public static LoggerContext initialize(final String name, final
> ClassLoader loader, final URI configLocation) {
>return initialize(name, loader, configLocation, null);
>}
>
> Even with properties file, is it still possible to configure monitor
> interval?
>
>
> 1. https://lists.apache.org/thread/gvfb3jkg6t11cyds4jmpo7lrswmx28w3
>
> On Fri, Mar 17, 2023 at 8:59 PM Viraj Jasani
> wrote:
>
>> Hi,
>>
>> Could you please help with log4j2 replacement for PropertyConfigurator
>> APIs configureAndWatch and configure?
>>
>> For instance, this is the logic we have that we need to migrate to
>> log4j2:
>>
>>
>>
>> protected void initLog() throws ServerException {
>>verifyDir(logDir);
>>LogManager.resetConfiguration();
>>File log4jFile = new File(configDir, name + "-log4j.properties");
>>if (log4jFile.exists()) {
>> PropertyConfigurator.configureAndWatch(log4jFile.toString(), 10
>> * 1000); //every 10 secs
>> log = LoggerFactory.getLogger(Server.class);
>>} else {
>> Properties
