[
https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang reassigned HADOOP-14701:
------------------------------------
Assignee: Andrew Sherman
> Configuration can log misleading warnings about an attempt to override final
> parameter
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-14701
> URL: https://issues.apache.org/jira/browse/HADOOP-14701
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Andrew Sherman
> Assignee: Andrew Sherman
> Attachments: HADOOP-14701.1.patch, HADOOP-14701.2.patch,
> HADOOP-14701.3.patch, HADOOP-14701.4.patch
>
>
> The class Configuration (in hadoop-common-project) is used to hold
> configuration properties.
> The method loadResources() is used to load properties from a list of default
> resources into a parameter called “properties” (which happens to always be
> the field called “properties”).
> This is done by calling the method loadResource() which is used to load
> properties from a single resource into a parameter called “properties”
> (which happens to always be the field called “properties”).
> This is done by calling loadProperty() which is used to load a single
> property into a parameter called “properties” (which is sometimes, but not
> always, the field called “properties”).
> In loadProperty() there is some logic for dealing with properties that are
> marked as final.
> (1) a final property is not set into the “properties” parameter.
> (2) a warning is printed for a final property if the value of the property
> with the same name in the “properties” parameter is not equal to the new
> value, and the property name is in the finalParameters field.
> When loadResource() is reading properties from a stream it creates a
> temporary Properties object that it populates by calling loadProperty(). This
> is the case where loadProperty() has a parameter called “properties” which
> is not the same as the field called “properties”).
> When loadProperty() is called on a final property then because of (1) it is
> not set into the “properties” parameter. But sometimes the property name is
> already in the finalParameters field and now (2) the warning is triggered
> because the property is not present in the “properties” parameter (although
> it IS present in the “properties” field). The warning is misleading and
> confusing.
> h3. Possible Fix
> One way of looking at the problem is that loadProperty is using the
> finalParameters field in combination with the “properties” parameter. Change
> the code to only generate a warning for a final property if the value of the
> property with the same name in the “properties” field is not equal to the new
> value, and the property name is in the finalParameters field.
> I’ve made a patch containing this tiny change.
> Note that I have done very minimal testing of this fix
> h3. Example warning:
> 2017-07-25 12:14:24,639 WARN org.apache.hadoop.conf.Configuration:
> [HiveServer2-Background-Pool: Thread-77]:
> hdfs://asherman-cdh6-subset-parcel-1.gce.cloudera.com:8020/user/hive/.staging/job_1501008923167_0001/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
