[
https://issues.apache.org/jira/browse/HADOOP-15007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347587#comment-16347587
]
Ajay Kumar commented on HADOOP-15007:
-------------------------------------
[[email protected]], Since excessive logging is the issue, will it be ok if we
log a single line at debug level.(something like " Invalid tag 'secret' found
for property:test.fs.s3a.name Source") without stacktrace? This debug line is
i think will be applicable even if change tags to String instead if enum.
{quote}Is this expected to be a stable change? Or at least: when do we expect
it to be stable?{quote}
We have tested it with older versions (xml config with no tags as well.). Ready
to make any changes required to make it more stable.
{quote}What is going to happen when an older version of Hadoop encounters an
XML file with a new field in it?{quote}
Tags will be left unprocessed.
{quote}we need some tests which actually set out to break the mechanism.
Invalid tags etc.{quote}
Will add a test case for this.
{quote}Javadocs and end-user docs to cover what can and cannot be done with
tags, how to use. Configuration's own javadocs are the defacto documentation of
the XML format: they need to be updated with the change.{quote}
Already added javadocs for changes in this functionality. Will add/update
javadocs if missed.
{quote}what's going to to happen when existing code which
serializes/deserializes configs using Hadoop writables encounters configs with
tags? Can an old hadoop-common lib deserialize, say core-default.xml with tags
added? I don't see any tests for that, and I'm assuming "no" unless it can be
demonstrated.{quote}
Will add test for this.
{quote}Can I add tags to a property retrieved with getPassword()?{quote}
Only if If getPassword falls back to config. Current change is not applocable
to {{CredentialProviderFactory}}. So
{{HADOOP_SECURITY_CREDENTIAL_PROVIDER_PATH}} can be tagged but not the
properties retrieved from it.
{quote}If I overrride a -default property in a file, does it inherit the tags
from the parent?{quote}
Yes
{quote}If I add tags to an overidden property, do the tags override or replace
existing ones?{quote}
New tag will not replace the old one but will be an addition to it. It will
result in same property being returned for two different tags by
{{getAllPropertiesByTag}}. Ex SECURITY, CLIENT
{quote}Configuration.readTagFromConfig. If there's an invalid tag in a -default
file, is it going to fill the log files with info messages on every load of
every single configuration file? If so, it's too noisy. One warning per
(file/property) the way we do for deprecated tags.{quote}
Will fix this.
{quote}Configuration.getPropertyTag returns a PropertyTag enum; which is marked
as Private/Evolving. But configuration is Public/Stable. Either we need to mark
PropertyTag as Public/ {Evolving/Unstable}
or getPropertyTag is marked as Private.{quote}
Open to both suggestions on this one.
{quote}Could there have been a way to allow PropertyTag enums to be registered
dynamically/via classname strings, so that they can be kept in the specific
modules. We've now tainted hadoop-common with details about yarn, hdfs,
ozone.{quote}
I see your point but how we will register a class which is not in classpath of
common? Another option is to create a common class for property tags and use it
for everything. i.e hdfs,yarn etc.
> Stabilize and document Configuration <tag> element
> --------------------------------------------------
>
> Key: HADOOP-15007
> URL: https://issues.apache.org/jira/browse/HADOOP-15007
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Affects Versions: 3.1.0
> Reporter: Steve Loughran
> Assignee: Anu Engineer
> Priority: Blocker
>
> HDFS-12350 (moved to HADOOP-15005). Adds the ability to tag properties with a
> <tag> value.
> We need to make sure that this feature is backwards compatible & usable in
> production. That's docs, testing, marshalling etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
