[
https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294482#comment-14294482
]
Colin Patrick McCabe commented on HADOOP-11223:
-----------------------------------------------
bq. Gopal wrote: The primary blockage I had trying to implement this is the
expected behaviour of addDefaultResource(). The overlays set programmatically
is somewhat easier to disable cleanly.
Yeah. {{addDefaultResource}} is certainly problematic. We could probably just
have a static copy of the configuration for each component (hadoop common,
hdfs, yarn, etc.), to get around the fact that different XML files will be
parsed for each one.
bq. Varun wrote: From the preceding discussion it seemed immutability was not
to be considered. And programmers using the class will take care of it.
I think if we've learned anything from Configuration.java, it's that the
programmers will *not* take care of it :)
bq. Can't we keep a simple flag in Configuration class to make it read only.
DefaultConfiguration can probably extend Configuration. We can make this flag
to be set only from DefaultConfiguration(in constructor) and disallow any
operations which involve setting keys or loading resources, if this flag is set.
I would rather not add a lot of complexity without benchmarks showing that it's
needed. Copying the configuration object is simple and we know it will work.
Plus, the real cost is reading the config files from the disk, not creating
more Java objects on the heap.
> Offer a read-only conf alternative to new Configuration()
> ---------------------------------------------------------
>
> Key: HADOOP-11223
> URL: https://issues.apache.org/jira/browse/HADOOP-11223
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Gopal V
> Assignee: Varun Saxena
> Labels: Performance
> Attachments: HADOOP-11223.001.patch
>
>
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily
> XML parsing at a point where the JIT won't be triggered & interpreter mode is
> essentially forced on the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}}
> alternative which disallows any modifications.
> At the very least, such a method would need to be called from
> # org.apache.hadoop.io.nativeio.NativeIO::<clinit>()
> # org.apache.hadoop.security.SecurityUtil::<clinit>()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::<clinit>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
