[
https://issues.apache.org/jira/browse/HADOOP-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14321007#comment-14321007
]
Chris Nauroth commented on HADOOP-8642:
---------------------------------------
Akira, sorry to go back and forth, but I'm now having second thoughts about
this change after a deeper review.
Digging into revision history, I can see that this functionality was first
introduced in HADOOP-1570. The git commit hash is
f9f6143876a1ff6e8eb1266917f4992f642aaea7. The property was named
{{hadoop.native.lib}} then. This was a point in the project's history when the
only native code in Hadoop was native compression codecs. The current codebase
of course has a lot more native code that's important for a lot of different
things beyond the compression codecs. I'm concerned about possible backwards
incompatibility. Existing deployments might be using this property to disable
the native compression codecs, but are still dependent on the rest of the
native code for other functionality. If we were to push this change to those
existing clusters, then they'd suddenly lose the rest of that important native
code.
Another concern is the instantiation of yet another {{Configuration}} and all
of the parsing that entails. I don't think there would be any way around that,
because this is all driven from static initialization.
All things considered, I have to vote -1 on changing the behavior of this
property, at least within the 2.x line where we need to maintain backwards
compatibility. I would be +1 for a patch that updates the description in
core-default.xml to clarify that it really only controls the native bz2 and
zlib compression codecs.
> io.native.lib.available only controls zlib
> ------------------------------------------
>
> Key: HADOOP-8642
> URL: https://issues.apache.org/jira/browse/HADOOP-8642
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Akira AJISAKA
> Attachments: HADOOP-8642.2.patch, HADOOP-8642.3.patch,
> HADOOP-8642.4.patch, HADOOP-8642.patch
>
>
> Per core-default.xml {{io.native.lib.available}} indicates "Should native
> hadoop libraries, if present, be used" however it looks like it only affects
> zlib. Since we always load the native library this means we may use native
> libraries even if io.native.lib.available is set to false.
> Let's make the flag to work as advertised - rather than always load the
> native hadoop library we only attempt to load the library (and report that
> native is available) if this flag is set. Since io.native.lib.available
> defaults to true the default behavior should remain unchanged (except that
> now we wont actually try to load the library if this flag is disabled).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
