[
https://issues.apache.org/jira/browse/HADOOP-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218362#comment-14218362
]
Andrew Wang commented on HADOOP-11319:
--------------------------------------
Hey Tim,
The compat issue I'm thinking of is if an application is depending on the exact
Guava 11 being exposed on the Hadoop classpath. If we change the Guava version,
this application will break when it tries to run against the new version of
Hadoop. Providing our own versions of removed functions isn't bulletproof since
an app could be depending on the behavior from our exact version of Guava. If a
function's behavior is different in different versions of Guava, we can't
expose all of them at once. I also don't think the Hadoop project wants to be
in the business of maintaining our own "compatible" version of Guava.
Basically we're hamstrung since we leak our dependencies. Old apps can depend
on our ancient deps, so we have to keep them for compat even though it's quite
painful. For Hadoop 3, one option is simply shading all our deps, but that's
still under discussion.
For Hadoop 2, there are some possible mitigations. YARN has some classloader
and classpath mangling features which can help, but that doesn't help if you're
running outside of a YARN container. You can also try just putting your newer
deps at the front of the classpath and hoping for the best, but that's
unsupported/untested, and is probably something you've already wrestled with.
> Update Guava to 18.0
> --------------------
>
> Key: HADOOP-11319
> URL: https://issues.apache.org/jira/browse/HADOOP-11319
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build
> Reporter: Tim Robertson
> Priority: Critical
>
> In the hadoop libraries you'll find both 11.0.2 (test scope IIRC) and 14.0.1
> which are both very outdated. 14.0.1 removes things used in 11.0.2 and 15.0
> has removed things in use by Hadoop code in 14.0.1.
> In our experience through CDH3,4 and 5 Guava (along with Jackson and SLF4J
> 1.7.5) have been the biggest cause for CP issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
