[
https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114676#comment-16114676
]
Allen Wittenauer commented on HADOOP-13952:
-------------------------------------------
bq. anyone not using HDFS for storage shouldn't be pulling in other
HDFS-specific things into their classpath.
>From a logistical stand point, this is correct. However, as you observe:
bq. Although I'd expect they actually do that in practice now?
In practice?
* 0.x-2.x always forces subproject directories into the classpath. Don't want
them? Too bad.
* 3.x at least looks to see if these directories exist and allows function
redefinition to cut them out.
* I don't know of a single distribution that *doesn't* include HDFS or YARN
(yes, including MapR). In fact, I'm not even sure it's possible to run Hadoop
without HDFS given things like hdfs getconf have no equivalent in common.
It's pretty much a guarantee these jars are already in the classpath. It's
worth pointing out that if they weren't, these tools likely wouldn't be working
now. :)
IMO, if someone is going to go through the effort of ripping out HDFS, then
changing the hadoop-tools dependency profile is the least of their concerns.
That said... we should probably at least add comments to the profile what the
implied dependencies are though to make that easier.
> tools dependency hooks are throwing errors
> ------------------------------------------
>
> Key: HADOOP-13952
> URL: https://issues.apache.org/jira/browse/HADOOP-13952
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 3.0.0-alpha2
> Reporter: Allen Wittenauer
> Assignee: Sean Mackrory
> Priority: Critical
> Attachments: HADOOP-13952.preview.patch
>
>
> During build, we are throwing these errors:
> {code}
> ERROR: hadoop-aliyun has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-aliyun has missing dependencies: json-lib-jdk15.jar
> ERROR: hadoop-archive-logs has missing dependencies:
> jasper-compiler-5.5.23.jar
> ERROR: hadoop-archives has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-aws has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-azure has missing dependencies:
> jetty-util-ajax-9.3.11.v20160721.jar
> ERROR: hadoop-azure-datalake has missing dependencies: okhttp-2.4.0.jar
> ERROR: hadoop-azure-datalake has missing dependencies: okio-1.4.0.jar
> ERROR: hadoop-extras has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-gridmix has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-kafka has missing dependencies: lz4-1.2.0.jar
> ERROR: hadoop-kafka has missing dependencies: kafka-clients-0.8.2.1.jar
> ERROR: hadoop-openstack has missing dependencies: commons-httpclient-3.1.jar
> ERROR: hadoop-rumen has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-sls has missing dependencies: jasper-compiler-5.5.23.jar
> ERROR: hadoop-sls has missing dependencies: metrics-core-3.0.1.jar
> ERROR: hadoop-streaming has missing dependencies: jasper-compiler-5.5.23.jar
> {code}
> Likely a variety of reasons for the failures. Kafka is HADOOP-12556, but
> others need to be investigated. Probably just need to look at more than just
> common/lib in dist-tools-hooks-maker now that shading has gone in.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
