This is an automated email from the ASF dual-hosted git repository.
hellostephen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new c6ee9cf25ea [fix](compile) auto-regenerate hadoop-deps/lib when
missing due to Ma… (#60964)
c6ee9cf25ea is described below
commit c6ee9cf25eaea7a50150f031e245412aa63960b4
Author: Dongyang Li <[email protected]>
AuthorDate: Tue Mar 3 14:25:34 2026 +0800
[fix](compile) auto-regenerate hadoop-deps/lib when missing due to Ma…
(#60964)
…ven cache
Problem:
- When Maven build cache is hit, the dependency:copy-dependencies plugin
execution is skipped
- This causes target/lib directory to be missing in hadoop-deps module
- Results in an incomplete BE output package without Hadoop dependencies
Solution:
- Add automatic regeneration logic in build.sh copy phase
- When lib directory is missing and BUILD_BE_JAVA_EXTENSIONS=1,
execute 'mvn dependency:copy-dependencies' to regenerate it
- Preserves Maven cache benefits while ensuring complete output
Impact:
- Fixes intermittent missing hadoop_hdfs jars in BE output
- No impact on normal build flow (only triggers when needed)
- Maintains build cache acceleration for other plugins
Related PR: #60819
---
build.sh | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/build.sh b/build.sh
index 122be348ad4..918c71dfcb0 100755
--- a/build.sh
+++ b/build.sh
@@ -970,6 +970,13 @@ EOF
mkdir "${BE_HADOOP_HDFS_DIR}"
HADOOP_DEPS_JAR_DIR="${DORIS_HOME}/fe/be-java-extensions/${HADOOP_DEPS_NAME}/target"
echo "HADOOP_DEPS_JAR_DIR: ${HADOOP_DEPS_JAR_DIR}"
+ if [[ "${BUILD_BE_JAVA_EXTENSIONS}" -eq 1 && ! -d
"${HADOOP_DEPS_JAR_DIR}/lib" ]]; then
+ echo "WARN: lib directory missing (likely due to Maven cache).
Regenerating..."
+ pushd "${DORIS_HOME}/fe/be-java-extensions/${HADOOP_DEPS_NAME}"
+ "${MVN_CMD}" dependency:copy-dependencies -DskipTests
-Dcheckstyle.skip=true
+ mv target/dependency target/lib
+ popd
+ fi
if [[ -f "${HADOOP_DEPS_JAR_DIR}/${HADOOP_DEPS_NAME}.jar" ]]; then
echo "Copy Be Extensions hadoop deps jar to
${BE_HADOOP_HDFS_DIR}"
cp "${HADOOP_DEPS_JAR_DIR}/${HADOOP_DEPS_NAME}.jar"
"${BE_HADOOP_HDFS_DIR}"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
