This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 0bdb7b674ab [fix](docker)(case) Fix kerberized-HDFS p2 suites: SASL
client property, Paimon fixture, iceberg root-cause reporting (#66433)
0bdb7b674ab is described below
commit 0bdb7b674abc71dc2b250a1c84f703211b31dddb
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Wed Aug 5 20:24:01 2026 +0800
[fix](docker)(case) Fix kerberized-HDFS p2 suites: SASL client property,
Paimon fixture, iceberg root-cause reporting (#66433)
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #65564, #66313, #66417
Problem Summary:
Master port of the remaining external-regression gaps diagnosed on the
branch-4.0 pipeline (#66417). #66313 already carried the keytab and
baseline
fixes to master; this PR carries the rest. All three defects were
verified to
exist on master by inspection — master shares the same code and the same
lightweight Kerberos environment (#65564, which #65939 backported to
4.0).
**1. Kerberos HDFS writes fail in the p2 suites (SASL client property
missing)**
#65564 created `kerberos/conf/hdfs-site.xml.tpl` with
`dfs.data.transfer.protection=authentication`. A client that does not
set the
same property sends a plain `writeBlock` op, `SaslDataTransferServer`
closes the
socket, the client excludes the only DataNode, and the write fails with
could only be written to 0 of the 1 minReplication nodes.
There are 1 datanode(s) running and 1 node(s) are excluded in this
operation.
(through iceberg this surfaces as `IllegalArgumentException:
Self-suppression
not permitted`). Metadata operations only reach the NameNode and the
HMS, so
`CREATE DATABASE` / `CREATE TABLE` still succeed and only the write
fails.
#65564 set the client property in the four `external_table_p0/kerberos`
suites
but not in the p2 suites that talk to the same kerberized HDFS. On the
branch-4.0 pipeline this turned `hive_on_hms_and_dlf` and
`iceberg_on_hms_and_filesystem_and_dlf` red on every run; master carries
the
identical omission.
Fix: set `dfs.data.transfer.protection` in every property block that
points at
the kerberized HDFS. A scan of `:8520`/`:8620` across the whole suite
tree finds
seven such blocks still missing it: two each in `hive_on_hms_and_dlf`,
`iceberg_on_hms_and_filesystem_and_dlf` and `test_paimon_hms_catalog`,
plus one
in `hdfs_all_test` (currently returns early because
`refactor_params_hdfs_kerberos_test` is unset — it would fail the moment
it is
enabled). The kerberos block of `test_information_schema_timezone` has
the same
gap but is disabled by an unrelated TODO, so it is left alone.
**2. `test_paimon_hms_catalog`: `Unknown database 'hdfs_db'`**
The lightweight Kerberos environment starts an empty metastore —
`schematool
-initSchema` and nothing else. The environment it replaced uploaded
`paimon_data` into HDFS and registered `hdfs_db` through
`create_paimon_hive_table.hql` on every start; `paimon_data/hdfs_db.db/`
is
still in the tree but the compose file never mounted it. This restores
the
fixture on `kerberos1` only (it owns metastore 9583, the only one the
suite
talks to), runs it before `DORIS_KERBEROS_READY` so nothing races the
suites,
and aborts the container on failure instead of handing out an
environment
silently missing `hdfs_db`.
Two deliberate differences from the branch-4.0 sibling commit:
- **`hdfs_db` only, no `ali_db`/OSS.** Master's pre-#65564 HQL
provisioned
exactly `hdfs_db`; the `oss_hms_kerberos` case that reads `ali_db`
through
the kerberized metastore exists only on branch-4.0, master's `ali_db`
cases
go through the Hive3 metastore, and the golden output here carries only
the
`hdfs_kerberos` / `hdfs_new_kerberos` blocks. This keeps the
jindo/aliyun
jars and OSS credentials out of the kerberos stack.
- **The fixture is gated on the container role (`HOST`).** Both kerberos
containers run the same entrypoint with the same rendered env switch,
but
only `kerberos1` carries the `sql/`/`paimon_data/`/`auxlib/` mounts —
without
the gate, `set -e` would abort `kerberos2` the moment `enablePaimonHms`
is
on. (The branch-4.0 sibling needs the same gate; a follow-up on #66417
will
add it.)
**3. `IcebergConnectorMetadata`: report the root cause of failed
metadata commits**
`TableMetadataParser.internalWrite` writes the metadata JSON inside
try-with-resources, and `DFSOutputStream` throws the same exception
instance
from `write` and from `close` once its pipeline has failed, so
`Throwable.addSuppressed(this)` replaces the failure with
`IllegalArgumentException: Self-suppression not permitted`. The FE log
keeps the
chain, but the message returned over the MySQL protocol is built from
`e.getMessage()` and carries only the wrapper.
This switches all nineteen metadata-commit paths — `createTable`, the
column/schema updates (including the nested-column and comment
variants), the
branch/tag operations, and the partition-field operations — to
`ExceptionUtils.getRootCauseMessage(e)`. The legacy `IcebergMetadataOps`
already reported the root cause for the partition-field trio; the
connector
rewrite lost that, so this also restores it. Read paths and the
drop/rename
operations cannot hit self-suppression and keep `e.getMessage()`.
**Intentionally not ported from #66417:** the file-cache de-flake.
Master's
`test_file_cache_features` (rewritten in #66313's lineage) already waits
for
`disk_resource_limit_mode` / `need_evict_cache_in_advance` to clear
before
returning, which closes the residual-state window at its source.
**Verification (local):** `fe-connector-iceberg` compiles and passes
checkstyle
(`mvn -pl fe-connector/fe-connector-iceberg package`); the four changed
suites
compile with the offline groovy compiler; `bash -n` passes on both shell
scripts; the compose/env/conf templates render with no leftover
placeholders,
the rendered XML parses, kerberos1 gets exactly the three new mounts
while
kerberos2 is unchanged; the settings resolution and the entrypoint
role-gate
were exercised for all switch/role combinations. The containers
themselves need
the CI environment and have not been run.
### Release note
None
### Check List (For Author)
- Test
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
---------
Signed-off-by: morningman <[email protected]>
---
.../kerberos/entrypoint-hive-master.sh | 26 ++++++++++++
.../docker-compose/kerberos/hadoop-hive.env.tpl | 4 ++
.../docker-compose/kerberos/kerberos.yaml.tpl | 6 +++
.../kerberos/sql/create_paimon_hive_table.hql | 22 ++++++++++
docker/thirdparties/run-thirdparties-docker.sh | 49 +++++++++++++++++++++-
.../iceberg/IcebergConnectorMetadata.java | 47 ++++++++++++---------
.../refactor_storage_param/hdfs_all_test.groovy | 1 +
.../paimon/test_paimon_hms_catalog.groovy | 2 +
.../hive_on_hms_and_dlf.groovy | 2 +
.../iceberg_on_hms_and_filesystem_and_dlf.groovy | 2 +
10 files changed, 141 insertions(+), 20 deletions(-)
diff --git
a/docker/thirdparties/docker-compose/kerberos/entrypoint-hive-master.sh
b/docker/thirdparties/docker-compose/kerberos/entrypoint-hive-master.sh
index 6735ae7ee40..885a415d996 100755
--- a/docker/thirdparties/docker-compose/kerberos/entrypoint-hive-master.sh
+++ b/docker/thirdparties/docker-compose/kerberos/entrypoint-hive-master.sh
@@ -136,9 +136,35 @@ kdestroy
report_stage "initialize-hive-metastore"
schematool -dbType derby -initSchema
+# The Paimon table declares no columns, so the metastore itself resolves its
+# schema through the storage handler. The handler class must therefore be on
the
+# metastore service's own classpath, not merely on the DDL client's.
+if [[ -d /opt/doris/auxlib ]]; then
+ export HIVE_AUX_JARS_PATH=/opt/doris/auxlib
+fi
start_service hive --service metastore -p "${HMS_PORT}"
wait_for_port "${HOST}" "${HMS_PORT}" "Hive Metastore"
+# Register the Paimon fixture consumed by test_paimon_hms_catalog. This runs
+# before the readiness marker below on purpose: run-thirdparties-docker.sh
+# releases the pipeline on DORIS_KERBEROS_READY, so anything published later
+# would race the suites. A failure here aborts the container (set -e) instead
of
+# handing out an environment that is silently missing hdfs_db.
+#
+# Both kerberos containers run this entrypoint with the same env switch, but
the
+# fixture and its mounts (sql/, paimon_data/, auxlib/) belong to kerberos1
only -
+# its metastore (9583) is the one the suite talks to. Gate on the container
role,
+# not on the mounts, so a broken mount on kerberos1 still fails loudly.
+if [[ "${enablePaimonHms:-false}" == "true" && "${HOST:-}" == "hadoop-master"
]]; then
+ report_stage "load-paimon-hms"
+ export KRB5CCNAME=FILE:/tmp/hive-admin.ccache
+ kinit -kt /data/keytabs/hive.keytab "${HIVE_PRINCIPAL}"
+ hdfs dfs -mkdir -p /user/hive/warehouse
+ hdfs dfs -put -f /opt/doris/paimon_data/* /user/hive/warehouse/
+ hive -f /opt/doris/sql/create_paimon_hive_table.hql
+ kdestroy
+fi
+
touch /tmp/SUCCESS
echo "Minimal Kerberos HDFS and Hive Metastore environment is ready"
echo "DORIS_KERBEROS_READY"
diff --git a/docker/thirdparties/docker-compose/kerberos/hadoop-hive.env.tpl
b/docker/thirdparties/docker-compose/kerberos/hadoop-hive.env.tpl
index 792d0c0491d..ff95e56f1ae 100644
--- a/docker/thirdparties/docker-compose/kerberos/hadoop-hive.env.tpl
+++ b/docker/thirdparties/docker-compose/kerberos/hadoop-hive.env.tpl
@@ -29,3 +29,7 @@ PRESTO_CLIENT_KEYTAB=${PRESTO_CLIENT_KEYTAB}
HADOOP_CONF_DIR=/opt/doris/conf
HIVE_CONF_DIR=/opt/doris/conf
KRB5_CONFIG=/etc/krb5.conf
+# Paimon-on-kerberized-HMS fixture switch. Read from hive-3x_settings.env by
+# start_kerberos() rather than duplicated into kerberos*_settings.env, so that
+# the pipeline keeps patching exactly one file and the two stacks cannot drift.
+enablePaimonHms=${enablePaimonHms}
diff --git a/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
b/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
index d21496516af..8e903ed6d7f 100644
--- a/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
+++ b/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
@@ -33,11 +33,17 @@ services:
<<: *kerberos-service
container_name: doris-${CONTAINER_UID}-kerberos1
hostname: hadoop-master
+ # auxlib/sql/paimon_data are mounted on kerberos1 only:
test_paimon_hms_catalog
+ # is the sole consumer and it talks to this metastore (9583). Leaving
kerberos2
+ # untouched keeps the second container as light as it was.
volumes:
- ./conf/kerberos1:/opt/doris/conf:ro
- ./conf/kerberos1/krb5.conf:/etc/krb5.conf:ro
- ./data/kerberos1:/data
- ./two-kerberos-hives:/keytabs
+ - ./auxlib:/opt/doris/auxlib:ro
+ - ./sql:/opt/doris/sql:ro
+ - ./paimon_data:/opt/doris/paimon_data:ro
env_file:
- ./hadoop-hive-1.env
diff --git
a/docker/thirdparties/docker-compose/kerberos/sql/create_paimon_hive_table.hql
b/docker/thirdparties/docker-compose/kerberos/sql/create_paimon_hive_table.hql
new file mode 100644
index 00000000000..d4150502b33
--- /dev/null
+++
b/docker/thirdparties/docker-compose/kerberos/sql/create_paimon_hive_table.hql
@@ -0,0 +1,22 @@
+-- Paimon table registered in the kerberized Hive Metastore, consumed by the
+-- hdfs_kerberos / hdfs_new_kerberos cases of
+-- external_table_p2/paimon/test_paimon_hms_catalog.groovy.
+--
+-- hdfs_db is backed by ../paimon_data, which the entrypoint uploads into HDFS.
+-- (branch-4.0 additionally registers ali_db over OSS here; master has no case
+-- that reads it through this metastore, so it is deliberately not
provisioned.)
+--
+-- The table declares no columns, so the metastore derives them through the
+-- Paimon storage handler (metastore.storage.schema.reader.impl is
+-- SerDeStorageSchemaReader) -- the Paimon jar has to be on the metastore's own
+-- classpath, not just on the classpath of whoever runs this script.
+
+CREATE DATABASE IF NOT EXISTS hdfs_db;
+
+USE hdfs_db;
+
+DROP TABLE IF EXISTS external_test_table;
+
+CREATE EXTERNAL TABLE external_test_table
+ STORED BY 'org.apache.paimon.hive.PaimonStorageHandler'
+LOCATION 'hdfs:///user/hive/warehouse/hdfs_db.db/external_test_table';
diff --git a/docker/thirdparties/run-thirdparties-docker.sh
b/docker/thirdparties/run-thirdparties-docker.sh
index 4ac5edb4a33..c4a9079e486 100755
--- a/docker/thirdparties/run-thirdparties-docker.sh
+++ b/docker/thirdparties/run-thirdparties-docker.sh
@@ -1595,6 +1595,48 @@ validate_kerberos_container() {
fi
}
+# Aux jar for the Paimon fixture in the kerberized metastore: the Paimon
storage
+# handler the metastore loads to derive the fixture table's schema. Same source
+# and same skip-if-present caching the Hive3 stack uses in
prepare-hive-data.sh.
+# (branch-4.0 also stages the jindo/aliyun jars for its ali_db-over-OSS case;
+# master has no such case in this metastore.)
+download_kerberos_paimon_jars() {
+ local auxlib_dir="$1"
+ local
url_prefix="https://${s3BucketName}.${s3Endpoint}/regression/docker/hive3";
+ local jars=(
+ paimon-hive-connector-3.1-1.3-SNAPSHOT.jar
+ )
+ local jar
+
+ for jar in "${jars[@]}"; do
+ if [[ -f "${auxlib_dir}/${jar}" ]]; then
+ echo "Reuse cached kerberos aux jar ${jar}"
+ continue
+ fi
+ echo "Download kerberos aux jar ${jar}"
+ curl -sSfL -o "${auxlib_dir}/${jar}" "${url_prefix}/${jar}"
+ done
+}
+
+# enablePaimonHms lives in hive-3x_settings.env, which the pipeline already
+# patches with the real value. Read it from there instead of duplicating the
+# switch into kerberos*_settings.env, where the two copies would drift and one
+# stack would silently lose the fixture. Sourced in a subshell so that hive3's
+# FS_PORT / HMS_PORT cannot leak into the kerberos settings sourced a few lines
+# below.
+resolve_kerberos_paimon_env() {
+ local hive3_settings="${ROOT}/docker-compose/hive/hive-3x_settings.env"
+
+ enablePaimonHms="false"
+ if [[ -f "${hive3_settings}" ]]; then
+ read -r enablePaimonHms < <(
+ . "${hive3_settings}" >/dev/null 2>&1
+ printf '%s\n' "${enablePaimonHms:-false}"
+ )
+ fi
+ export enablePaimonHms
+}
+
start_kerberos() {
echo "RUN_KERBEROS"
local KERBEROS_DIR="${ROOT}/docker-compose/kerberos"
@@ -1609,8 +1651,13 @@ start_kerberos() {
export CONTAINER_UID=${CONTAINER_UID}
envsubst <"${KERBEROS_DIR}/kerberos.yaml.tpl"
>"${KERBEROS_DIR}/kerberos.yaml"
+ # auxlib must exist even when the fixture is off: kerberos1 mounts it
read-only.
mkdir -p "${KERBEROS_DIR}/conf/kerberos1" "${KERBEROS_DIR}/conf/kerberos2"
\
- "${KERBEROS_DIR}/two-kerberos-hives"
+ "${KERBEROS_DIR}/two-kerberos-hives" "${KERBEROS_DIR}/auxlib"
+ resolve_kerberos_paimon_env
+ if [[ "${enablePaimonHms}" == "true" && "${STOP}" -ne 1 ]]; then
+ download_kerberos_paimon_jars "${KERBEROS_DIR}/auxlib"
+ fi
for i in {1..2}; do
. "${KERBEROS_DIR}/kerberos${i}_settings.env"
envsubst <"${KERBEROS_DIR}/hadoop-hive.env.tpl"
>"${KERBEROS_DIR}/hadoop-hive-${i}.env"
diff --git
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadata.java
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadata.java
index 3b32742842c..dd1fc6ab38e 100644
---
a/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadata.java
+++
b/fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorMetadata.java
@@ -52,6 +52,7 @@ import org.apache.doris.thrift.TIcebergTable;
import org.apache.doris.thrift.TTableDescriptor;
import org.apache.doris.thrift.TTableType;
+import org.apache.commons.lang3.exception.ExceptionUtils;
import org.apache.iceberg.BaseTable;
import org.apache.iceberg.MetadataTableType;
import org.apache.iceberg.MetadataTableUtils;
@@ -965,7 +966,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to create Iceberg table "
- + request.getDbName() + "." + request.getTableName() + ":
" + e.getMessage(), e);
+ + request.getDbName() + "." + request.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1158,7 +1160,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to add column " +
column.getName() + " to Iceberg table "
- + iceHandle.getDbName() + "." + iceHandle.getTableName() +
": " + e.getMessage(), e);
+ + iceHandle.getDbName() + "." + iceHandle.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1177,7 +1180,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to add columns to
Iceberg table "
- + iceHandle.getDbName() + "." + iceHandle.getTableName() +
": " + e.getMessage(), e);
+ + iceHandle.getDbName() + "." + iceHandle.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1192,7 +1196,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to drop column " +
columnName + " from Iceberg table "
- + iceHandle.getDbName() + "." + iceHandle.getTableName() +
": " + e.getMessage(), e);
+ + iceHandle.getDbName() + "." + iceHandle.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1209,7 +1214,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to rename column " +
oldName + " to " + newName
+ " in Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1255,7 +1260,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to modify column " +
column.getName()
+ " in Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1299,7 +1304,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to reorder columns in
Iceberg table "
- + iceHandle.getDbName() + "." + iceHandle.getTableName() +
": " + e.getMessage(), e);
+ + iceHandle.getDbName() + "." + iceHandle.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1345,7 +1351,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to add nested column " +
path.getFullPath()
+ " to Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1367,7 +1373,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to drop nested column "
+ path.getFullPath()
+ " from Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1391,7 +1397,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to rename nested column
" + path.getFullPath()
+ " to " + newName + " in Iceberg table " +
iceHandle.getDbName() + "."
- + iceHandle.getTableName() + ": " + e.getMessage(), e);
+ + iceHandle.getTableName() + ": " +
ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1440,7 +1446,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to modify nested column
" + path.getFullPath()
+ " in Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1462,7 +1468,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to modify comment for
column " + path.getFullPath()
+ " in Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1485,7 +1491,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to create or replace
branch " + branch.getName()
+ " on Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1501,7 +1507,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to create or replace tag
" + tag.getName()
+ " on Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1517,7 +1523,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to drop branch " +
branch.getName()
+ " from Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1533,7 +1539,7 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
} catch (Exception e) {
throw new DorisConnectorException("Failed to drop tag " +
tag.getName()
+ " from Iceberg table " + iceHandle.getDbName() + "." +
iceHandle.getTableName()
- + ": " + e.getMessage(), e);
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1556,7 +1562,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to add partition field
to Iceberg table "
- + iceHandle.getDbName() + "." + iceHandle.getTableName() +
": " + e.getMessage(), e);
+ + iceHandle.getDbName() + "." + iceHandle.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1572,7 +1579,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to drop partition field
from Iceberg table "
- + iceHandle.getDbName() + "." + iceHandle.getTableName() +
": " + e.getMessage(), e);
+ + iceHandle.getDbName() + "." + iceHandle.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
@@ -1588,7 +1596,8 @@ public class IcebergConnectorMetadata implements
ConnectorMetadata {
});
} catch (Exception e) {
throw new DorisConnectorException("Failed to replace partition
field in Iceberg table "
- + iceHandle.getDbName() + "." + iceHandle.getTableName() +
": " + e.getMessage(), e);
+ + iceHandle.getDbName() + "." + iceHandle.getTableName()
+ + ": " + ExceptionUtils.getRootCauseMessage(e), e);
}
}
diff --git
a/regression-test/suites/external_table_p0/refactor_storage_param/hdfs_all_test.groovy
b/regression-test/suites/external_table_p0/refactor_storage_param/hdfs_all_test.groovy
index 52639c8a822..d326ee8dc7b 100644
---
a/regression-test/suites/external_table_p0/refactor_storage_param/hdfs_all_test.groovy
+++
b/regression-test/suites/external_table_p0/refactor_storage_param/hdfs_all_test.groovy
@@ -77,6 +77,7 @@ suite("refactor_params_hdfs_all_test", "p0,external") {
def hdfsNonXmlParams = "\"fs.defaultFS\" =
\"hdfs://${externalEnvIp}:8520\",\n" +
"\"dfs.namenode.kerberos.principal\" =
\"hdfs/[email protected]\",\n" +
"\"dfs.client.use.datanode.hostname\" = \"true\",\n" +
+ "\"dfs.data.transfer.protection\" = \"authentication\",\n" +
"\"hadoop.security.token.service.use_ip\" = \"false\",\n" +
"\"hadoop.kerberos.min.seconds.before.relogin\" = \"5\",\n" +
"\"hadoop.security.authentication\" = \"kerberos\",\n" +
diff --git
a/regression-test/suites/external_table_p2/paimon/test_paimon_hms_catalog.groovy
b/regression-test/suites/external_table_p2/paimon/test_paimon_hms_catalog.groovy
index fd2337bd8d8..722b50531f5 100644
---
a/regression-test/suites/external_table_p2/paimon/test_paimon_hms_catalog.groovy
+++
b/regression-test/suites/external_table_p2/paimon/test_paimon_hms_catalog.groovy
@@ -124,6 +124,7 @@ suite("test_paimon_hms_catalog", "p2,external") {
"fs.defaultFS" = "hdfs://${extHiveHmsHost}:8520",
"dfs.namenode.kerberos.principal" =
"hdfs/[email protected]",
"dfs.client.use.datanode.hostname" = "true",
+ "dfs.data.transfer.protection" = "authentication",
"hadoop.security.token.service.use_ip" = "false",
"hadoop.security.authentication" = "kerberos",
"hadoop.kerberos.principal"="hive/[email protected]",
@@ -134,6 +135,7 @@ suite("test_paimon_hms_catalog", "p2,external") {
"fs.defaultFS" = "hdfs://${extHiveHmsHost}:8520",
"dfs.namenode.kerberos.principal" =
"hdfs/[email protected]",
"dfs.client.use.datanode.hostname" = "true",
+ "dfs.data.transfer.protection" = "authentication",
"hadoop.security.token.service.use_ip" = "false",
"hdfs.authentication.type" = "kerberos",
"hdfs.authentication.kerberos.principal"="hive/[email protected]",
diff --git
a/regression-test/suites/external_table_p2/refactor_catalog_param/hive_on_hms_and_dlf.groovy
b/regression-test/suites/external_table_p2/refactor_catalog_param/hive_on_hms_and_dlf.groovy
index 87d276b4d61..9fddd9ba979 100644
---
a/regression-test/suites/external_table_p2/refactor_catalog_param/hive_on_hms_and_dlf.groovy
+++
b/regression-test/suites/external_table_p2/refactor_catalog_param/hive_on_hms_and_dlf.groovy
@@ -483,6 +483,7 @@ suite("hive_on_hms_and_dlf", "p2,external") {
"fs.defaultFS" = "hdfs://${externalEnvIp}:8520",
"dfs.namenode.kerberos.principal" =
"hdfs/[email protected]",
"dfs.client.use.datanode.hostname" = "true",
+ "dfs.data.transfer.protection" = "authentication",
"hadoop.security.token.service.use_ip" = "false",
"hadoop.security.authentication" = "kerberos",
@@ -493,6 +494,7 @@ suite("hive_on_hms_and_dlf", "p2,external") {
"fs.defaultFS" = "hdfs://${externalEnvIp}:8520",
"dfs.namenode.kerberos.principal" =
"hdfs/[email protected]",
"dfs.client.use.datanode.hostname" = "true",
+ "dfs.data.transfer.protection" = "authentication",
"hadoop.security.token.service.use_ip" = "false",
"hdfs.authentication.type" = "kerberos",
"hdfs.authentication.kerberos.principal"="hive/[email protected]",
diff --git
a/regression-test/suites/external_table_p2/refactor_catalog_param/iceberg_on_hms_and_filesystem_and_dlf.groovy
b/regression-test/suites/external_table_p2/refactor_catalog_param/iceberg_on_hms_and_filesystem_and_dlf.groovy
index 6358e6e1159..b900ece716d 100644
---
a/regression-test/suites/external_table_p2/refactor_catalog_param/iceberg_on_hms_and_filesystem_and_dlf.groovy
+++
b/regression-test/suites/external_table_p2/refactor_catalog_param/iceberg_on_hms_and_filesystem_and_dlf.groovy
@@ -479,6 +479,7 @@ suite("iceberg_on_hms_and_filesystem_and_dlf",
"p2,external") {
"fs.defaultFS" = "hdfs://${externalEnvIp}:8520",
"dfs.namenode.kerberos.principal" =
"hdfs/[email protected]",
"dfs.client.use.datanode.hostname" = "true",
+ "dfs.data.transfer.protection" = "authentication",
"hadoop.security.token.service.use_ip" = "false",
"hadoop.security.authentication" = "kerberos",
"hadoop.kerberos.principal"="hive/[email protected]",
@@ -489,6 +490,7 @@ suite("iceberg_on_hms_and_filesystem_and_dlf",
"p2,external") {
"fs.defaultFS" = "hdfs://${externalEnvIp}:8520",
"dfs.namenode.kerberos.principal" =
"hdfs/[email protected]",
"dfs.client.use.datanode.hostname" = "true",
+ "dfs.data.transfer.protection" = "authentication",
"hadoop.security.token.service.use_ip" = "false",
"hdfs.authentication.type" = "kerberos",
"hdfs.authentication.kerberos.principal"="hive/[email protected]",
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
