This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 1e20b8a87df [fix](insert-overwrite) skip deleted partitions before
replace to avoid DdlException (#60914)
1e20b8a87df is described below
commit 1e20b8a87dfaaf025223f6c5bfe042bffe560354
Author: morrySnow <[email protected]>
AuthorDate: Mon Mar 2 11:15:44 2026 +0800
[fix](insert-overwrite) skip deleted partitions before replace to avoid
DdlException (#60914)
When performing insert overwrite on a whole table, partition names are
snapshotted at planning time without holding the table lock. A
concurrent DROP PARTITION between the snapshot and the replace step
causes Env.replaceTempPartition() to throw DdlException because the
original partition no longer exists.
Fix: after acquiring the table write lock in
InsertOverwriteUtil.replacePartition(), filter out any original
partitions that have been deleted before constructing
ReplacePartitionOp. The temp partition list is kept intact so all
written data remains visible.
---
.../apache/doris/insertoverwrite/InsertOverwriteUtil.java | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/insertoverwrite/InsertOverwriteUtil.java
b/fe/fe-core/src/main/java/org/apache/doris/insertoverwrite/InsertOverwriteUtil.java
index ce2e063171f..8d831dc98d5 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/insertoverwrite/InsertOverwriteUtil.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/insertoverwrite/InsertOverwriteUtil.java
@@ -89,10 +89,21 @@ public class InsertOverwriteUtil {
if (!olapTable.writeLockIfExist()) {
return;
}
+ // Filter out partitions that were deleted between the
snapshot time and now.
+ // The write lock is held here, so this check and the
subsequent replace are atomic.
+ // The temp partition list is kept as-is so all written data
remains visible.
+ List<String> validPartitionNames = new ArrayList<>();
+ for (int i = 0; i < partitionNames.size(); i++) {
+ if (((OlapTable)
olapTable).checkPartitionNameExist(partitionNames.get(i), false)) {
+ validPartitionNames.add(partitionNames.get(i));
+ } else {
+ LOG.warn("partition [{}] has been deleted before
replace, skipping", partitionNames.get(i));
+ }
+ }
Map<String, String> properties = Maps.newHashMap();
properties.put(PropertyAnalyzer.PROPERTIES_USE_TEMP_PARTITION_NAME, "false");
ReplacePartitionOp replacePartitionOp = new ReplacePartitionOp(
- new PartitionNamesInfo(false, partitionNames),
+ new PartitionNamesInfo(false, validPartitionNames),
new PartitionNamesInfo(true, tempPartitionNames),
isForce, properties);
if (replacePartitionOp.getTempPartitionNames().isEmpty()) {
return;
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
