This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7d83f416a1a [fix](cloud)Fix `create dynamic table` race with `insert
overwrite` (#59489)
7d83f416a1a is described below
commit 7d83f416a1a0e42ae224e0ca091d7abdf7f25cb1
Author: deardeng <[email protected]>
AuthorDate: Mon Jan 5 13:30:48 2026 +0800
[fix](cloud)Fix `create dynamic table` race with `insert overwrite` (#59489)
### What problem does this PR solve?
1. Thread 1 executed a DROP TABLE operation, followed by a CREATE TABLE
operation, initializing a dynamic partition (Partition A) in memory.
2. Due to an optimization in the cloud, batch partition edit logs were
attempted; however, the edit log for Partition A was not yet
synchronized to the followers.
3. Thread 2 began an INSERT INTO ... OVERWRITE PARTITION (*) operation,
identifying Partition A in memory and writing a replace partition edit
log.
4. In follower, the replay thread attempted to replay the replace
partition edit log but found a dependency on Partition A, which was
absent from memory, resulting in an exception and subsequent
termination.
Fix
```
2025-12-30 16:10:42,774 ERROR (replayer|123) [EditLog.loadJournal():1445]
replay Operation Type 210, log id: 1910
java.lang.NullPointerException
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:906)
at
org.apache.doris.catalog.OlapTable.checkPartition(OlapTable.java:2835)
at
org.apache.doris.catalog.OlapTable.replaceTempPartitions(OlapTable.java:2799)
at
org.apache.doris.catalog.Env.replayReplaceTempPartition(Env.java:6750)
at org.apache.doris.persist.EditLog.loadJournal(EditLog.java:986)
at org.apache.doris.catalog.Env.replayJournal(Env.java:3100)
at org.apache.doris.catalog.Env$4.runOneCycle(Env.java:2862)
at org.apache.doris.common.util.Daemon.run(Daemon.java:119)
2025-12-30 16:10:42,775 INFO (Thread-0|32) [DorisFE.lambda$start$0():159]
Received shutdown signal, starting graceful shutdown...
2025-12-30 16:10:42,776 INFO (Thread-0|32) [DorisFE.gracefulShutdown():639]
graceful shutdown finished
```
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---
.../main/java/org/apache/doris/alter/Alter.java | 2 +-
.../main/java/org/apache/doris/catalog/Env.java | 13 +--
.../doris/clone/DynamicPartitionScheduler.java | 74 +++++++++++------
.../apache/doris/datasource/InternalCatalog.java | 29 +++----
.../org/apache/doris/mtmv/MTMVPartitionUtil.java | 2 +-
.../java/org/apache/doris/persist/EditLog.java | 22 +++++-
.../apache/doris/service/FrontendServiceImpl.java | 2 +-
.../different_serialize.groovy} | 2 +-
...eate_partition_and_insert_overwrite_race.groovy | 92 ++++++++++++++++++++++
9 files changed, 189 insertions(+), 49 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
index ab1bf1be369..be11cbe7340 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
@@ -678,7 +678,7 @@ public class Alter {
DynamicPartitionUtil.checkAlterAllowed(
(OlapTable) db.getTableOrMetaException(tableName,
TableType.OLAP));
}
- Env.getCurrentEnv().addPartition(db, tableName,
(AddPartitionOp) alterOp, false, 0, true);
+ Env.getCurrentEnv().addPartition(db, tableName,
(AddPartitionOp) alterOp, false, 0, true, null);
} else if (alterOp instanceof AddPartitionLikeOp) {
if (!((AddPartitionLikeOp) alterOp).getTempPartition()) {
DynamicPartitionUtil.checkAlterAllowed(
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index 95f66f14a4d..6dda1e152b9 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -3480,14 +3480,17 @@ public class Env {
* @param isCreateTable this call is for creating table
* @param generatedPartitionId the preset partition id for the partition
to add
* @param writeEditLog whether to write an edit log for this addition
- * @return PartitionPersistInfo to be written to editlog. It may be null
if no partitions added.
+ * @batchPartitions output parameter, used to batch write edit log outside
this function, can be null.
+ * first is editlog PartitionPersistInfo, second is the added Partition
* @throws DdlException
*/
- public PartitionPersistInfo addPartition(Database db, String tableName,
AddPartitionOp addPartitionOp,
+ public void addPartition(Database db, String tableName, AddPartitionOp
addPartitionOp,
boolean isCreateTable, long
generatedPartitionId,
- boolean writeEditLog) throws
DdlException {
- return getInternalCatalog().addPartition(db, tableName, addPartitionOp,
- isCreateTable, generatedPartitionId, writeEditLog);
+ boolean writeEditLog,
+ List<Pair<PartitionPersistInfo,
Partition>> batchPartitions)
+ throws DdlException {
+ getInternalCatalog().addPartition(db, tableName, addPartitionOp,
+ isCreateTable, generatedPartitionId, writeEditLog,
batchPartitions);
}
public void addMultiPartitions(Database db, String tableName,
AlterMultiPartitionOp multiPartitionOp)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
index f77ec517667..495e8f9cdec 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
@@ -774,7 +774,7 @@ public class DynamicPartitionScheduler extends MasterDaemon
{
cloudBatchBeforeCreatePartitions(executeFirstTime,
addPartitionOps, olapTable, indexIds,
db, tableName, generatedPartitionIds);
- List<PartitionPersistInfo> partsInfo = new ArrayList<>();
+ List<Pair<PartitionPersistInfo, Partition>> batchPartsInfo =
new ArrayList<>();
for (int i = 0; i < addPartitionOps.size(); i++) {
try {
boolean needWriteEditLog = true;
@@ -783,15 +783,10 @@ public class DynamicPartitionScheduler extends
MasterDaemon {
if (Config.isCloudMode()) {
needWriteEditLog = !executeFirstTime;
}
- PartitionPersistInfo info =
- Env.getCurrentEnv().addPartition(db,
tableName, addPartitionOps.get(i),
+ Env.getCurrentEnv().addPartition(db, tableName,
addPartitionOps.get(i),
executeFirstTime,
executeFirstTime && Config.isCloudMode() ?
generatedPartitionIds.get(i) : 0,
- needWriteEditLog);
- if (info == null) {
- throw new Exception("null persisted partition
returned");
- }
- partsInfo.add(info);
+ needWriteEditLog, batchPartsInfo);
clearCreatePartitionFailedMsg(olapTable.getId());
} catch (Exception e) {
recordCreatePartitionFailedMsg(db.getFullName(),
tableName, e.getMessage(), olapTable.getId());
@@ -802,7 +797,7 @@ public class DynamicPartitionScheduler extends MasterDaemon
{
}
}
}
- cloudBatchAfterCreatePartitions(executeFirstTime, partsInfo,
+ cloudBatchAfterCreatePartitions(executeFirstTime,
batchPartsInfo,
addPartitionOps, db, olapTable, indexIds, tableName);
// ATTN: Breaking up dynamic partition table scheduling,
consuming peak CPU consumption
@@ -822,15 +817,16 @@ public class DynamicPartitionScheduler extends
MasterDaemon {
}
}
- private void cloudBatchAfterCreatePartitions(boolean executeFirstTime,
List<PartitionPersistInfo> partsInfo,
-
ArrayList<AddPartitionOp> addPartitionOps, Database db,
- OlapTable olapTable,
List<Long> indexIds,
- String tableName)
throws DdlException {
+ private void cloudBatchAfterCreatePartitions(boolean executeFirstTime,
+
List<Pair<PartitionPersistInfo, Partition>> batchPartsInfo,
+ ArrayList<AddPartitionOp>
addPartitionOps, Database db,
+ OlapTable olapTable,
List<Long> indexIds,
+ String tableName) throws
DdlException {
if (Config.isNotCloudMode()) {
return;
}
- List<Long> succeedPartitionIds =
partsInfo.stream().map(partitionPersistInfo
- ->
partitionPersistInfo.getPartition().getId()).collect(Collectors.toList());
+ List<Long> succeedPartitionIds =
batchPartsInfo.stream().map(partitionInfo
+ ->
partitionInfo.first.getPartition().getId()).collect(Collectors.toList());
if (!executeFirstTime || addPartitionOps.isEmpty()) {
LOG.info("cloud commit rpc in batch, {}-{}", !executeFirstTime,
addPartitionOps.size());
return;
@@ -847,7 +843,7 @@ public class DynamicPartitionScheduler extends MasterDaemon
{
succeedPartitionIds, indexIds, true /* isCreateTable */,
false /* isBatchCommit */);
LOG.info("begin write edit log to add partitions in batch, "
+ "numPartitions: {}, db: {}, table: {}, tableId: {}",
- partsInfo.size(), db.getFullName(), tableName,
olapTable.getId());
+ batchPartsInfo.size(), db.getFullName(), tableName,
olapTable.getId());
// ATTN: here, edit log must after commit cloud partition,
// prevent commit RPC failure from causing data loss
if
(DebugPointUtil.isEnable("FE.DynamicPartitionScheduler.before.logEditPartitions"))
{
@@ -855,20 +851,48 @@ public class DynamicPartitionScheduler extends
MasterDaemon {
// committed, but not log edit
throw new Exception("debug point
FE.DynamicPartitionScheduler.before.commitCloudPartition");
}
- for (int i = 0; i < partsInfo.size(); i++) {
-
Env.getCurrentEnv().getEditLog().logAddPartition(partsInfo.get(i));
- if
(DebugPointUtil.isEnable("FE.DynamicPartitionScheduler.in.logEditPartitions")) {
- if (i == partsInfo.size() / 2) {
- LOG.info("debug point
FE.DynamicPartitionScheduler.in.logEditPartitions, throw e");
- // committed, but log some edit, others failed
- throw new Exception("debug point
FE.DynamicPartitionScheduler"
- + ".in.commitCloudPartition");
+
+ for (int i = 0; i < batchPartsInfo.size(); i++) {
+ // get table write lock to add partition, edit log and modify
table state must be atomic
+ olapTable.writeLockOrDdlException();
+ try {
+ boolean isTempPartition =
addPartitionOps.get(i).isTempPartition();
+ Partition toAddPartition = batchPartsInfo.get(i).second;
+ String partitionName = toAddPartition.getName();
+ // ATTN: Check here to see if the newly created dynamic
+ // partition has already been added by another process.
+ // If it has, do not add this dynamic partition again,
+ // and call `onErasePartition` to clean up any remaining
information.
+ Partition checkIsAdded =
olapTable.getPartition(partitionName, isTempPartition);
+ if (checkIsAdded != null) {
+ LOG.warn("dynamic partition has been added, skip it. "
+ + "db: {}, table: {}, partition: {}, tableId:
{}",
+ db.getFullName(), tableName, partitionName,
olapTable.getId());
+ Env.getCurrentEnv().onErasePartition(toAddPartition);
+ continue;
+ }
+ if (isTempPartition) {
+ olapTable.addTempPartition(toAddPartition);
+ } else {
+ olapTable.addPartition(toAddPartition);
}
+
+
Env.getCurrentEnv().getEditLog().logAddPartition(batchPartsInfo.get(i).first);
+ if
(DebugPointUtil.isEnable("FE.DynamicPartitionScheduler.in.logEditPartitions")) {
+ if (i == batchPartsInfo.size() / 2) {
+ LOG.info("debug point
FE.DynamicPartitionScheduler.in.logEditPartitions, throw e");
+ // committed, but log some edit, others failed
+ throw new Exception("debug point
FE.DynamicPartitionScheduler"
+ + ".in.commitCloudPartition");
+ }
+ }
+ } finally {
+ olapTable.writeUnlock();
}
}
LOG.info("finish write edit log to add partitions in batch, "
+ "numPartitions: {}, db: {}, table: {}, tableId: {}",
- partsInfo.size(), db.getFullName(), tableName,
olapTable.getId());
+ batchPartsInfo.size(), db.getFullName(), tableName,
olapTable.getId());
} catch (Exception e) {
LOG.warn("cloud in commit step, dbName {}, tableName {}, tableId
{} exception {}",
db.getFullName(), tableName, olapTable.getId(),
e.getMessage());
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
index 454063d4f92..6302dbc9d18 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
@@ -1354,7 +1354,7 @@ public class InternalCatalog implements
CatalogIf<Database> {
} finally {
table.readUnlock();
}
- addPartition(db, tableName, addPartitionOp, false, 0, true);
+ addPartition(db, tableName, addPartitionOp, false, 0, true, null);
} catch (UserException e) {
throw new DdlException("Failed to ADD PARTITION " +
addPartitionLikeOp.getPartitionName()
@@ -1443,9 +1443,11 @@ public class InternalCatalog implements
CatalogIf<Database> {
}
}
- public PartitionPersistInfo addPartition(Database db, String tableName,
AddPartitionOp addPartitionOp,
+ public void addPartition(Database db, String tableName, AddPartitionOp
addPartitionOp,
boolean isCreateTable, long
generatedPartitionId,
- boolean writeEditLog) throws
DdlException {
+ boolean writeEditLog,
+ List<Pair<PartitionPersistInfo,
Partition>> batchPartitions)
+ throws DdlException {
// in cloud mode, isCreateTable == true, create dynamic partition use,
so partitionId must have been generated.
// isCreateTable == false, other case, partitionId generate in below,
must be set 0
if (!FeConstants.runningUnitTest && Config.isCloudMode()
@@ -1474,7 +1476,7 @@ public class InternalCatalog implements
CatalogIf<Database> {
if (singlePartitionDesc.isSetIfNotExists()) {
LOG.info("table[{}] add partition[{}] which already
exists", olapTable.getName(), partitionName);
if
(!DebugPointUtil.isEnable("InternalCatalog.addPartition.noCheckExists")) {
- return null;
+ return;
}
} else {
ErrorReport.reportDdlException(ErrorCode.ERR_SAME_NAME_PARTITION,
partitionName);
@@ -1641,7 +1643,7 @@ public class InternalCatalog implements
CatalogIf<Database> {
db, tableName, olapTable, partitionName, singlePartitionDesc);
if (ownerFutureOr.isErr()) {
if (ownerFutureOr.unwrapErr() == null) {
- return null;
+ return;
} else {
throw ownerFutureOr.unwrapErr();
}
@@ -1697,7 +1699,7 @@ public class InternalCatalog implements
CatalogIf<Database> {
LOG.info("table[{}] add partition[{}] which already
exists", olapTable.getName(), partitionName);
if (singlePartitionDesc.isSetIfNotExists()) {
failedCleanCallback.run();
- return null;
+ return;
} else {
ErrorReport.reportDdlException(ErrorCode.ERR_SAME_NAME_PARTITION,
partitionName);
}
@@ -1755,12 +1757,6 @@ public class InternalCatalog implements
CatalogIf<Database> {
// update partition info
partitionInfo.handleNewSinglePartitionDesc(singlePartitionDesc, partitionId,
isTempPartition);
- if (isTempPartition) {
- olapTable.addTempPartition(partition);
- } else {
- olapTable.addPartition(partition);
- }
-
// log
PartitionPersistInfo info = null;
if (partitionInfo.getType() == PartitionType.RANGE) {
@@ -1786,11 +1782,16 @@ public class InternalCatalog implements
CatalogIf<Database> {
}
if (writeEditLog) {
Env.getCurrentEnv().getEditLog().logAddPartition(info);
+ if (isTempPartition) {
+ olapTable.addTempPartition(partition);
+ } else {
+ olapTable.addPartition(partition);
+ }
LOG.info("succeed in creating partition[{}], temp: {}",
partitionId, isTempPartition);
} else {
+ batchPartitions.add(Pair.of(info, partition));
LOG.info("postpone creating partition[{}], temp: {}",
partitionId, isTempPartition);
}
- return info;
} finally {
olapTable.writeUnlock();
}
@@ -1837,7 +1838,7 @@ public class InternalCatalog implements
CatalogIf<Database> {
AddPartitionOp addPartitionOp = new AddPartitionOp(
singlePartitionDesc.translateToPartitionDefinition(), null,
multiPartitionOp.getProperties(), false);
- addPartition(db, tableName, addPartitionOp, false, 0, true);
+ addPartition(db, tableName, addPartitionOp, false, 0, true, null);
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionUtil.java
b/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionUtil.java
index eee02b62a98..924f51ce064 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionUtil.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionUtil.java
@@ -405,7 +405,7 @@ public class MTMVPartitionUtil {
mtmv.getDefaultDistributionInfo().toDistributionDesc().toDistributionDescriptor(),
partitionProperties, false);
Env.getCurrentEnv().addPartition((Database) mtmv.getDatabase(),
mtmv.getName(), addPartitionClause,
- false, 0, true);
+ false, 0, true, null);
}
/**
diff --git a/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
b/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
index 0102d1a32a7..4f126fcf214 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
@@ -1666,7 +1666,21 @@ public class EditLog {
}
public long logAddPartition(PartitionPersistInfo info) {
+ if (DebugPointUtil.isEnable("FE.logAddPartition.slow")) {
+ DebugPointUtil.DebugPoint debugPoint =
DebugPointUtil.getDebugPoint("FE.logAddPartition.slow");
+ String pName = debugPoint.param("pName", "");
+ if (info.getPartition().getName().equals(pName)) {
+ int sleepMs = debugPoint.param("sleep", 1000);
+ LOG.info("logAddPartition debug point hit, pName {}, sleep {}
s", pName, sleepMs);
+ try {
+ Thread.sleep(sleepMs);
+ } catch (InterruptedException e) {
+ LOG.warn("sleep interrupted", e);
+ }
+ }
+ }
long logId = logEdit(OperationType.OP_ADD_PARTITION, info);
+ LOG.info("log add partition, logId:{}, info: {}", logId,
info.toJson());
AddPartitionRecord record = new AddPartitionRecord(logId, info);
Env.getCurrentEnv().getBinlogManager().addAddPartitionRecord(record);
return logId;
@@ -1674,6 +1688,7 @@ public class EditLog {
public long logDropPartition(DropPartitionInfo info) {
long logId = logEdit(OperationType.OP_DROP_PARTITION, info);
+ LOG.info("log drop partition, logId:{}, info: {}", logId,
info.toJson());
Env.getCurrentEnv().getBinlogManager().addDropPartitionRecord(info,
logId);
return logId;
}
@@ -1684,6 +1699,7 @@ public class EditLog {
public void logRecoverPartition(RecoverInfo info) {
long logId = logEdit(OperationType.OP_RECOVER_PARTITION, info);
+ LOG.info("log recover partition, logId:{}, info: {}", logId,
info.toJson());
Env.getCurrentEnv().getBinlogManager().addRecoverTableRecord(info,
logId);
}
@@ -1702,6 +1718,7 @@ public class EditLog {
public void logDropTable(DropInfo info) {
long logId = logEdit(OperationType.OP_DROP_TABLE, info);
+ LOG.info("log drop table, logId : {}, infos: {}", logId, info);
if (Strings.isNullOrEmpty(info.getCtl()) ||
info.getCtl().equals(InternalCatalog.INTERNAL_CATALOG_NAME)) {
DropTableRecord record = new DropTableRecord(logId, info);
Env.getCurrentEnv().getBinlogManager().addDropTableRecord(record);
@@ -1714,11 +1731,13 @@ public class EditLog {
public void logRecoverTable(RecoverInfo info) {
long logId = logEdit(OperationType.OP_RECOVER_TABLE, info);
+ LOG.info("log recover table, logId : {}, infos: {}", logId, info);
Env.getCurrentEnv().getBinlogManager().addRecoverTableRecord(info,
logId);
}
public void logDropRollup(DropInfo info) {
long logId = logEdit(OperationType.OP_DROP_ROLLUP, info);
+ LOG.info("log drop rollup, logId : {}, infos: {}", logId, info);
Env.getCurrentEnv().getBinlogManager().addDropRollup(info, logId);
}
@@ -1835,7 +1854,8 @@ public class EditLog {
}
public void logDatabaseRename(DatabaseInfo databaseInfo) {
- logEdit(OperationType.OP_RENAME_DB, databaseInfo);
+ long logId = logEdit(OperationType.OP_RENAME_DB, databaseInfo);
+ LOG.info("log database rename, logId : {}, infos: {}", logId,
databaseInfo);
}
public void logTableRename(TableInfo tableInfo) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
index 3a1f56c7672..1720a27001a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
@@ -3631,7 +3631,7 @@ public class FrontendServiceImpl implements
FrontendService.Iface {
for (AddPartitionOp addPartitionOp : addPartitionClauseMap.values()) {
try {
// here maybe check and limit created partitions num
- Env.getCurrentEnv().addPartition(db, olapTable.getName(),
addPartitionOp, false, 0, true);
+ Env.getCurrentEnv().addPartition(db, olapTable.getName(),
addPartitionOp, false, 0, true, null);
} catch (DdlException e) {
LOG.warn(e);
errorStatus.setErrorMsgs(
diff --git
a/regression-test/suites/cloud_p0/diffrent_serialize/diffrent_serialize.groovy
b/regression-test/suites/cloud_p0/different_serialize/different_serialize.groovy
similarity index 99%
rename from
regression-test/suites/cloud_p0/diffrent_serialize/diffrent_serialize.groovy
rename to
regression-test/suites/cloud_p0/different_serialize/different_serialize.groovy
index 682a8fff9ad..9433c97e130 100644
---
a/regression-test/suites/cloud_p0/diffrent_serialize/diffrent_serialize.groovy
+++
b/regression-test/suites/cloud_p0/different_serialize/different_serialize.groovy
@@ -17,7 +17,7 @@
import org.codehaus.groovy.runtime.IOGroovyMethods
-suite ("diffrent_serialize_cloud") {
+suite ("different_serialize_cloud") {
sql """ DROP TABLE IF EXISTS d_table; """
diff --git
a/regression-test/suites/cloud_p0/partition/test_create_partition_and_insert_overwrite_race.groovy
b/regression-test/suites/cloud_p0/partition/test_create_partition_and_insert_overwrite_race.groovy
new file mode 100644
index 00000000000..58c259db816
--- /dev/null
+++
b/regression-test/suites/cloud_p0/partition/test_create_partition_and_insert_overwrite_race.groovy
@@ -0,0 +1,92 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+import com.mysql.cj.jdbc.StatementImpl
+import org.apache.doris.regression.suite.ClusterOptions
+import org.apache.doris.regression.util.NodeType
+import org.apache.doris.regression.suite.SuiteCluster
+
+suite("test_create_partition_and_insert_overwrite_race", 'p0, docker') {
+ if (!isCloudMode()) {
+ return
+ }
+ def options = new ClusterOptions()
+ options.enableDebugPoints()
+ // one master, one observer
+ options.setFeNum(2)
+ options.feConfigs.add('sys_log_verbose_modules=org')
+ options.setBeNum(3)
+ options.cloudMode = true
+
+ // 1. connect to observer
+ options.connectToFollower = true
+ docker(options) {
+ sql """set enable_sql_cache=false"""
+ def tbl = 'test_create_partition_and_insert_overwrite_race_tbl'
+ def tbl2 = 'test_create_partition_and_insert_overwrite_race_tbl2'
+ def createTableSql = { String tableName ->
+ sql """
+ CREATE TABLE ${tableName} (
+ order_id BIGINT,
+ create_dt datetime,
+ username VARCHAR(20)
+ )
+ DUPLICATE KEY(order_id)
+ PARTITION BY RANGE(create_dt) ()
+ DISTRIBUTED BY HASH(order_id) BUCKETS 10
+ PROPERTIES(
+ "dynamic_partition.enable" = "true",
+ "dynamic_partition.time_unit" = "DAY",
+ "dynamic_partition.start" = "-5",
+ "dynamic_partition.end" = "5",
+ "dynamic_partition.prefix" = "p",
+ "dynamic_partition.create_history_partition" = "true"
+ );
+ """
+ }
+
+ createTableSql(tbl)
+ createTableSql(tbl2)
+
+ // Generate insert statements with dates: current date -2, -1, 0, +1,
+2 days
+ def now = new Date()
+ def dateFormat = new java.text.SimpleDateFormat("yyyy-MM-dd")
+ for (def i = -2; i <= 2; i++) {
+ def targetDate = new Date(now.time + i * 24 * 60 * 60 * 1000L)
+ def dateStr = dateFormat.format(targetDate)
+ def hour = String.format("%02d", Math.abs(i) + 1)
+ def insertDate = "${dateStr} ${hour}:00:00"
+ sql """insert into ${tbl2} values (${i + 3}, '${insertDate}',
'test')"""
+ }
+
+ sql """DROP TABLE ${tbl}"""
+ def partitionNameFormat = new java.text.SimpleDateFormat("yyyyMMdd")
+ def currentPartitionName = "p" + partitionNameFormat.format(now)
+ cluster.injectDebugPoints(NodeType.FE,
['FE.logAddPartition.slow':[pName:currentPartitionName, sleep:50 * 1000]])
+ def futrue = thread {
+ for (def i = 0; i < 55; i++) {
+ try_sql """INSERT OVERWRITE TABLE ${tbl} partition(*) select *
from ${tbl2}"""
+ sleep(1 * 1000)
+ cluster.checkFeIsAlive(2, true)
+ }
+ }
+ def future1 = thread {
+ createTableSql(tbl)
+ }
+ futrue.get()
+ future1.get()
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
