This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 44a1c4ee6e Revert "[feature](property) Add table property
"is_being_synced" (#22314)"
44a1c4ee6e is described below
commit 44a1c4ee6e339dd72c35f641415a4c28dbf4f69a
Author: Kang <kxiao.ti...@gmail.com>
AuthorDate: Tue Aug 1 09:31:22 2023 +0800
Revert "[feature](property) Add table property "is_being_synced" (#22314)"
This reverts commit bf6ca76d29d9cc709c390ec96d7e304b8edce1c0.
---
docs/en/docs/advanced/autobucket.md | 4 +-
docs/en/docs/advanced/cold_hot_separation.md | 4 +-
docs/en/docs/advanced/is-being-synced.md | 65 ----------------------
.../docs/advanced/partition/dynamic-partition.md | 4 +-
.../join-optimization/colocation-join.md | 4 +-
.../Create/CREATE-TABLE.md | 10 ----
docs/zh-CN/docs/advanced/autobucket.md | 5 +-
docs/zh-CN/docs/advanced/cold_hot_separation.md | 4 +-
docs/zh-CN/docs/advanced/is-being-synced.md | 65 ----------------------
.../docs/advanced/partition/dynamic-partition.md | 2 -
.../join-optimization/colocation-join.md | 4 +-
.../Create/CREATE-TABLE.md | 10 ----
.../main/java/org/apache/doris/alter/Alter.java | 6 +-
.../org/apache/doris/alter/AlterOperations.java | 8 +--
.../analysis/ModifyTablePropertiesClause.java | 16 +++---
.../org/apache/doris/analysis/RestoreStmt.java | 23 +-------
.../org/apache/doris/backup/BackupHandler.java | 4 +-
.../java/org/apache/doris/backup/RestoreJob.java | 20 ++-----
.../org/apache/doris/binlog/CreateTableRecord.java | 4 +-
.../org/apache/doris/catalog/DistributionInfo.java | 6 +-
.../main/java/org/apache/doris/catalog/Env.java | 22 +-------
.../apache/doris/catalog/HashDistributionInfo.java | 4 +-
.../java/org/apache/doris/catalog/OlapTable.java | 43 ++++----------
.../doris/catalog/RandomDistributionInfo.java | 4 +-
.../org/apache/doris/catalog/TableProperty.java | 29 +++-------
.../doris/clone/DynamicPartitionScheduler.java | 1 -
.../doris/common/util/DynamicPartitionUtil.java | 1 -
.../apache/doris/common/util/PropertyAnalyzer.java | 14 +----
.../apache/doris/datasource/InternalCatalog.java | 14 +----
.../java/org/apache/doris/qe/ShowExecutor.java | 2 +-
.../apache/doris/service/FrontendServiceImpl.java | 2 +-
.../analysis/CreateTableAsSelectStmtTest.java | 20 -------
.../org/apache/doris/backup/RestoreJobTest.java | 2 +-
.../org/apache/doris/catalog/OlapTableTest.java | 4 +-
34 files changed, 71 insertions(+), 359 deletions(-)
diff --git a/docs/en/docs/advanced/autobucket.md
b/docs/en/docs/advanced/autobucket.md
index bc05e3e72f..d17e8d41ef 100644
--- a/docs/en/docs/advanced/autobucket.md
+++ b/docs/en/docs/advanced/autobucket.md
@@ -32,9 +32,7 @@ DISTRIBUTED BY ... BUCKETS auto
</version>
-Users often set inappropriate buckets, leading to various problems. For now,
it only works for olap tables
-
-Node: This feature will be disabled when synchronized by CCR. If this table is
copied by CCR, that is, PROPERTIES contains `is_being_synced = true`, it will
be displayed as enabled in show create table, but will not actually take
effect. When `is_being_synced` is set to `false`, these features will resume
working, but the `is_being_synced` property is for CCR peripheral modules only
and should not be manually set during CCR synchronization.
+Users often set inappropriate buckets, leading to various problems. For now,
it only works for olap tables
# Implementation
diff --git a/docs/en/docs/advanced/cold_hot_separation.md
b/docs/en/docs/advanced/cold_hot_separation.md
index df441ded63..5d7ce3063f 100644
--- a/docs/en/docs/advanced/cold_hot_separation.md
+++ b/docs/en/docs/advanced/cold_hot_separation.md
@@ -54,9 +54,7 @@ The storage policy is the entry to use the cold and hot
separation function. Use
<version since="dev"></version> When creating an S3 RESOURCE, the S3 remote
link verification will be performed to ensure that the RESOURCE is created
correctly.
-In addition, fe configuration needs to be added: `enable_storage_policy=true`
-
-Note: This property will not be synchronized by CCR. If this table is copied
by CCR, that is, PROPERTIES contains `is_being_synced = true`, this property
will be erased in this table.
+In addition, fe configuration needs to be added: `enable_storage_policy=true`
For example:
diff --git a/docs/en/docs/advanced/is-being-synced.md
b/docs/en/docs/advanced/is-being-synced.md
deleted file mode 100644
index 028e740ec9..0000000000
--- a/docs/en/docs/advanced/is-being-synced.md
+++ /dev/null
@@ -1,65 +0,0 @@
----
-{
- "title": "property is_being_synced",
- "language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Background
-
-<version since="2.0">
-
-"is_being_synced" = "true"
-
-</version>
-
-CCR will create replica tables (referred to as target tables, located in the
dest cluster) of the tables (referred to as source tables, located in the
source cluster) in the synchronization scope of the source cluster in the dest
cluster when establishing synchronization, but some functions and properties
need to be invalidated or erased when creating replica tables to ensure the
correctness of the synchronization process.
-
-such as：
-- The source table contains information that may not have been synchronized to
the dest cluster, such as `storage_policy`, etc., which may cause the target
table creation to fail or behave abnormally.
-- The source table may contain some dynamic functions, such as dynamic
partitioning, etc., which may cause the behavior of the target table to be out
of syncer control and cause partition inconsistency.
-
-The properties that need to be erased due to invalidation when being
replicated are:
-- `storage_policy`
-- `colocate_with`
-
-The functions that need to be invalidated when being synchronized are:
-- auto bucket
-- dynamic partition
-
-# Implementation
-When creating the target table, this properties will be added or deleted by
syncer control. In CCR, there are two ways to create a target table:
-1. When synchronizing tables, syncer uses backup/restore method to perform
full replication of the source table to obtain the target table.
-2. When synchronizing databases, for existing tables, syncer also uses
backup/restore method to obtain the target table, for incremental tables,
syncer will create the target table through binlog with CreateTableRecord.
-
-In summary, there are two entry points for inserting `is_being_synced`
attribute: the restore process in full synchronization and the getDdlStmt in
incremental synchronization.
-
-In the restore process of full synchronization, syncer will initiate the
restore of the snapshot in the original cluster through rpc. In this process,
it will add `is_being_synced` attribute to RestoreStmt and take effect in the
final restoreJob, executing the relevant logic of `isBeingSynced`.
-In the getDdlStmt of incremental synchronization, add the parameter `boolean
getDdlForSync` to the getDdlStmt method to distinguish whether it is an
operation to convert to the target table ddl under control, and execute the
relevant logic of `isBeingSynced` when creating the target table.
-
-There is no need to say more about the erasure of invalid properties. The
invalidation of the above functions needs to be explained:
-1. auto bucket
- Auto bucket takes effect when creating a table, calculating the current
appropriate number of buckets, which may cause the number of buckets in the
source table and the destination table to be inconsistent. Therefore, when
synchronizing, you need to obtain the number of buckets in the source table,
and also need to obtain the information whether the source table is an
automatic bucketing table in order to restore the function after ending the
synchronization. The current approach is [...]
-2. dynamic partition
- Dynamic partition is achieved by adding `olapTable.isBeingSynced()` to the
judgment of whether to execute add/drop partition, so that the target table
will not periodically execute add/drop partition operation during the
synchronization process.
-# Note
-When no exception occurs, the `is_being_synced` attribute should be completely
controlled by syncer to turn on or off, and users should not modify the
attribute by themselves.
\ No newline at end of file
diff --git a/docs/en/docs/advanced/partition/dynamic-partition.md
b/docs/en/docs/advanced/partition/dynamic-partition.md
index 0511b93af4..08c3e66cbf 100644
--- a/docs/en/docs/advanced/partition/dynamic-partition.md
+++ b/docs/en/docs/advanced/partition/dynamic-partition.md
@@ -30,9 +30,7 @@ Dynamic partition is a new feature introduced in Doris
version 0.12. It's design
At present, the functions of dynamically adding partitions and dynamically
deleting partitions are realized.
-Dynamic partitioning is only supported for Range partitions.
-
-Node: This feature will be disabled when synchronized by CCR. If this table is
copied by CCR, that is, PROPERTIES contains `is_being_synced = true`, it will
be displayed as enabled in show create table, but will not actually take
effect. When `is_being_synced` is set to `false`, these features will resume
working, but the `is_being_synced` property is for CCR peripheral modules only
and should not be manually set during CCR synchronization.
+Dynamic partitioning is only supported for Range partitions.
## Noun Interpretation
diff --git
a/docs/en/docs/query-acceleration/join-optimization/colocation-join.md
b/docs/en/docs/query-acceleration/join-optimization/colocation-join.md
index a9b6c2ab79..15357c2913 100644
--- a/docs/en/docs/query-acceleration/join-optimization/colocation-join.md
+++ b/docs/en/docs/query-acceleration/join-optimization/colocation-join.md
@@ -30,9 +30,7 @@ Colocation Join is a new feature introduced in Doris 0.9. The
purpose of this pa
The original design, implementation and effect can be referred to [ISSUE
245](https://github.com/apache/incubator-doris/issues/245).
-The Colocation Join function has undergone a revision, and its design and use
are slightly different from the original design. This document mainly
introduces Colocation Join's principle, implementation, usage and precautions.
-
-Note: This property will not be synchronized by CCR. If this table is copied
by CCR, that is, PROPERTIES contains `is_being_synced = true`, this property
will be erased in this table.
+The Colocation Join function has undergone a revision, and its design and use
are slightly different from the original design. This document mainly
introduces Colocation Join's principle, implementation, usage and precautions.
## Noun Interpretation
diff --git
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
index ab9e3a6f0f..a2f8c4731b 100644
---
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
+++
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
@@ -309,16 +309,6 @@ Set table properties. The following attributes are
currently supported:
Set the copy distribution according to Tag. This attribute can completely
cover the function of the `replication_num` attribute.
-* `is_being_synced`
-
- Used to identify whether this table is copied by CCR and is being
synchronized by syncer. The default is `false`.
-
- If set to `true`:
- `colocate_with`, `storage_policy` properties will be erased
- `dynamic partition`, `auto bucket` features will be disabled, that is,
they will be displayed as enabled in `show create table`, but will not actually
take effect. When `is_being_synced` is set to `false`, these features will
resume working.
-
- This property is for CCR peripheral modules only and should not be
manually set during CCR synchronization.
-
* `storage_medium/storage_cooldown_time`
Data storage medium. `storage_medium` is used to declare the initial
storage medium of the table data, and `storage_cooldown_time` is used to set
the expiration time. Example:
diff --git a/docs/zh-CN/docs/advanced/autobucket.md
b/docs/zh-CN/docs/advanced/autobucket.md
index 43a1124db7..880f5214f5 100644
--- a/docs/zh-CN/docs/advanced/autobucket.md
+++ b/docs/zh-CN/docs/advanced/autobucket.md
@@ -32,10 +32,7 @@ DISTRIBUTED BY ... BUCKETS auto
</version>
-用户经常设置不合适的bucket，导致各种问题，这里提供一种方式，来自动设置分桶数。暂时而言只对olap表生效
-
-注意：这个功能在被CCR同步时将会失效。如果这个表是被CCR复制而来的，即PROPERTIES中包含`is_being_synced =
true`时，在`show create table`中会显示开启状态，但不会实际生效。当`is_being_synced`被设置为 `false`
时，这些功能将会恢复生效，但`is_being_synced`属性仅供CCR外围模块使用，在CCR同步的过程中不要手动设置。
-
+用户经常设置不合适的bucket，导致各种问题，这里提供一种方式，来自动设置分桶数。暂时而言只对olap表生效
# 实现
diff --git a/docs/zh-CN/docs/advanced/cold_hot_separation.md
b/docs/zh-CN/docs/advanced/cold_hot_separation.md
index 3d157c8e6c..e952b42f23 100644
--- a/docs/zh-CN/docs/advanced/cold_hot_separation.md
+++ b/docs/zh-CN/docs/advanced/cold_hot_separation.md
@@ -53,9 +53,7 @@ under the License.
<version since="dev"></version> 创建S3 RESOURCE的时候，会进行S3远端的链接校验，以保证RESOURCE创建的正确。
-此外，需要新增fe配置：`enable_storage_policy=true`
-
-注意：这个属性不会被CCR同步，如果这个表是被CCR复制而来的，即PROPERTIES中包含`is_being_synced =
true`时，这个属性将会在这个表中被擦除。
+此外，需要新增fe配置：`enable_storage_policy=true`
例如：
diff --git a/docs/zh-CN/docs/advanced/is-being-synced.md
b/docs/zh-CN/docs/advanced/is-being-synced.md
deleted file mode 100644
index 1d98a3b4e9..0000000000
--- a/docs/zh-CN/docs/advanced/is-being-synced.md
+++ /dev/null
@@ -1,65 +0,0 @@
----
-{
- "title": "is_being_synced属性",
- "language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# 背景
-
-<version since="2.0">
-
-"is_being_synced" = "true"
-
-</version>
-
-CCR功能在建立同步时，会在目标集群中创建源集群同步范围中表（后称源表，位于源集群）的副本表（后称目标表，位于目标集群），但是在创建副本表时需要失效或者擦除一些功能和属性以保证同步过程中的正确性。
-
-如：
-- 源表中包含了可能没有被同步到目标集群的信息，如`storage_policy`等，可能会导致目标表创建失败或者行为异常。
-- 源表中可能包含一些动态功能，如动态分区等，可能导致目标表的行为不受syncer控制导致partition不一致。
-
-在被复制时因失效而需要擦除的属性有：
-- `storage_policy`
-- `colocate_with`
-
-在被同步时需要失效的功能有：
-- 自动分桶
-- 动态分区
-
-# 实现
-在创建目标表时，这条属性将会由syncer控制添加或者删除，在CCR功能中，创建一个目标表有两个途径：
-1. 在表同步时，syncer通过backup/restore的方式对源表进行全量复制来得到目标表。
-2.
在库同步时，对于存量表而言，syncer同样通过backup/restore的方式来得到目标表，对于增量表而言，syncer会通过携带有CreateTableRecord的binlog来创建目标表。
-
-综上，对于插入`is_being_synced`属性有两个切入点：全量同步中的restore过程和增量同步时的getDdlStmt。
-
-在全量同步的restore过程中，syncer会通过rpc发起对原集群中snapshot的restore，在这个过程中为会为RestoreStmt添加`is_being_synced`属性，并在最终的restoreJob中生效，执行`isBeingSynced`的相关逻辑。
-在增量同步时的getDdlStmt中，为getDdlStmt方法添加参数`boolean
getDdlForSync`，以区分是否为受控转化为目标表ddl的操作，并在创建目标表时执行`isBeingSynced`的相关逻辑。
-
-对于失效属性的擦除无需多言，对于上述功能的失效需要进行说明：
-1. 自动分桶
-
自动分桶会在创建表时生效，计算当前合适的bucket数量，这就可能导致源表和目的表的bucket数目不一致。因此在同步时需要获得源表的bucket数目，并且也要获得源表是否为自动分桶表的信息以便结束同步后恢复功能。当前的做法是在获取distribution信息时默认autobucket为false，在恢复表时通过检查`_auto_bucket`属性来判断源表是否为自动分桶表，如是则将目标表的autobucket字段设置为true，以此来达到跳过计算bucket数量，直接应用源表bucket数量的目的。
-2. 动态分区
- 动态分区则是通过将`olapTable.isBeingSynced()`添加到是否执行add/drop
partition的判断中来实现的，这样目标表在被同步的过程中就不会周期性的执行add/drop partition操作。
-# 注意
-在未出现异常时，`is_being_synced`属性应该完全由syncer控制开启或关闭，用户不要自行修改该属性。
\ No newline at end of file
diff --git a/docs/zh-CN/docs/advanced/partition/dynamic-partition.md
b/docs/zh-CN/docs/advanced/partition/dynamic-partition.md
index 6aca4aafc3..38ea866070 100644
--- a/docs/zh-CN/docs/advanced/partition/dynamic-partition.md
+++ b/docs/zh-CN/docs/advanced/partition/dynamic-partition.md
@@ -32,8 +32,6 @@ under the License.
动态分区只支持 Range 分区。
-注意：这个功能在被CCR同步时将会失效。如果这个表是被CCR复制而来的，即PROPERTIES中包含`is_being_synced =
true`时，在`show create table`中会显示开启状态，但不会实际生效。当`is_being_synced`被设置为 `false`
时，这些功能将会恢复生效，但`is_being_synced`属性仅供CCR外围模块使用，在CCR同步的过程中不要手动设置。
-
## 原理
在某些使用场景下，用户会将表按照天进行分区划分，每天定时执行例行任务，这时需要使用方手动管理分区，否则可能由于使用方没有创建分区导致数据导入失败，这给使用方带来了额外的维护成本。
diff --git
a/docs/zh-CN/docs/query-acceleration/join-optimization/colocation-join.md
b/docs/zh-CN/docs/query-acceleration/join-optimization/colocation-join.md
index 95661219cd..e357683dc8 100644
--- a/docs/zh-CN/docs/query-acceleration/join-optimization/colocation-join.md
+++ b/docs/zh-CN/docs/query-acceleration/join-optimization/colocation-join.md
@@ -30,9 +30,7 @@ Colocation Join 是在 Doris 0.9 版本中引入的新功能。旨在为某些 J
最初的设计、实现和效果可以参阅 [ISSUE
245](https://github.com/apache/incubator-doris/issues/245)。
-Colocation Join 功能经过一次改版，设计和使用方式和最初设计稍有不同。本文档主要介绍 Colocation Join
的原理、实现、使用方式和注意事项。
-
-注意：这个属性不会被CCR同步，如果这个表是被CCR复制而来的，即PROPERTIES中包含`is_being_synced =
true`时，这个属性将会在这个表中被擦除。
+Colocation Join 功能经过一次改版，设计和使用方式和最初设计稍有不同。本文档主要介绍 Colocation Join
的原理、实现、使用方式和注意事项。
## 名词解释
diff --git
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
index 2fc5d23009..01266687c0 100644
---
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
+++
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md
@@ -293,16 +293,6 @@ UNIQUE KEY(k1, k2)
根据 Tag 设置副本分布情况。该属性可以完全覆盖 `replication_num` 属性的功能。
-* `is_being_synced`
-
- 用于标识此表是否是被CCR复制而来并且正在被syncer同步，默认为 `false`。
-
- 如果设置为 `true`：
- `colocate_with`，`storage_policy`属性将被擦除
- `dynamic partition`，`auto bucket`功能将会失效，即在`show create
table`中显示开启状态，但不会实际生效。当`is_being_synced`被设置为 `false` 时，这些功能将会恢复生效。
-
- 这个属性仅供CCR外围模块使用，在CCR同步的过程中不要手动设置。
-
* `storage_medium/storage_cooldown_time`
数据存储介质。`storage_medium` 用于声明表数据的初始存储介质，而 `storage_cooldown_time`
用于设定到期时间。示例：
diff --git a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
index 53eb9052e8..a82209a866 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
@@ -208,8 +208,8 @@ public class Alter {
olapTable.setStoragePolicy(currentStoragePolicy);
needProcessOutsideTableLock = true;
- } else if (currentAlterOps.checkIsBeingSynced(alterClauses)) {
-
olapTable.setIsBeingSynced(currentAlterOps.isBeingSynced(alterClauses));
+ } else if (currentAlterOps.checkCcrEnable(alterClauses)) {
+ olapTable.setCcrEnable(currentAlterOps.isCcrEnable(alterClauses));
needProcessOutsideTableLock = true;
} else if (currentAlterOps.checkBinlogConfigChange(alterClauses)) {
if (!Config.enable_feature_binlog) {
@@ -514,7 +514,7 @@ public class Alter {
// currently, only in memory and storage policy property could
reach here
Preconditions.checkState(properties.containsKey(PropertyAnalyzer.PROPERTIES_INMEMORY)
||
properties.containsKey(PropertyAnalyzer.PROPERTIES_STORAGE_POLICY)
- ||
properties.containsKey(PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED));
+ ||
properties.containsKey(PropertyAnalyzer.PROPERTIES_CCR_ENABLE));
((SchemaChangeHandler)
schemaChangeHandler).updateTableProperties(db, tableName, properties);
} else {
throw new DdlException("Invalid alter operation: " +
alterClause.getOpType());
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/alter/AlterOperations.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/AlterOperations.java
index 22104522ed..632b60be41 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/AlterOperations.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/AlterOperations.java
@@ -78,10 +78,10 @@ public class AlterOperations {
).map(c -> ((ModifyTablePropertiesClause)
c).getStoragePolicy()).findFirst().orElse("");
}
- public boolean checkIsBeingSynced(List<AlterClause> alterClauses) {
+ public boolean checkCcrEnable(List<AlterClause> alterClauses) {
return alterClauses.stream().filter(clause ->
clause instanceof ModifyTablePropertiesClause
- ).anyMatch(clause ->
clause.getProperties().containsKey(PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED));
+ ).anyMatch(clause ->
clause.getProperties().containsKey(PropertyAnalyzer.PROPERTIES_CCR_ENABLE));
}
public boolean checkBinlogConfigChange(List<AlterClause> alterClauses) {
@@ -93,10 +93,10 @@ public class AlterOperations {
||
clause.getProperties().containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_MAX_HISTORY_NUMS));
}
- public boolean isBeingSynced(List<AlterClause> alterClauses) {
+ public boolean isCcrEnable(List<AlterClause> alterClauses) {
return alterClauses.stream().filter(clause ->
clause instanceof ModifyTablePropertiesClause
- ).map(c -> ((ModifyTablePropertiesClause)
c).isBeingSynced()).findFirst().orElse(false);
+ ).map(c -> ((ModifyTablePropertiesClause)
c).isCcrEnable()).findFirst().orElse(false);
}
// MODIFY_TABLE_PROPERTY is also processed by SchemaChangeHandler
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
index 93ece524f2..43c5211e07 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
@@ -44,14 +44,14 @@ public class ModifyTablePropertiesClause extends
AlterTableClause {
private String storagePolicy;
- private boolean isBeingSynced = false;
+ private boolean ccrEnable = false;
- public void setIsBeingSynced(boolean isBeingSynced) {
- this.isBeingSynced = isBeingSynced;
+ public void setCcrEnable(boolean ccrEnable) {
+ this.ccrEnable = ccrEnable;
}
- public boolean isBeingSynced() {
- return isBeingSynced;
+ public boolean isCcrEnable() {
+ return ccrEnable;
}
public ModifyTablePropertiesClause(Map<String, String> properties) {
@@ -133,10 +133,10 @@ public class ModifyTablePropertiesClause extends
AlterTableClause {
throw new AnalysisException("Can not change
enable_duplicate_without_keys_by_default");
} else if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_ENABLE_LIGHT_SCHEMA_CHANGE))
{
// do nothing, will be alter in
SchemaChangeHandler.updateTableProperties
- } else if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED)) {
+ } else if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_CCR_ENABLE)) {
this.needTableStable = false;
- setIsBeingSynced(Boolean.parseBoolean(properties.getOrDefault(
- PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED, "false")));
+ setCcrEnable(
+
Boolean.parseBoolean(properties.getOrDefault(PropertyAnalyzer.PROPERTIES_CCR_ENABLE,
"false")));
} else if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_ENABLE)
||
properties.containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_TTL_SECONDS)
||
properties.containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_MAX_BYTES)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
index 3b5f46a37c..895ad48ce9 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
@@ -39,7 +39,6 @@ public class RestoreStmt extends AbstractBackupStmt {
private static final String PROP_META_VERSION = "meta_version";
private static final String PROP_RESERVE_REPLICA = "reserve_replica";
private static final String PROP_RESERVE_DYNAMIC_PARTITION_ENABLE =
"reserve_dynamic_partition_enable";
- private static final String PROP_IS_BEING_SYNCED =
PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED;
private boolean allowLoad = false;
private ReplicaAllocation replicaAlloc =
ReplicaAllocation.DEFAULT_ALLOCATION;
@@ -48,7 +47,6 @@ public class RestoreStmt extends AbstractBackupStmt {
private boolean reserveReplica = false;
private boolean reserveDynamicPartitionEnable = false;
private boolean isLocal = false;
- private boolean isBeingSynced = false;
private byte[] meta = null;
private byte[] jobInfo = null;
@@ -100,12 +98,8 @@ public class RestoreStmt extends AbstractBackupStmt {
return jobInfo;
}
- public void setIsBeingSynced() {
- setProperty(PROP_IS_BEING_SYNCED, "true");
- }
-
- public boolean isBeingSynced() {
- return isBeingSynced;
+ public void disableDynamicPartition() {
+ setProperty(PROP_RESERVE_DYNAMIC_PARTITION_ENABLE, "false");
}
@Override
@@ -201,19 +195,6 @@ public class RestoreStmt extends AbstractBackupStmt {
copiedProperties.remove(PROP_META_VERSION);
}
- // is being synced
- if (copiedProperties.containsKey(PROP_IS_BEING_SYNCED)) {
- if
(copiedProperties.get(PROP_IS_BEING_SYNCED).equalsIgnoreCase("true")) {
- isBeingSynced = true;
- } else if
(copiedProperties.get(PROP_IS_BEING_SYNCED).equalsIgnoreCase("false")) {
- isBeingSynced = false;
- } else {
- ErrorReport.reportAnalysisException(ErrorCode.ERR_COMMON_ERROR,
- "Invalid is being synced value: " +
copiedProperties.get(PROP_IS_BEING_SYNCED));
- }
- copiedProperties.remove(PROP_IS_BEING_SYNCED);
- }
-
if (!copiedProperties.isEmpty()) {
ErrorReport.reportAnalysisException(ErrorCode.ERR_COMMON_ERROR,
"Unknown restore job properties: " +
copiedProperties.keySet());
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/backup/BackupHandler.java
b/fe/fe-core/src/main/java/org/apache/doris/backup/BackupHandler.java
index 5279175fa0..6619c457b9 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/BackupHandler.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/BackupHandler.java
@@ -450,7 +450,7 @@ public class BackupHandler extends MasterDaemon implements
Writable {
restoreJob = new RestoreJob(stmt.getLabel(), backupTimestamp,
db.getId(), db.getFullName(), jobInfo,
stmt.allowLoad(), stmt.getReplicaAlloc(),
stmt.getTimeoutMs(), stmt.getMetaVersion(),
stmt.reserveReplica(),
- stmt.reserveDynamicPartitionEnable(),
stmt.isBeingSynced(),
+ stmt.reserveDynamicPartitionEnable(),
env, Repository.KEEP_ON_LOCAL_REPO_ID, backupMeta);
} catch (IOException e) {
throw new DdlException(e.getMessage());
@@ -459,7 +459,7 @@ public class BackupHandler extends MasterDaemon implements
Writable {
restoreJob = new RestoreJob(stmt.getLabel(),
stmt.getBackupTimestamp(),
db.getId(), db.getFullName(), jobInfo, stmt.allowLoad(),
stmt.getReplicaAlloc(),
stmt.getTimeoutMs(), stmt.getMetaVersion(),
stmt.reserveReplica(), stmt.reserveDynamicPartitionEnable(),
- stmt.isBeingSynced(), env, repository.getId());
+ env, repository.getId());
}
env.getEditLog().logRestoreJob(restoreJob);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
index a5256dac64..6bd606a378 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
@@ -60,7 +60,6 @@ import org.apache.doris.common.MetaNotFoundException;
import org.apache.doris.common.Pair;
import org.apache.doris.common.io.Text;
import org.apache.doris.common.util.DynamicPartitionUtil;
-import org.apache.doris.common.util.PropertyAnalyzer;
import org.apache.doris.common.util.TimeUtils;
import org.apache.doris.resource.Tag;
import org.apache.doris.task.AgentBatchTask;
@@ -105,7 +104,6 @@ import java.util.stream.Collectors;
public class RestoreJob extends AbstractJob {
private static final String PROP_RESERVE_REPLICA = "reserve_replica";
private static final String PROP_RESERVE_DYNAMIC_PARTITION_ENABLE =
"reserve_dynamic_partition_enable";
- private static final String PROP_IS_BEING_SYNCED =
PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED;
private static final Logger LOG = LogManager.getLogger(RestoreJob.class);
@@ -167,9 +165,6 @@ public class RestoreJob extends AbstractJob {
// NOTICE: because we do not persist it, this info may be lost if Frontend
restart,
// and if you don't want to losing it, backup your data again by using
latest Doris version.
private int metaVersion = -1;
-
- private boolean isBeingSynced = false;
-
// restore properties
private Map<String, String> properties = Maps.newHashMap();
@@ -179,7 +174,7 @@ public class RestoreJob extends AbstractJob {
public RestoreJob(String label, String backupTs, long dbId, String dbName,
BackupJobInfo jobInfo, boolean allowLoad,
ReplicaAllocation replicaAlloc, long timeoutMs, int metaVersion,
boolean reserveReplica,
- boolean reserveDynamicPartitionEnable, boolean isBeingSynced, Env
env, long repoId) {
+ boolean reserveDynamicPartitionEnable, Env env, long repoId) {
super(JobType.RESTORE, label, dbId, dbName, timeoutMs, env, repoId);
this.backupTimestamp = backupTs;
this.jobInfo = jobInfo;
@@ -189,17 +184,15 @@ public class RestoreJob extends AbstractJob {
this.metaVersion = metaVersion;
this.reserveReplica = reserveReplica;
this.reserveDynamicPartitionEnable = reserveDynamicPartitionEnable;
- this.isBeingSynced = isBeingSynced;
properties.put(PROP_RESERVE_REPLICA, String.valueOf(reserveReplica));
properties.put(PROP_RESERVE_DYNAMIC_PARTITION_ENABLE,
String.valueOf(reserveDynamicPartitionEnable));
- properties.put(PROP_IS_BEING_SYNCED, String.valueOf(isBeingSynced));
}
public RestoreJob(String label, String backupTs, long dbId, String dbName,
BackupJobInfo jobInfo, boolean allowLoad,
ReplicaAllocation replicaAlloc, long timeoutMs, int metaVersion,
boolean reserveReplica,
- boolean reserveDynamicPartitionEnable, boolean isBeingSynced, Env
env, long repoId, BackupMeta backupMeta) {
+ boolean reserveDynamicPartitionEnable, Env env, long repoId,
BackupMeta backupMeta) {
this(label, backupTs, dbId, dbName, jobInfo, allowLoad, replicaAlloc,
timeoutMs, metaVersion, reserveReplica,
- reserveDynamicPartitionEnable, isBeingSynced, env, repoId);
+ reserveDynamicPartitionEnable, env, repoId);
this.backupMeta = backupMeta;
}
@@ -219,10 +212,6 @@ public class RestoreJob extends AbstractJob {
return metaVersion;
}
- public boolean isBeingSynced() {
- return isBeingSynced;
- }
-
public synchronized boolean finishTabletSnapshotTask(SnapshotTask task,
TFinishTaskRequest request) {
if (checkTaskStatus(task, task.getJobId(), request)) {
return false;
@@ -699,7 +688,7 @@ public class RestoreJob extends AbstractJob {
// Reset properties to correct values.
remoteOlapTbl.resetPropertiesForRestore(reserveDynamicPartitionEnable,
reserveReplica,
- replicaAlloc,
isBeingSynced);
+ replicaAlloc);
// DO NOT set remote table's new name here, cause we will
still need the origin name later
//
remoteOlapTbl.setName(jobInfo.getAliasByOriginNameIfSet(tblInfo.name));
@@ -2146,7 +2135,6 @@ public class RestoreJob extends AbstractJob {
}
reserveReplica =
Boolean.parseBoolean(properties.get(PROP_RESERVE_REPLICA));
reserveDynamicPartitionEnable =
Boolean.parseBoolean(properties.get(PROP_RESERVE_DYNAMIC_PARTITION_ENABLE));
- isBeingSynced =
Boolean.parseBoolean(properties.get(PROP_IS_BEING_SYNCED));
}
@Override
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/binlog/CreateTableRecord.java
b/fe/fe-core/src/main/java/org/apache/doris/binlog/CreateTableRecord.java
index 145b45396f..50557195b4 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/binlog/CreateTableRecord.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/binlog/CreateTableRecord.java
@@ -63,8 +63,8 @@ public class CreateTableRecord {
table.readLock();
try {
- Env.getSyncedDdlStmt(table, createTableStmt, addPartitionStmt,
createRollupStmt,
- false, false /* show password */, -1L);
+ Env.getDdlStmt(table, createTableStmt, addPartitionStmt,
createRollupStmt, false, false /* show password */,
+ -1L);
} finally {
table.readUnlock();
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/DistributionInfo.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/DistributionInfo.java
index a104ff8b57..fdb3513f7d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/DistributionInfo.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/DistributionInfo.java
@@ -95,12 +95,8 @@ public abstract class DistributionInfo implements Writable {
type = DistributionInfoType.valueOf(Text.readString(in));
}
- public String toSql(boolean forSync) {
- return "";
- }
-
public String toSql() {
- return toSql(false);
+ return "";
}
@Override
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index dc5598b710..5d380fe7c7 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -2832,14 +2832,7 @@ public class Env {
List<String> createRollupStmt, boolean
separatePartition, boolean hidePassword,
long specificVersion) {
getDdlStmt(null, null, table, createTableStmt, addPartitionStmt,
createRollupStmt, separatePartition,
- hidePassword, false, specificVersion, false, false);
- }
-
- public static void getSyncedDdlStmt(TableIf table, List<String>
createTableStmt, List<String> addPartitionStmt,
- List<String> createRollupStmt, boolean
separatePartition, boolean hidePassword,
- long specificVersion) {
- getDdlStmt(null, null, table, createTableStmt, addPartitionStmt,
createRollupStmt, separatePartition,
- hidePassword, false, specificVersion, false, true);
+ hidePassword, false, specificVersion, false);
}
/**
@@ -2851,7 +2844,7 @@ public class Env {
List<String> addPartitionStmt, List<String>
createRollupStmt,
boolean separatePartition,
boolean hidePassword, boolean getDdlForLike,
long specificVersion,
- boolean getBriefDdl, boolean getDdlForSync) {
+ boolean getBriefDdl) {
StringBuilder sb = new StringBuilder();
// 1. create table
@@ -3044,15 +3037,6 @@ public class Env {
sb.append(partition.getVisibleVersion()).append("\"");
}
- // mark this table is being synced
-
sb.append(",\n\"").append(PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED).append("\"
= \"");
- sb.append(String.valueOf(olapTable.isBeingSynced() ||
getDdlForSync)).append("\"");
- // mark this table if it is a auto bucket table
- if (getDdlForSync && olapTable.isAutoBucket()) {
-
sb.append(",\n\"").append(PropertyAnalyzer.PROPERTIES_AUTO_BUCKET).append("\" =
\"");
- sb.append("true").append("\"");
- }
-
// colocateTable
String colocateTable = olapTable.getColocateGroup();
if (colocateTable != null) {
@@ -4511,7 +4495,7 @@ public class Env {
}
tableProperty.buildInMemory()
.buildStoragePolicy()
- .buildIsBeingSynced();
+ .buildCcrEnable();
// need to update partition info meta
for (Partition partition : table.getPartitions()) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/HashDistributionInfo.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/HashDistributionInfo.java
index b4dc9f85a0..5f30bc2098 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/HashDistributionInfo.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/HashDistributionInfo.java
@@ -138,7 +138,7 @@ public class HashDistributionInfo extends DistributionInfo {
}
@Override
- public String toSql(boolean forSync) {
+ public String toSql() {
StringBuilder builder = new StringBuilder();
builder.append("DISTRIBUTED BY HASH(");
@@ -149,7 +149,7 @@ public class HashDistributionInfo extends DistributionInfo {
String colList = Joiner.on(", ").join(colNames);
builder.append(colList);
- if (autoBucket && !forSync) {
+ if (autoBucket) {
builder.append(") BUCKETS AUTO");
} else {
builder.append(") BUCKETS ").append(bucketNum);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
index 977e7f9e45..6664dfe537 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
@@ -210,15 +210,6 @@ public class OlapTable extends Table {
getOrCreatTableProperty().setBinlogConfig(binlogConfig);
}
- public void setIsBeingSynced(boolean isBeingSynced) {
-
getOrCreatTableProperty().modifyTableProperties(PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED,
- String.valueOf(isBeingSynced));
- }
-
- public boolean isBeingSynced() {
- return getOrCreatTableProperty().isBeingSynced();
- }
-
public void setTableProperty(TableProperty tableProperty) {
this.tableProperty = tableProperty;
}
@@ -508,18 +499,10 @@ public class OlapTable extends Table {
* Reset properties to correct values.
*/
public void resetPropertiesForRestore(boolean
reserveDynamicPartitionEnable, boolean reserveReplica,
- ReplicaAllocation replicaAlloc,
boolean isBeingSynced) {
+ ReplicaAllocation replicaAlloc) {
if (tableProperty != null) {
tableProperty.resetPropertiesForRestore(reserveDynamicPartitionEnable,
reserveReplica, replicaAlloc);
}
- if (isBeingSynced) {
- TableProperty tableProperty = getOrCreatTableProperty();
- tableProperty.setIsBeingSynced();
- tableProperty.removeInvalidProperties();
- if (isAutoBucket()) {
- markAutoBucket();
- }
- }
// remove colocate property.
setColocateGroup(null);
}
@@ -770,10 +753,6 @@ public class OlapTable extends Table {
return defaultDistributionInfo;
}
- public void markAutoBucket() {
- defaultDistributionInfo.markAutoBucket();
- }
-
public Set<String> getDistributionColumnNames() {
Set<String> distributionColumnNames = Sets.newHashSet();
if (defaultDistributionInfo instanceof RandomDistributionInfo) {
@@ -1709,15 +1688,6 @@ public class OlapTable extends Table {
return hasChanged;
}
- public void ignoreInvaildPropertiesWhenSynced(Map<String, String>
properties) {
- // ignore colocate table
- PropertyAnalyzer.analyzeColocate(properties);
- // ignore storage policy
- if (!PropertyAnalyzer.analyzeStoragePolicy(properties).isEmpty()) {
- properties.remove(PropertyAnalyzer.PROPERTIES_STORAGE_POLICY);
- }
- }
-
public void setReplicationAllocation(ReplicaAllocation replicaAlloc) {
getOrCreatTableProperty().setReplicaAlloc(replicaAlloc);
}
@@ -1799,6 +1769,17 @@ public class OlapTable extends Table {
return "";
}
+ public void setCcrEnable(boolean ccrEnable) throws UserException {
+ // TODO(Drogon): Config.enable_ccr
+ TableProperty tableProperty = getOrCreatTableProperty();
+
tableProperty.modifyTableProperties(PropertyAnalyzer.PROPERTIES_CCR_ENABLE,
Boolean.toString(ccrEnable));
+ tableProperty.buildCcrEnable();
+ }
+
+ public boolean isCcrEnable() {
+ return tableProperty != null && tableProperty.isCcrEnable();
+ }
+
public void setDisableAutoCompaction(boolean disableAutoCompaction) {
TableProperty tableProperty = getOrCreatTableProperty();
tableProperty.modifyTableProperties(PropertyAnalyzer.PROPERTIES_DISABLE_AUTO_COMPACTION,
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/RandomDistributionInfo.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/RandomDistributionInfo.java
index 1352f8af3f..3c63546ffc 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/RandomDistributionInfo.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/RandomDistributionInfo.java
@@ -53,9 +53,9 @@ public class RandomDistributionInfo extends DistributionInfo {
}
@Override
- public String toSql(boolean forSync) {
+ public String toSql() {
StringBuilder builder = new StringBuilder();
- if (autoBucket && !forSync) {
+ if (autoBucket) {
builder.append("DISTRIBUTED BY RANDOM BUCKETS AUTO");
} else {
builder.append("DISTRIBUTED BY RANDOM BUCKETS ").append(bucketNum);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
index bced77425b..550d190604 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
@@ -60,7 +60,7 @@ public class TableProperty implements Writable {
private boolean isInMemory = false;
private String storagePolicy = "";
- private Boolean isBeingSynced = null;
+ private Boolean ccrEnable = null;
private BinlogConfig binlogConfig;
private boolean isDynamicSchema = false;
@@ -112,7 +112,7 @@ public class TableProperty implements Writable {
case OperationType.OP_MODIFY_IN_MEMORY:
buildInMemory();
buildStoragePolicy();
- buildIsBeingSynced();
+ buildCcrEnable();
break;
default:
break;
@@ -223,27 +223,16 @@ public class TableProperty implements Writable {
return storagePolicy;
}
- public TableProperty buildIsBeingSynced() {
- isBeingSynced = Boolean.parseBoolean(properties.getOrDefault(
- PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED, "false"));
+ public TableProperty buildCcrEnable() {
+ ccrEnable =
Boolean.parseBoolean(properties.getOrDefault(PropertyAnalyzer.PROPERTIES_CCR_ENABLE,
"false"));
return this;
}
- public void setIsBeingSynced() {
- properties.put(PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED, "true");
- isBeingSynced = true;
- }
-
- public boolean isBeingSynced() {
- if (isBeingSynced == null) {
- buildIsBeingSynced();
+ public boolean isCcrEnable() {
+ if (ccrEnable == null) {
+ buildCcrEnable();
}
- return isBeingSynced;
- }
-
- public void removeInvalidProperties() {
- properties.remove(PropertyAnalyzer.PROPERTIES_STORAGE_POLICY);
- properties.remove(PropertyAnalyzer.PROPERTIES_COLOCATE_WITH);
+ return ccrEnable;
}
public TableProperty buildBinlogConfig() {
@@ -440,7 +429,7 @@ public class TableProperty implements Writable {
.buildDataSortInfo()
.buildCompressionType()
.buildStoragePolicy()
- .buildIsBeingSynced()
+ .buildCcrEnable()
.buildBinlogConfig()
.buildEnableLightSchemaChange()
.buildStoreRowColumn()
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
index a368e33a77..eac10bb2d0 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/clone/DynamicPartitionScheduler.java
@@ -501,7 +501,6 @@ public class DynamicPartitionScheduler extends MasterDaemon
{
olapTable = (OlapTable) db.getTableNullable(tableId);
// Only OlapTable has DynamicPartitionProperty
if (olapTable == null
- || olapTable.isBeingSynced()
|| !olapTable.dynamicPartitionExists()
||
!olapTable.getTableProperty().getDynamicPartitionProperty().getEnable()) {
iterator.remove();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java
index 274e1a3e15..54c0594482 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/common/util/DynamicPartitionUtil.java
@@ -670,7 +670,6 @@ public class DynamicPartitionUtil {
public static void checkAlterAllowed(OlapTable olapTable) throws
DdlException {
TableProperty tableProperty = olapTable.getTableProperty();
if (tableProperty != null &&
tableProperty.getDynamicPartitionProperty() != null
- && !tableProperty.isBeingSynced()
&& tableProperty.getDynamicPartitionProperty().isExist()
&& tableProperty.getDynamicPartitionProperty().getEnable()) {
throw new DdlException("Cannot add/drop partition on a Dynamic
Partition Table, "
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
index b80d511256..dd137077db 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
@@ -129,7 +129,7 @@ public class PropertyAnalyzer {
public static final String PROPERTIES_MUTABLE = "mutable";
- public static final String PROPERTIES_IS_BEING_SYNCED = "is_being_synced";
+ public static final String PROPERTIES_CCR_ENABLE = "ccr_enable";
// binlog.enable, binlog.ttl_seconds, binlog.max_bytes,
binlog.max_history_nums
public static final String PROPERTIES_BINLOG_PREFIX = "binlog.";
@@ -458,7 +458,7 @@ public class PropertyAnalyzer {
}
// analyze the colocation properties of table
- public static String analyzeColocate(Map<String, String> properties) {
+ public static String analyzeColocate(Map<String, String> properties)
throws AnalysisException {
String colocateGroup = null;
if (properties != null &&
properties.containsKey(PROPERTIES_COLOCATE_WITH)) {
colocateGroup = properties.get(PROPERTIES_COLOCATE_WITH);
@@ -667,7 +667,7 @@ public class PropertyAnalyzer {
return estimatePartitionSize;
}
- public static String analyzeStoragePolicy(Map<String, String> properties) {
+ public static String analyzeStoragePolicy(Map<String, String> properties)
throws AnalysisException {
String storagePolicy = "";
if (properties != null &&
properties.containsKey(PROPERTIES_STORAGE_POLICY)) {
storagePolicy = properties.get(PROPERTIES_STORAGE_POLICY);
@@ -815,14 +815,6 @@ public class PropertyAnalyzer {
return binlogConfigMap;
}
- public static boolean analyzeIsBeingSynced(Map<String, String> properties,
boolean defaultValue) {
- if (properties != null &&
properties.containsKey(PROPERTIES_IS_BEING_SYNCED)) {
- String value = properties.remove(PROPERTIES_IS_BEING_SYNCED);
- return Boolean.valueOf(value);
- }
- return defaultValue;
- }
-
// There are 2 kinds of replication property:
// 1. "replication_num" = "3"
// 2. "replication_allocation" = "tag.location.zone1: 2,
tag.location.zone2: 1"
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
index 28bac7a540..132f2cfa5f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java
@@ -1147,7 +1147,7 @@ public class InternalCatalog implements
CatalogIf<Database> {
}
Env.getDdlStmt(stmt, stmt.getDbName(), table, createTableStmt,
null, null, false, false, true, -1L,
- false, false);
+ false);
if (createTableStmt.isEmpty()) {
ErrorReport.reportDdlException(ErrorCode.ERROR_CREATE_TABLE_LIKE_EMPTY,
"CREATE");
}
@@ -2071,17 +2071,6 @@ public class InternalCatalog implements
CatalogIf<Database> {
}
olapTable.setIsInMemory(false);
- boolean isBeingSynced =
PropertyAnalyzer.analyzeIsBeingSynced(properties, false);
- olapTable.setIsBeingSynced(isBeingSynced);
- if (isBeingSynced) {
- // erase colocate table, storage policy
- olapTable.ignoreInvaildPropertiesWhenSynced(properties);
- // remark auto bucket
- if (isAutoBucket) {
- olapTable.markAutoBucket();
- }
- }
-
boolean storeRowColumn = false;
try {
storeRowColumn =
PropertyAnalyzer.analyzeStoreRowColumn(properties);
@@ -2326,6 +2315,7 @@ public class InternalCatalog implements
CatalogIf<Database> {
if
(DynamicPartitionUtil.checkDynamicPartitionPropertiesExist(properties)) {
throw new DdlException(
"Only support dynamic partition properties
on range partition table");
+
}
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
index 322dcdf500..59bfbada21 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
@@ -964,7 +964,7 @@ public class ShowExecutor {
}
List<String> createTableStmt = Lists.newArrayList();
Env.getDdlStmt(null, null, table, createTableStmt, null, null,
false,
- true /* hide password */, false, -1L,
showStmt.isNeedBriefDdl(), false);
+ true /* hide password */, false, -1L,
showStmt.isNeedBriefDdl());
if (createTableStmt.isEmpty()) {
resultSet = new ShowResultSet(showStmt.getMetaData(), rows);
return;
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
index 8b25aac5f9..93b4d7f687 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
@@ -2569,7 +2569,7 @@ public class FrontendServiceImpl implements
FrontendService.Iface {
Map<String, String> properties = request.getProperties();
RestoreStmt restoreStmt = new RestoreStmt(label, repoName, null,
properties, request.getMeta(),
request.getJobInfo());
- restoreStmt.setIsBeingSynced();
+ restoreStmt.disableDynamicPartition();
LOG.trace("restore snapshot info, restoreStmt: {}", restoreStmt);
try {
ConnectContext ctx = ConnectContext.get();
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/analysis/CreateTableAsSelectStmtTest.java
b/fe/fe-core/src/test/java/org/apache/doris/analysis/CreateTableAsSelectStmtTest.java
index 348c8571f6..2e031b1d36 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/analysis/CreateTableAsSelectStmtTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/analysis/CreateTableAsSelectStmtTest.java
@@ -92,7 +92,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -114,7 +113,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`_col0`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -132,7 +130,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`_col0`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -168,7 +165,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -193,7 +189,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`_col0`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -220,7 +215,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`_col0`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -244,7 +238,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`amount`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" = \"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -264,7 +257,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`alias_name`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -291,7 +283,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -315,7 +306,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId1`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -343,7 +333,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`user`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -368,7 +357,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -392,7 +380,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -412,7 +399,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`id`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" = \"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -439,7 +425,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -465,7 +450,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -490,7 +474,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`username`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -516,7 +499,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`userId`) BUCKETS 10\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -566,7 +548,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 1\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
@@ -584,7 +565,6 @@ public class CreateTableAsSelectStmtTest extends
TestWithFeService {
+ "DISTRIBUTED BY HASH(`k1`) BUCKETS 1\n"
+ "PROPERTIES (\n"
+ "\"replication_allocation\" =
\"tag.location.default: 1\",\n"
- + "\"is_being_synced\" = \"false\",\n"
+ "\"storage_format\" = \"V2\",\n"
+ "\"light_schema_change\" = \"true\",\n"
+ "\"disable_auto_compaction\" = \"false\",\n"
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java
b/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java
index 64e92b35c8..dcc76fdfcd 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java
@@ -243,7 +243,7 @@ public class RestoreJobTest {
db.dropTable(expectedRestoreTbl.getName());
job = new RestoreJob(label, "2018-01-01 01:01:01", db.getId(),
db.getFullName(), jobInfo, false,
- new ReplicaAllocation((short) 3), 100000, -1, false, false,
false, env, repo.getId());
+ new ReplicaAllocation((short) 3), 100000, -1, false, false,
env, repo.getId());
List<Table> tbls = Lists.newArrayList();
List<Resource> resources = Lists.newArrayList();
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
b/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
index aa85178c08..340161495d 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/OlapTableTest.java
@@ -92,7 +92,7 @@ public class OlapTableTest {
Assert.assertTrue(olapTable.getDefaultReplicaAllocation() ==
ReplicaAllocation.DEFAULT_ALLOCATION);
ReplicaAllocation replicaAlloc = new ReplicaAllocation((short) 4);
- olapTable.resetPropertiesForRestore(false, false, replicaAlloc, false);
+ olapTable.resetPropertiesForRestore(false, false, replicaAlloc);
Assert.assertEquals(tableProperty.getProperties(),
olapTable.getTableProperty().getProperties());
Assert.assertFalse(tableProperty.getDynamicPartitionProperty().isExist());
Assert.assertFalse(olapTable.isColocateTable());
@@ -112,7 +112,7 @@ public class OlapTableTest {
tableProperty = new TableProperty(properties);
olapTable.setTableProperty(tableProperty);
- olapTable.resetPropertiesForRestore(false, false,
ReplicaAllocation.DEFAULT_ALLOCATION, false);
+ olapTable.resetPropertiesForRestore(false, false,
ReplicaAllocation.DEFAULT_ALLOCATION);
Map<String, String> expectedProperties = Maps.newHashMap(properties);
expectedProperties.put(DynamicPartitionProperty.ENABLE, "false");
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
