This is an automated email from the ASF dual-hosted git repository.
w41ter pushed a commit to branch update_ccr_docs
in repository https://gitbox.apache.org/repos/asf/doris-website.git
commit 13e6c7dcd5f33027fa8f14535fc69cc6663eacd2
Author: w41ter <w41te...@gmail.com>
AuthorDate: Fri Dec 20 10:57:02 2024 +0000
Update ccr feature details
---
docs/admin-manual/data-admin/ccr/feature.md | 359 +++++++++++----------
.../current/admin-manual/data-admin/ccr/feature.md | 263 +++++++--------
2 files changed, 322 insertions(+), 300 deletions(-)
diff --git a/docs/admin-manual/data-admin/ccr/feature.md
b/docs/admin-manual/data-admin/ccr/feature.md
index 055ab4af88f..8b53953c048 100644
--- a/docs/admin-manual/data-admin/ccr/feature.md
+++ b/docs/admin-manual/data-admin/ccr/feature.md
@@ -5,13 +5,14 @@
}
---
-The Cross-Cluster Replication (CCR) feature in Doris is primarily designed to
efficiently synchronize data across multiple clusters, enhancing business
continuity and disaster recovery capabilities. CCR supports various operations
in Doris, ensuring data consistency across clusters.
+## Cross-Cluster Replication (CCR) in Doris
-Below is a detailed overview of the main Doris operations supported by CCR.
+The Cross-Cluster Replication (CCR) feature in Doris is primarily used to
efficiently synchronize data across multiple clusters, enhancing business
continuity and disaster recovery. CCR supports various operations in Doris,
ensuring data consistency across different clusters. Below are the details of
the main Doris operations supported by CCR.
:::note
-`-` Indicates Doris version 2.0 and above, all versions of CCR. It is
recommended to use Doris version 2.0.15 or 2.1.6 or newer.
+1. The `-` in Doris Version indicates Doris 2.0 or higher versions, supporting
all CCR versions. It is recommended to use Doris 2.0.15 or 2.1.6 or later
versions.
+2. CCR Syncer and Doris version requirements: Syncer Version >= downstream
Doris Version >= upstream Doris Version. Therefore, first upgrade Syncer, then
the downstream Doris, and finally the upstream Doris.
:::
@@ -19,228 +20,238 @@ Below is a detailed overview of the main Doris operations
supported by CCR.
### Database Properties
-Database-level tasks will synchronize the properties of the database during
Full Sync.
+Database-level tasks synchronize database properties during Full Sync.
-| Property | Supported | Doris version | CCR version | Sync
Method | Description |
-|------------------------|-----------|---------------|-------------|-------------|-------------|
-| replication_allocation | Supported | - | - | FULL
SYNC | |
-| data quota | Not supported | | |
| |
-| replica quota | Not supported | | |
| |
+| Property | Supported | Doris Version | Sync Method |
Description |
+| --------------------- | --------- | ------------- | ------------ |
----------- |
+| replication_allocation | Supported | - | Full Sync |
|
+| data quota | Not Supported | | |
|
+| replica quota | Not Supported | | |
|
-### Modify Database Properties
+### Modifying Database Properties
-CCR tasks do not synchronize operations that modify database properties.
+CCR tasks do not synchronize database property modification operations.
-| Property | Supported | Can upstream operate | Can downstream
operate | Description |
-|------------------------|-----------|----------------------|------------------------|------------------------------------------|
-| replication_allocation | Not supported | No | No
| Operations by upstream and downstream will cause CCR tasks to be
interrupted |
-| data quota | Not supported | Yes | Yes
| |
-| replica quota | Not supported | Yes | Yes
| |
+| Property | Supported | Upstream Can Modify | Downstream Can
Modify | Description |
+| --------------------- | --------- | ------------------ |
-------------------- | --------------------------------- |
+| replication_allocation | Not Supported | No | No
| Modifications cause CCR task failure |
+| data quota | Not Supported | Yes | Yes
| |
+| replica quota | Not Supported | Yes | Yes
| |
-### Rename Database
+### Renaming Database
-Renaming is not supported for upstream and downstream; doing so may cause
views to stop working.
+Renaming is not supported for upstream and downstream. Doing so might cause
views to stop working.
## Table
+
### Table Properties
-| Property | Supported | Doris version |
CCR version | Sync Method | Description
|
-|-----------------------------------------------|-----------|---------------|-------------|-------------|---------------------------------------------------------------|
-| Table Model (duplicate, unique, aggregate) | Supported | - |
- | SQL |
|
-| Partitioning and Bucketing | Supported | - |
- | SQL |
|
-| replication_num | Supported | - |
- | SQL |
|
-| replication_allocation (resource group) | Supported | - | -
| SQL | Upstream must be consistent with downstream; BE tags
must be consistent, otherwise CCR tasks will fail |
-| colocate_with | Not supported | |
| |
|
-| storage_policy | Not supported | |
| |
|
-| dynamic_partition | Supported | - |
- | SQL |
|
-| storage_medium | Supported | - |
- | SQL |
|
-| auto_bucket | Supported | - |
- | SQL |
|
-| group_commit series | Supported | - |
- | SQL |
|
-| enable_unique_key_merge_on_write | Supported | - |
- | SQL |
|
-| enable_single_replica_compaction | Supported | - |
- | SQL |
|
-| disable_auto_compaction | Supported | - |
- | SQL |
|
-| compaction_policy | Supported | - |
- | SQL |
|
-| time_series_compaction series | Supported | - |
- | SQL |
|
-| binlog series | Supported | - |
- | SQL | To be confirmed
|
-| variant_enable_flatten_nested | Supported | - |
- | SQL |
|
-| skip_write_index_on_load | Supported | - |
- | SQL |
|
-| row_store series | Supported | - |
- | SQL |
|
-| seq column | Supported | - |
- | SQL |
|
-| enable_light_schema_change | Supported | - |
- | SQL |
|
-| compression_type | Supported | - |
- | SQL |
|
-| index | Supported | - |
- | SQL |
|
-| bloom_filter_columns | Supported | - |
- | SQL |
|
-| bloom_filter_fpp | Not supported | |
| |
|
-| storage_cooldown_time | Not supported | |
| |
|
-| generated column | Supported | - |
- | SQL |
|
-| auto-increment id | Not supported | |
| | Issues exist
|
+| Property | Supported | Doris Version | Sync
Method | Description |
+| ------------------------------------ | --------- | ------------- |
----------- | ----------- |
+| Table model (duplicate, unique, aggregate) | Supported | - | SQL
| |
+| Partition and bucket | Supported | - | SQL
| |
+| replication_num | Supported | - | SQL
| |
+| replication_allocation (resource group) | Supported | - | SQL
| Upstream and downstream must match; BE tags must match, otherwise the
CCR task will fail |
+| colocate_with | Not Supported | |
| |
+| storage_policy | Not Supported | |
| |
+| dynamic_partition | Supported | - | SQL
| |
+| storage_medium | Supported | - | SQL
| |
+| auto_bucket | Supported | - | SQL
| |
+| group_commit series | Supported | - | SQL
| |
+| enable_unique_key_merge_on_write | Supported | - | SQL
| |
+| enable_single_replica_compaction | Supported | - | SQL
| |
+| disable_auto_compaction | Supported | - | SQL
| |
+| compaction_policy | Supported | - | SQL
| |
+| time_series_compaction series | Supported | - | SQL
| |
+| binlog series | Supported | - | SQL
| |
+| variant_enable_flatten_nested | Supported | - | SQL
| |
+| skip_write_index_on_load | Supported | - | SQL
| |
+| row_store series | Supported | - | SQL
| |
+| seq column | Supported | - | SQL
| |
+| enable_light_schema_change | Supported | - | SQL
| |
+| compression_type | Supported | - | SQL
| |
+| index | Supported | - | SQL
| |
+| bloom_filter_columns | Supported | - | SQL
| |
+| bloom_filter_fpp | Supported | |
| |
+| storage_cooldown_time | Not Supported | |
| |
+| generated column | Supported | - | SQL
| |
+| Auto-increment ID | Not Supported | |
| Issues with this feature |
### Basic Table Operations
-| Operation | Supported | Doris version
| CCR version | Sync Method | Can downstream operate
separately | Description |
-|-----------------|---------------------------------|-------------------------|-------------|---------------------------------------|-----------------------------------|----------------------------------------------|
-| create table | Supported | -
| - | SQL | Not supported for
tables synchronized by CCR tasks. | Reference properties for creating tables
|
-| drop table | Supported | -
| - | Before 2.0.15/2.1.6: Full Sync, after: SQL | Same as above
| |
-| rename table | Table-level tasks do not support database-level tasks |
master (2.0/2.1 not supported) | | SQL
| Same as above | Table-level task rename will cause
CCR tasks to stop |
-| replace table | Not supported |
| | | Same as above
| |
-| truncate table | Supported | -
| | SQL | Same as above
| |
-| restore table | Unknown |
| | | Same as above
| |
+| Operation | Supported | Doris Version | Sync Method | Downstream Can
Operate Independently | Description |
+| --------------- | --------- | ------------- | ----------- |
----------------------------------- | ----------- |
+| create table | Supported | - | SQL/Partial Sync | No
| Table creation synced by SQL or partial sync
depending on certain settings |
+| drop table | Supported | - | SQL/Full Sync | No
| 2.0.15/2.1.6 and earlier use Full Sync, later use SQL |
+| rename table | Not Supported | 2.1.8/3.0.4 | SQL | No
| Renaming will stop the CCR task |
+| replace table | Supported | 2.1.8/3.0.4 | SQL/Full Sync | No
| Full Sync triggered for table-level tasks |
+| truncate table | Supported | - | SQL | No
| |
+| restore table | Not Supported | | |
| |
-### Modify Table Properties
+### Modifying Table Properties
Sync method is SQL.
-| Property | Supported | Doris version | CCR version |
Can upstream operate | Can downstream operate |
Description |
-|-------------------------------|-----------|---------------|-------------|----------------------|-------------------------------------------------|------------------------------------------------|
-| colocate | Not supported | | |
Yes | No |
|
-| distribution type | Not supported | | |
No | Same as above |
|
-| dynamic partition | Not supported | | |
Yes | Same as above |
|
-| replication_num | Not supported | | |
No | No |
|
-| replication_allocation | Not supported | | |
No | |
|
-| storage policy | Not supported | | |
No | No |
|
-| enable_light_schema_change | Not supported | | |
| | CCR can
only synchronize lightweight schema changes of tables. |
-| row_store | Unknown | | |
| |
|
-| bloom_filter_columns | Unknown | | |
| |
|
-| bucket num | Not supported | | |
Yes | No |
|
-| isBeingSyced | Not supported | | |
No | No |
|
-| compaction series properties | Not supported | | |
Yes | No, triggering full sync downstream operations will
cause data loss | |
-| skip_write_index_on_load | Not supported | | |
Yes | Same as above |
|
-| seq column | Supported | - | - |
Yes | No, triggering full sync downstream operations will
cause data loss | |
-| delete sign column | Supported | - | - |
Yes | Same as above |
|
-| comment | Not supported | | |
Yes | No, triggering full sync downstream operations will
cause data loss | |
+| Property | Supported | Doris Version | Upstream Can Modify |
Downstream Can Modify | Description |
+| ----------------------- | --------- | ------------- | ------------------ |
--------------------- | ---------------------------------------- |
+| colocate | Not Supported | | Yes |
No | Triggering Full Sync causes data loss at downstream |
+| distribution type | Not Supported | | No |
No | |
+| dynamic partition | Not Supported | | Yes |
No | |
+| replication_num | Not Supported | | No |
No | |
+| replication_allocation | Not Supported | | No |
| |
+| storage policy | Not Supported | | No |
No | |
+| enable_light_schema_change | Not Supported | |
| | CCR can only sync light schema changes |
+| row_store | Supported | 2.1.8/3.0.4 | |
| Through Partial Sync |
+| bloom_filter_columns | Supported | 2.1.8/3.0.4 | |
| Through Partial Sync |
+| bloom_filter_fpp | Supported | 2.1.8/3.0.4 | |
| Through Partial Sync |
+| bucket num | Not Supported | | Yes |
No | |
+| isBeingSynced | Not Supported | | No |
No | |
+| compaction series | Not Supported | | Yes |
No | |
+| skip_write_index_on_load | Not Supported | | Yes
| No | |
+| seq column | Supported | - | Yes |
No | |
+| delete sign column | Supported | - | Yes |
No | |
+| comment | Supported | 2.1.8/3.0.4 | Yes | No
| |
### Column Operations
-Column operations on Base Index in the table.
+Base Index Column Operations
+
+| Operation | Supported | Doris Version | Sync Method |
Downstream Can Operate Independently | Description |
+| --------------------- | --------- | ------------- | ---------------- |
----------------------------------- | ----------- |
+| add key column | Supported | - | Partial Sync | No
| |
+| add value column | Supported | - | SQL | No
| |
+| drop key column | Supported | - | Partial Sync | No
| |
+| drop value column | Supported | - | SQL | No
| |
+| modify column | Supported | - | Partial Sync | No
| |
+| order by | Supported | - | Partial Sync | No
| |
+| rename | Supported | 2.1.8/3.0.4 | SQL | No
| |
+| comment | Supported | 2.1.8/3.0.4 | SQL | No
| |
-| Operation | Supported | Doris version | CCR version | Sync Method
| Can downstream operate | Remarks
|
-|-------------------|-----------|---------------|-------------|-----------------------------|-----------------------------------|-----------------------------------|
-| add key column | Supported | - | - |
Database-level task Partial Sync, Table-level task Partial Sync | No
| |
-| add value column | Supported | - | - | SQL
| No |
|
-| drop key column | Supported | - | - |
Database-level task Partial Sync, Table-level task Partial Sync | Same as above
| |
-| drop value column | Supported | - | - | SQL
| Same as above |
|
-| modify column | Supported | - | - | Full Sync /
Partial Sync | Same as above | Will attempt to delete
the downstream table before starting |
-| order by | Supported | - | - | Full Sync /
Partial Sync | Same as above | Will attempt to delete
the downstream table before starting |
-| rename | | | |
| Same as above |
|
-| comment | Not supported | | |
| Same as above |
|
+:::note
+`add/drop value column` requires setting `"light_schema_change" = "true"`
during table creation.
-Column operations on Rollup Index in the table.
+:::
-| Operation | Supported | Remarks |
-|-------------------|-----------|---------------------|
-| add column | Unknown | Causes CCR task interruption |
-| drop column | Unknown | Same as above |
-| modify column | Unknown | Same as above |
-| order by | Unknown | Same as above |
+Rollup Index Column Operations
+| Operation | Supported | Doris Version | Sync Method |
Description |
+| --------------------- | --------- | ------------- | -------------- |
----------- |
+| add key column | Supported | 2.1.8/3.0.4 | Partial Sync |
|
+| add value column | Supported | 2.1.8/3.0.4 | SQL | Needs
light schema change enabled |
+| drop column | Supported | 2.1.8/3.0.4 | Partial Sync |
|
+| modify column | Unknown | 2.1.8/3.0.4 | Partial Sync | Doris
does not support directly modifying rollup column types |
+| order by | Supported | 2.1.8/3.0.4 | Partial Sync |
|
-### Rollup
+### Rollup Operations
-| Operation | Supported | Remarks |
-|-------------------|-----------|------------------|
-| add rollup | Not supported | |
-| drop rollup | Not supported | |
-| rename rollup | Not supported | CCR task interruption |
+| Operation | Supported | Doris Version | Sync Method |
Description |
+| --------------------- | --------- | ------------- | -------------- |
----------- |
+| add rollup | Supported | 2.1.8/3.0.4
-### Index
+ | Partial Sync | |
+| drop rollup | Supported | 2.1.8/3.0.4 | SQL |
|
+| rename rollup | Supported | 2.1.8/3.0.4 | SQL |
|
+### Index Operations
Inverted Index
-| Operation | Supported | Remarks |
-|-------------------|----------------------|-------------|
-| create index | Not supported | Unknown impact |
-| drop index | Unknown | Unknown impact |
-
+| Operation | Supported | Doris Version | Sync Method |
Description |
+| --------------------- | --------- | ------------- | -------------- |
----------- |
+| create index | Supported | 2.1.8/3.0.4 | Partial Sync |
|
+| drop index | Supported | 2.1.8/3.0.4 | SQL |
|
+| build index | Supported | 2.1.8/3.0.4 | SQL |
|
Bloom Filter
-| Operation | Supported | Remarks |
-|-------------------|----------------------|-------------|
-| add bloom filter | Not supported | Unknown impact |
-| alter bloom filter | Not supported | Unknown impact |
-| drop bloom filter | Not supported | Unknown impact |
+| Operation | Supported | Doris Version | Sync Method |
Description |
+| --------------------- | --------- | ------------- | -------------- |
----------- |
+| add bloom filter | Supported | 2.1.8/3.0.4 | Partial Sync |
|
+| alter bloom filter | Supported | 2.1.8/3.0.4 | Partial Sync | Refers
to modifying `bloom_filter_columns` |
+| drop bloom filter | Supported | 2.1.8/3.0.4 | Partial Sync |
|
## Data
-### Import
+### Import Operations
-| Import Method | Supported | Doris version | CCR version |
Sync Method | Can downstream operate
| Description |
-|-------------------|----------------------|---------------|-------------|-------------|-------------------------------------------------------------------|-------------------------------------------------------------|
-| stream load | Supported (except for temporary partitions) | -
| - | TXN | No, if downstream imports, subsequent triggers
of full or Partial Sync will cause data loss | Upstream transactions are
visible, i.e., data is visible when generating binlog, downstream starts
syncing. |
-| broker load | Supported (except for temporary partitions) | -
| - | TXN | Same as above
| Same as above |
-| routine load | Supported (except for temporary partitions) | -
| - | TXN | Same as above
| Same as above |
-| mysql load | Supported (except for temporary partitions) | -
| - | TXN | Same as above
| Same as above |
-| group commit | Supported (except for temporary partitions) | 2.1
| 2.1 | TXN | Same as above
| Same as above |
+| Import Method | Supported | Doris Version | Sync Method | Downstream
Can Operate Independently | Description |
+| -------------------- | --------- | ------------- | ----------- |
----------------------------------- | ----------- |
+| stream load | Supported (except temp partitions) | - | TXN | No |
Data is visible when the upstream transaction is committed |
+| broker load | Supported (except temp partitions) | - | TXN | No |
Same as stream load |
+| routine load | Supported (except temp partitions) | - | TXN | No |
Same as stream load |
+| mysql load | Supported (except temp partitions) | - | TXN | No |
Same as stream load |
+| group commit | Supported (except temp partitions) | 2.1 | TXN | No |
Same as stream load |
### Data Operations
-| Operation | Supported | Doris version | CCR
version | Sync Method | Can downstream operate
| Description |
-|-------------------------------|----------------------|---------------|-------------|------------------|-------------------------------------------------------------------|-------------------------------------------------------------|
-| delete | Supported | - | -
| TXN | No, if downstream operates, subsequent triggers of
full or Partial Sync will cause data loss | Upstream transactions are visible,
i.e., data is visible when generating binlog, downstream starts syncing. |
-| update | Supported | - | -
| TXN | Same as above
| Same as above |
-| insert | Supported | - | -
| TXN | Same as above
| Same as above |
-| insert into overwrite | Supported (except for temporary partitions)
| 2.1.6 | | Partial Sync | Same as above
| Same as above
|
-| insert into overwrite | Supported (except for temporary partitions)
| 2.0 | | full sync | Same as above
| Same as above
|
-| Explicit transaction (3.0) begin commit | Not supported | |
| |
| |
-
-## Partition Operations
+| Operation | Supported | Doris Version | Sync Method | Downstream Can
Operate Independently | Description |
+| ---------------- | --------- | ------------- | ----------- |
----------------------------------- | ----------- |
+| delete | Supported | - | TXN | No
| Data is visible in binlog when committed, downstream
starts syncing |
+| update | Supported | - | TXN | No
| Same as delete |
+| insert | Supported | - | TXN | No
| Same as delete |
+| insert into overwrite | Supported (except temp partitions) | 2.1.6 | Partial
Sync | No | Same as insert |
+| insert into overwrite | Supported (except temp partitions) | 2.0 | Full Sync
| No | Same as insert |
+
+### Partition Operations
+
+| Operation | Supported | Doris Version | Sync Method | Downstream Can
Operate Independently | Description |
+| ---------------- | --------- | ------------- | ----------- |
----------------------------------- | ----------- |
+| add partition | Supported | - | SQL | No
| Triggers Full Sync or Partial Sync, causing data loss in
downstream |
+| add temp partition | Not Supported | | | No
| Temporary partitions not supported for backup;
workaround in newer versions |
+| drop partition | Supported | - | SQL/Full Sync | No
| 2.0.15/2.1.6 and earlier use Full Sync, later use SQL |
+| replace partition | Supported | 2.1.7/3.0.3 | Partial Sync | No
| Only supports strict range and non-tmp partitions for
replace |
+| modify partition | Not Supported | | | No
| Modifying partition properties is not supported |
+| rename partition | Supported | 2.1.8/3.0.4 | SQL | No
| |
+
+### View Operations
+
+| Operation | Supported | Doris Version | Sync Method | Description |
+| --------------- | --------- | ------------- | ----------- | ----------- |
+| create view | Supported | - | SQL | Works if
upstream and downstream names match |
+| alter view | Supported | 2.1.8/3.0.4 | SQL | |
+| drop view | Supported | 2.1.8/3.0.4 | SQL | |
-| Operation | Supported | Doris version |
CCR version | Sync Method | Can downstream operate separately
| Description
|
-|------------------------|---------------------------------|---------------|-------------|------------------------|-------------------------------------------------------------------------|---------------------------------------------------------------------|
-| add partition | Supported | - | -
| SQL | No, subsequent triggers of Full Sync or
Partial Sync will cause downstream operations to be lost | cooldown time
property and its behavior unknown |
-| add temp partition | Not supported | |
| | Same as above
|
|
-| drop partition | Supported | - | -
| Before 2.0.15/2.1.6: Full Sync, after: SQL | Same as above
|
|
-| replace partition | Supported | |
| Partial Sync | Same as above
| Partial Sync **only supports strict range and
non-temp partition replace method**, otherwise it will trigger Full Sync. |
-| modify partition | Not supported | Not released |
Not released | SQL | Same as above
|
|
-| rename partition | Table-level tasks do not support database-level
tasks | Not released | Not released | SQL | Same as above
| Table-level task
rename will cause CCR tasks to stop |
-
-## View
-
-| Operation | Supported | Doris version | CCR version | Sync Method |
Remarks |
-|------------------|-----------|---------------|-------------|-------------|-------------------------------------|
-| create view | Supported | - | - | SQL |
Can work when upstream and downstream have the same name. |
-| alter view | Not supported | | | |
No binlog |
-| drop view | Not supported | | | |
|
+:::note
+Due to Doris limitations, column names or view names cannot match the database
name.
-## Materialized View
+:::
-Synchronous Materialized View
+### Materialized Views
-| Operation | Supported | Remarks
|
-|-------------------------------|-----------|----------------------------------------------------------------|
-| create materialized view | Unknown | Can work when upstream and
downstream have the same name; if different names, downstream needs to manually
recreate the view. |
-| drop materialized view | Unknown |
|
+Synchronized Materialized View
+| Operation | Supported | Doris Version | Sync Method |
Description |
+| ----------------------- | --------- | ------------- | -------------- |
----------- |
+| create materialized view | Supported | 2.1.8/3.0.4 | Partial Sync | Same
name at upstream and downstream works; different names require manual rebuild
at downstream |
+| drop materialized view | Supported | 2.1.8/3.0.4 | SQL |
|
Asynchronous Materialized View
-| Operation | Supported |
-|-------------------------------------|-----------|
-| create async materialized view | Not supported |
-| alter async materialized view | Not supported |
-| drop async materialized view | Not supported |
-| refresh | Not supported |
-| pause | Not supported |
-| resume | Not supported |
-
+| Operation | Supported |
+| ------------------------ | --------- |
+| create async materialized view | Not Supported |
+| alter async materialized view | Not Supported |
+| drop async materialized view | Not Supported |
+| refresh | Not Supported |
+| pause | Not Supported |
+| resume | Not Supported |
## Statistics
-Not synchronized between upstream and downstream, work independently.
-
-## Others
-
-| Operation | Supported |
-|-----------------------|-----------|
-| external table | Not supported |
-| recycle bin | Not supported |
-| catalog | Not supported |
-| workload group | Not supported |
-| job | Not supported |
-| function | Not supported |
-| policy | Not supported |
-| user | Not supported |
-| cancel alter job | Supported |
\ No newline at end of file
+Statistics are not synchronized between upstream and downstream, and work
independently.
+
+## Other Operations
+
+| Operation | Supported |
+| ---------------- | --------- |
+| external table | Not Supported |
+| recycle bin | Not Supported |
+| catalog | Not Supported |
+| workload group | Not Supported |
+| job | Not Supported |
+| function | Not Supported |
+| policy | Not Supported |
+| user | Not Supported |
+| cancel alter job | Supported |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/ccr/feature.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/ccr/feature.md
index 08d3e25dc5d..78d6bd9262e 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/ccr/feature.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/ccr/feature.md
@@ -28,7 +28,8 @@ Doris 中的跨集群复制 (CCR) 功能，主要用于在多个集群之间高
:::note
-`-` 表示 Doris 2.0 及以上版本，CCR 所有版本。建议使用 Doris 使用 2.0.15 或者 2.1.6 或者更新的版本。
+1. Doris Version 中的 `-` 表示 Doris 2.0 及以上版本，CCR 所有版本。建议使用 Doris 使用 2.0.15 或者
2.1.6 或者更新的版本。
+2. CCR Syncer 和 Doris 的版本要求: Syncer Version >= 下游 Doris Version >= 上游 Doris
Version。因此升级前先升 Syncer，再升下游 Doris，最后升上游 Doris。
:::
@@ -38,11 +39,11 @@ Doris 中的跨集群复制 (CCR) 功能，主要用于在多个集群之间高
库级别任务在 Full Sync 时会同步库的属性。
-| 属性 | 是否支持 | Doris version | CCR version | 同步方式 | 说明 |
-| ---------------------- | -------- | ------------- | ----------- | ---------
| ---- |
-| replication_allocation | 支持 | - | - | FULL SYNC |
|
-| data quota | 不支持 | | | |
|
-| replica quota | 不支持 | | | |
|
+| 属性 | 是否支持 | Doris version | 同步方式 | 说明 |
+| ---------------------- | -------- | ------------- | --------- | ---- |
+| replication_allocation | 支持 | - | Full Sync | |
+| data quota | 不支持 | | | |
+| replica quota | 不支持 | | | |
### 修改库属性
@@ -61,175 +62,185 @@ CCR 任务不同步修改库属性操作。
## 表
### 表属性
-| 属性 | 是否支持 | Doris version | CCR version
| 同步方式 | 说明 |
-| ----------------------------------------- | -------- | ------------- |
----------- | -------- |
-------------------------------------------------------- |
-| 表模型（duplicate，unique，aggregate） | 支持 | - | -
| SQL | |
-| 分区分桶 | 支持 | - | -
| SQL | |
-| replication_num | 支持 | - | -
| SQL | |
-| replication_allocation （resource group） | 支持 | - | -
| SQL | 上游必须与下游一致，BE tag 必须一致，否则 CCR 任务会失败 |
+| 属性 | 是否支持 | Doris version | 同步方式 | 说明
|
+| ----------------------------------------- | -------- | ------------- |
-------- | -------------------------------------------------------- |
+| 表模型（duplicate，unique，aggregate） | 支持 | - | SQL
| |
+| 分区分桶 | 支持 | - | SQL |
|
+| replication_num | 支持 | - | SQL
| |
+| replication_allocation （resource group） | 支持 | - | SQL
| 上游必须与下游一致，BE tag 必须一致，否则 CCR 任务会失败 |
| colocate_with | 不支持 | |
| | |
| storage_policy | 不支持 | |
| | |
-| dynamic_partition | 支持 | - | -
| SQL | |
-| storage_medium | 支持 | - | -
| SQL | |
-| auto_bucket | 支持 | - | -
| SQL | |
-| group_commit 系列 | 支持 | - | -
| SQL | |
-| enable_unique_key_merge_on_write | 支持 | - | -
| SQL | |
-| enable_single_replica_compaction | 支持 | - | -
| SQL | |
-| disable_auto_compaction | 支持 | - | -
| SQL | |
-| compaction_policy | 支持 | - | -
| SQL | |
-| time_series_compaction 系列 | 支持 | - | -
| SQL | |
-| binlog 系列 | 支持 | - | -
| SQL | 待确认 |
-| variant_enable_flatten_nested | 支持 | - | -
| SQL | |
-| skip_write_index_on_load | 支持 | - | -
| SQL | |
-| row_strore 系列 | 支持 | - | -
| SQL | |
-| seq 列 | 支持 | - | -
| SQL | |
-| enable_light_schema_change | 支持 | - | -
| SQL | |
-| compression_type | 支持 | - | -
| SQL | |
-| index | 支持 | - | -
| SQL | |
-| bloom_filter_columns | 支持 | - | -
| SQL | |
-| bloom_filter_fpp | 不支持 | |
| | |
-| storage_cooldown_time | 不支持 | |
| | |
-| generated column | 支持 | - | -
| SQL | |
-| 自增 id | 不支持 | |
| | 有问题 |
+| dynamic_partition | 支持 | - | SQL
| |
+| storage_medium | 支持 | - | SQL
| |
+| auto_bucket | 支持 | - | SQL
| |
+| group_commit 系列 | 支持 | - | SQL
| |
+| enable_unique_key_merge_on_write | 支持 | - | SQL
| |
+| enable_single_replica_compaction | 支持 | - | SQL
| |
+| disable_auto_compaction | 支持 | - | SQL
| |
+| compaction_policy | 支持 | - | SQL
| |
+| time_series_compaction 系列 | 支持 | - | SQL
| |
+| binlog 系列 | 支持 | - | SQL
| |
+| variant_enable_flatten_nested | 支持 | - | SQL
| |
+| skip_write_index_on_load | 支持 | - | SQL
| |
+| row_strore 系列 | 支持 | - | SQL
| |
+| seq 列 | 支持 | - | SQL
| |
+| enable_light_schema_change | 支持 | - | SQL
| |
+| compression_type | 支持 | - | SQL
| |
+| index | 支持 | - | SQL
| |
+| bloom_filter_columns | 支持 | - | SQL
| |
+| bloom_filter_fpp | 支持 | |
| |
+| storage_cooldown_time | 不支持 | |
| |
+| generated column | 支持 | - | SQL
| |
+| 自增 id | 不支持 | | | 有问题
|
### 基础表操作
-| 操作 | 是否支持 | Doris version | CCR
version | 同步方式 | 下游是否可以单独操作 | 说明
|
-| -------------- | ------------------------------- | ----------------------- |
----------- | ---------------------------------- |
----------------------------- | ------------------------------------- |
-| create table | 支持 | - | -
| SQL | 不支持操作 CCR 任务同步的表。 | 属性参考创建表部分
|
-| drop table | 支持 | - | -
| 2.0.15/2.1.6 前：Full Sync，之后：SQL | 同上 |
|
-| rename table | 表级别任务不支持库级别任务支持 | master(2.0/2.1还不支持) | | SQL
| 同上 | 表级别任务 rename 会导致
CCR 任务停止 |
-| replace table | 不支持 | |
| | 同上 |
|
-| truncate table | 支持 | - |
| SQL | 同上 |
|
-| restore table | 未知 | |
| | 同上 |
|
+| 操作 | 是否支持 | Doris version | 同步方式 | 下游是否可以单独操作 | 说明 |
+| -------------- | ------------------------------- | ----------------------- |
---------------------------------- | ----------------------------- |
------------------------------------- |
+| create table | 支持 | - | SQL/Partial Sync | 不支持操作 CCR 任务同步的表。 |
属性参考创建表部分；大部分情况下使用 SQL 同步；部分操作，比如用户建表时设置了打开了某些 session
variables，或建表语句中有倒排索引，则使用部分同步 |
+| drop table | 支持 | - | SQL/Full Sync | 同上 | 2.0.15/2.1.6 前：Full
Sync，之后：SQL |
+| rename table | 表级别任务不支持库级别任务支持 | 2.1.8/3.0.4 | SQL | 同上 | 表级别任务 rename 会导致
CCR 任务停止 |
+| replace table | 支持 | 2.1.8/3.0.4 | SQL/Full Sync| 同上 | DB 级别使用 SQL
同步；表级别触发全量同步 |
+| truncate table | 支持 | - | SQL | 同上 | |
+| restore table | 不支持 | | | 同上 | |
### 修改表属性
同步方式为 SQL。
-| 属性 | 是否支持 | Doris version | CCR version | 上游是否可以操作 |
下游是否可以操作 | 说明 |
-| -------------------------- | -------- | ------------- | ----------- |
---------------- | ------------------------------------------ |
--------------------------------------- |
-| colocate | 不支持 | | | 可以
| 不可以，触发full sync 下游操作会丢失 | |
-| distribution type | 不支持 | | | 不可以
| 同上 |
|
-| dynamic partition | 不支持 | | | 可以
| 同上 |
|
-| replication_num | 不支持 | | | 不可以
| 不可以 |
|
-| replication_allocation | 不支持 | | | 不可以
| |
|
-| storage policy | 不支持 | | | 不可以
| 不可以 |
|
-| enable_light_schema_change | 不支持 | | |
| | CCR 只能同步轻量级 schema change
的表。 |
-| row_store | 未知 | | |
| |
|
-| bloom_filter_columns | 未知 | | |
| |
|
-| bucket num | 不支持 | | | 可以
| 不可以，触发full sync 下游操作会丢失 | |
-| isBeingSyced | 不支持 | | | 不可以
| 不可以 |
|
-| compaction 系列属性 | 不支持 | | | 可以
| 不可以，触发full sync 下游操作会丢失不可以 | |
-| skip_write_index_on_load | 不支持 | | | 可以
| 同上 |
|
-| seq 列 | 支持 | - | - | 可以
| 不可以，触发full sync 下游操作会丢失不可以 | |
-| delete sign 列 | 支持 | - | - | 可以
| 同上 |
|
-| comment | 不支持 | | | 可以
| 不可以，触发full sync 下游操作会丢失不可以 | |
+| 属性 | 是否支持 | Doris version | 上游是否可以操作 | 下游是否可以操作
| 说明 |
+| -------------------------- | -------- | ------------- | ---------------- |
------------------------------------------ |
--------------------------------------- |
+| colocate | 不支持 | | 可以 |
不可以，触发full sync 下游操作会丢失 | |
+| distribution type | 不支持 | | 不可以 | 同上
| |
+| dynamic partition | 不支持 | | 可以 | 同上
| |
+| replication_num | 不支持 | | 不可以 | 不可以
| |
+| replication_allocation | 不支持 | | 不可以 |
| |
+| storage policy | 不支持 | | 不可以 | 不可以
| |
+| enable_light_schema_change | 不支持 | | |
| CCR 只能同步轻量级 schema change 的表。 |
+| row_store | 支持 | 2.1.8/3.0.4 | |
| 通过 Partial Sync |
+| bloom_filter_columns | 支持 | 2.1.8/3.0.4 | |
| 通过 Partial Sync |
+| bloom_filter_fpp | 支持 | 2.1.8/3.0.4 | |
| 通过 Partial Sync |
+| bucket num | 不支持 | | 可以 |
不可以，触发full sync 下游操作会丢失 | |
+| isBeingSynced | 不支持 | | 不可以 | 不可以
| |
+| compaction 系列属性 | 不支持 | | 可以 | 不可以，触发full
sync 下游操作会丢失不可以 | |
+| skip_write_index_on_load | 不支持 | | 可以 | 同上
| |
+| seq 列 | 支持 | - | 可以 |
不可以，触发full sync 下游操作会丢失不可以 | |
+| delete sign 列 | 支持 | - | 可以 | 同上
| |
+| comment | 支持 | 2.1.8/3.0.4 | 可以 |
不可以，触发full sync 下游操作会丢失不可以 | |
### 列操作
表中 Base Index 上的列操作。
-| 操作 | 是否支持 | Doris version | CCR version | 同步方式 |
下游是否可以操作 | 备注 |
-| ------------- | -------- | ------------- | ----------- |
----------------------- | --------------------------- |
------------------------------- |
-| add key column | 支持 | - | - | 库级别任务 Partial
Sync, 表级别任务 Partial Sync | 不可以，会导致 CCR 任务中断 |
|
-| add value column | 支持 | - | - | SQL |
不可以，会导致 CCR 任务中断 | |
-| drop key column | 支持 | - | - | 库级别任务 Partial
Sync, 表级别任务 Partial Sync | 同上 |
|
-| drop value column | 支持 | - | - | SQL
| 同上 | |
-| modify column | 支持 | - | - | Full Sync / Partial
Sync | 同上 | 在开始前会尝试删除下游的表 |
-| order by | 支持 | - | - | Full Sync / Partial
Sync | 同上 | 在开始前会尝试删除下游的表 |
-| rename | | | |
| 同上 | |
-| comment | 不支持 | | |
| 同上 | |
+| 操作 | 是否支持 | Doris version | 同步方式 | 下游是否可以操作
| 备注 |
+| ------------- | -------- | ------------- | ----------------------- |
--------------------------- | ------------------------------- |
+| add key column | 支持 | - | Partial Sync |
不可以，会导致 CCR 任务中断 | |
+| add value column | 支持 | - | SQL | 不可以，会导致 CCR 任务中断 |
|
+| drop key column | 支持 | - | Partial Sync | 同上
| |
+| drop value column | 支持 | - | SQL | 同上
| |
+| modify column | 支持 | - | Partial Sync | 同上
| |
+| order by | 支持 | - | Partial Sync | 同上
| |
+| rename | 支持 | 2.1.8/3.0.4 | SQL | 同上 |
|
+| comment | 支持 | 2.1.8/3.0.4 | SQL | 同上 |
|
+:::note
-表中 Rollup Index 上的列操作。
+add/drop value column 要求建表时设置 property `"light_schema_change" = "true"`。
-| 操作 | 是否支持 | 备注 |
-| ------------- | -------- | ----------------- |
-| add column | 未知 | 导致 CCR 任务中断 |
-| drop column | 未知 | 同上 |
-| modify column | 未知 | 同上 |
-| order by | 未知 | 同上 |
+:::
+
+表中 Rollup Index 上的列操作。
+| 操作 | 是否支持 | Doris Version | 同步方式 | 备注 |
+| ------------- | -------- | ------------- | ---------- | ----------------- |
+| add key column | 支持 | 2.1.8/3.0.4 | Partial Sync | |
+| add value column | 支持 | 2.1.8/3.0.4 | SQL | 需要开启 lightning schema
change |
+| drop column | 支持 | 2.1.8/3.0.4 | Partial Sync | |
+| modify column | 未知 | 2.1.8/3.0.4 | Partial Sync | Doris 不支持直接修改 rollup
column 类型 |
+| order by | 支持 | 2.1.8/3.0.4 | Partial sync | |
### Rollup
-| 操作 | 是否支持 | 备注 |
-| ------------- | -------- | ------------ |
-| add rollup | 不支持 | |
-| drop rollup | 不支持 | |
-| rename rollup | 不支持 | CCR 任务中断 |
+| 操作 | 是否支持 | Doris Version | 同步方式 | 备注 |
+| ------------- | -------- | ------------- | ------- | ------------ |
+| add rollup | 支持 | 2.1.8/3.0.4 | Partial Sync | |
+| drop rollup | 支持 | 2.1.8/3.0.4 | SQL | |
+| rename rollup | 支持 | 2.1.8/3.0.4 | SQL | |
### 索引
-
Inverted Index
-| 操作 | 是否支持 | 备注 |
-| ------------ | -------------------- | -------- |
-| create index | 不支持 | 影响未知 |
-| drop index | 未知 | 影响未知 |
-
+| 操作 | 是否支持 | Doris Version | 同步方式 | 备注 |
+| ------------ | -------- | ------------- | -------- | --------- |
+| create index | 支持 | 2.1.8/3.0.4 | Partial Sync | |
+| drop index | 支持 | 2.1.8/3.0.4 | SQL | |
+| build index | 支持 | 2.1.8/3.0.4 | SQL | |
Bloom Filter
-| 操作 | 是否支持 | 备注 |
-| ------------ | -------------------- | -------- |
-| add bloom filter | 不支持 | 影响未知 |
-| alter bloom filter | 不支持 | 影响未知 |
-| drop bloom filter | 不支持 | 影响未知 |
+| 操作 | 是否支持 | Doris Version | 同步方式 | 备注 |
+| ------------ | -------- | ------------- | -------- | --------- |
+| add bloom filter | 支持 | 2.1.8/3.0.4 | Partial Sync | |
+| alter bloom filter | 支持 | 2.1.8/3.0.4 | Partial Sync | 这里指修改
bloom_filter_columns |
+| drop bloom filter | 支持 | 2.1.8/3.0.4 | Partial Sync | |
## 数据
### 导入
-| 导入方式 | 是否支持 | Doris version | CCR version | 同步方式 | 下游是否可以操作
| 说明
|
-| ------------ | -------------------- | ------------- | ----------- | --------
| ------------------------------------------------------------ |
---------------------------------------------------- |
-| stream load | 支持（临时分区除外） | - | - | TXN |
不可以，如果下游导入了，后续触发full或者Partial Sync，下游导入的数据会丢失 | 上游事务可见，即数据可见时生成binlog，下游开始同步。 |
-| broker load | 支持（临时分区除外） | - | - | TXN | 同上
| 同上
|
-| routine load | 支持（临时分区除外） | - | - | TXN | 同上
| 同上
|
-| mysql load | 支持（临时分区除外） | - | - | TXN | 同上
| 同上
|
-| group commit | 支持（临时分区除外） | 2.1 | 2.1 | TXN | 同上
| 同上
|
+| 导入方式 | 是否支持 | Doris version | 同步方式 | 下游是否可以操作
| 说明
|
+| ------------ | -------------------- | ------------- | -------- |
------------------------------------------------------------ |
---------------------------------------------------- |
+| stream load | 支持（临时分区除外） | - | TXN |
不可以，如果下游导入了，后续触发full或者Partial Sync，下游导入的数据会丢失 | 上游事务可见，即数据可见时生成binlog，下游开始同步。 |
+| broker load | 支持（临时分区除外） | - | TXN | 同上
| 同上
|
+| routine load | 支持（临时分区除外） | - | TXN | 同上
| 同上
|
+| mysql load | 支持（临时分区除外） | - | TXN | 同上
| 同上
|
+| group commit | 支持（临时分区除外） | 2.1 | TXN | 同上
| 同上
|
### 数据操作
-| 操作 | 是否支持 | Doris version | CCR version |
同步方式 | 下游是否可以操作 | 说明
|
-| ------------------------- | -------------------- | ------------- |
----------- | ------------ |
------------------------------------------------------------ |
---------------------------------------------------- |
-| delete | 支持 | - | -
| TXN | 不可以，如果下游操作，后续触发full或者Partial Sync，下游操作会丢失 |
上游事务可见，即数据可见时生成binlog，下游开始同步。 |
-| update | 支持 | - | -
| TXN | 同上 |
同上 |
-| insert | 支持 | - | -
| TXN | 同上 |
同上 |
-| insert into overwrite | 支持（临时分区除外） | 2.1.6 | |
Partial Sync | 同上 | 同上
|
-| insert into overwrite | 支持（临时分区除外） | 2.0 | | full
sync | 同上 | 同上
|
-| 显式事务(3.0)begin commit | 不支持 | | |
| |
|
+| 操作 | 是否支持 | Doris version | 同步方式 |
下游是否可以操作 | 说明
|
+| ------------------------- | -------------------- | ------------- |
------------ | ------------------------------------------------------------ |
---------------------------------------------------- |
+| delete | 支持 | - | TXN
| 不可以，如果下游操作，后续触发full或者Partial Sync，下游操作会丢失 | 上游事务可见，即数据可见时生成binlog，下游开始同步。 |
+| update | 支持 | - | TXN
| 同上 | 同上
|
+| insert | 支持 | - | TXN
| 同上 | 同上
|
+| insert into overwrite | 支持（临时分区除外） | 2.1.6 | Partial Sync | 同上
| 同上
|
+| insert into overwrite | 支持（临时分区除外） | 2.0 | full sync | 同上
| 同上
|
+| 显式事务(3.0)begin commit | 不支持 | | |
|
|
## 分区操作
-| 操作 | 是否支持 | Doris version | CCR version
| 同步方式 | 下游是否可以单独操作 |
说明 |
-| ------------------ | ------------------------------- | ------------- |
----------- | ---------------------------------- |
----------------------------------------------------------- |
------------------------------------------------------------ |
-| add partition | 支持 | - | -
| SQL | 不能，后续触发Full Sync 或者 Partial Sync
会导致下游操作丢失 | cooldown time 属性及其行为未知 |
-| add temp partition | 不支持 | |
| | 同上
|
|
-| drop partition | 支持 | - | -
| 2.0.15/2.1.6 前：Full Sync之后：SQL | 同上
|
|
-| replace partition | 支持 | |
| Partial Sync | 同上
| Partial Sync **只支持 strict range 和 non-tmp partition 的
replace 方式**，否则会触发 Full Sync。 |
-| modify partition | 不支持 | 未发版本 | 未发版本 |
SQL | 同上
| |
-| rename partition | 表级别任务不支持库级别任务支持 | 未发版本 | 未发版本 | SQL
| 同上
| 表级别任务 rename 会导致 CCR 任务停止 |
+| 操作 | 是否支持 | Doris version | 同步方式
| 下游是否可以单独操作 | 说明
|
+| ------------------ | ------------------------------- | ------------- |
---------------------------------- |
----------------------------------------------------------- |
------------------------------------------------------------ |
+| add partition | 支持 | - | SQL
| 不能，后续触发Full Sync 或者 Partial Sync 会导致下游操作丢失 |
cooldown time 属性及其行为未知 |
+| add temp partition | 不支持 | |
| 同上
| backup 不支持 tmp partition，从 doris 2.1.8/3.0.4 开始，可以修改上游 FE
配置：`ignore_backup_tmp_partitions` 绕过该问题 |
+| drop partition | 支持 | - |
SQL/Full Sync | 同上 |
2.0.15/2.1.6 前：Full Sync，之后：SQL |
+| replace partition | 支持 | 2.1.7/3.0.3 | Partial
Sync | 同上
| Partial Sync **只支持 strict range 和 non-tmp partition 的 replace
方式**，否则会触发 Full Sync。 |
+| modify partition | 不支持 | | | 同上
| 指修改 partition 的 property |
+| rename partition | 支持 | 2.1.8/3.0.4 | SQL | 同上
| |
## 视图
-| 操作 | 是否支持 | Doris version | CCR version | 同步方式 | 备注
|
-| ----------- | -------- | ------------- | ----------- | -------- |
-------------------------------- |
-| create view | 支持 | - | - | SQL | 上下游同名时可以工作。 |
-| alter view | 不支持 | | | | 没有binlog
|
-| drop view | 不支持 | | | |
|
+| 操作 | 是否支持 | Doris version | 同步方式 | 备注 |
+| ----------- | -------- | ------------- | -------- |
-------------------------------- |
+| create view | 支持 | - | SQL |
上下游同名时可以工作；如果下游已经存在，则会先删除在创建 |
+| alter view | 支持 | 2.1.8/3.0.4 | SQL | |
+| drop view | 支持 | 2.1.8/3.0.4 | SQL | |
+
+::: note
+由于 doris 实现的限制，view 中的 column name/view name 不能和 db name 相同。
+
+:::
## 物化视图
同步物化视图
-| 操作 | 是否支持 | 备注
|
-| ------------------------ | -------- |
------------------------------------------------------------ |
-| create materialized view | 未知 | 上下游同名时可以工作，不同名时需要下游手动重建 view。 |
-| drop materialized view | 未知 |
|
+| 操作 | 是否支持 | Doris Version | 同步方式 | 备注
|
+| ------------------------ | -------- | ------------- | -------- |
------------------------------------------------------------ |
+| create materialized view | 支持 | 2.1.8/3.0.4 | Partial Sync |
上下游同名时可以工作，不同名时需要下游手动重建 view。 |
+| drop materialized view | 支持 | 2.1.8/3.0.4 | SQL | |
异步物化视图
@@ -260,4 +271,4 @@ Bloom Filter
| function | 不支持 |
| policy | 不支持 |
| user | 不支持 |
-| cancel alter job | 支持 |
\ No newline at end of file
+| cancel alter job | 支持 |
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
