This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 43c446e0685d [docs] Align hidden Sequence load examples with
tab-separated input (#4617)
43c446e0685d is described below
commit 43c446e0685d274c731a8718716929cf7d2b2f7b
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Fri Nov 13 08:02:32 2026 +0800
[docs] Align hidden Sequence load examples with tab-separated input (#4617)
The Sequence-column loading guide mixed a visible `modify_date` example
with unrelated hidden-column templates. The Broker Load template used a
comma separator and placed `source_sequence` in the third position, so
it did not explain the tab-separated data whose sequence value is
fourth.
Add an independent hidden-Sequence table and matching tab-separated
input. Use the same fourth-field mapping in Stream Load, Broker Load,
and Routine Load, and explicitly return to the original table for the
subsequent `modify_date` examples. Apply the correction to English and
Chinese dev, 4.x, and 3.x. Fixes #3332.
Validation: checked source-column mapping validation in Doris branch-3.0
and branch-4.0 `DataDescription`; all six pages preserve front matter
and pass MDX parsing after stripping HTML comments; checked identical
commands/data across versions and locales; `git diff --check` passed.
Load jobs were not executed against external HDFS/Kafka services. Page
paths, existing headings, and navigation are preserved.
Merge coordination: #4624 separately clarifies Sequence requirements and
overlaps the 3.x introductory sentence. Merge either PR first, then
rebase the other while retaining both the concrete hidden-table example
and the requirement clarification.
---------
Co-authored-by: morningman <[email protected]>
---
.../update/unique-update-concurrent-control.md | 81 ++++++++++++--------
.../update/unique-update-concurrent-control.md | 81 ++++++++++++--------
.../update/unique-update-concurrent-control.md | 86 +++++++++++++---------
.../update/unique-update-concurrent-control.md | 81 ++++++++++++--------
.../update/unique-update-concurrent-control.md | 86 +++++++++++++---------
.../update/unique-update-concurrent-control.md | 81 ++++++++++++--------
6 files changed, 302 insertions(+), 194 deletions(-)
diff --git a/docs/data-operate/update/unique-update-concurrent-control.md
b/docs/data-operate/update/unique-update-concurrent-control.md
index 858a11d3e549..b7c6ed895b61 100644
--- a/docs/data-operate/update/unique-update-concurrent-control.md
+++ b/docs/data-operate/update/unique-update-concurrent-control.md
@@ -173,70 +173,87 @@ Because `2020-03-05` is the largest value in the Sequence
column (`modify_date`)
#### 3. Specifying the Sequence Column in Different Import Methods
-When you use `function_column.sequence_type` at table creation (that is, when
the Sequence column is not directly mapped to a column in the table), you must
explicitly specify the mapping from the Sequence column to a data column at
import time.
+When the table uses `function_column.sequence_type`, the Sequence column is
hidden and must be mapped to an input field during loading. The following
examples use a **separate table**, not the `test_table` whose Sequence column
maps to `modify_date` above.
+
+```sql
+CREATE DATABASE IF NOT EXISTS test;
+CREATE TABLE test.test_sequence_hidden
+(
+ k1 BIGINT NOT NULL,
+ k2 DATE NOT NULL,
+ v1 BIGINT,
+ v2 VARCHAR(128)
+)
+UNIQUE KEY(k1, k2)
+DISTRIBUTED BY HASH(k1) BUCKETS 1
+PROPERTIES (
+ "function_column.sequence_type" = "DATE",
+ "replication_num" = "1"
+);
+```
+
+Prepare `sequence_data.tsv` with the following tab-separated rows. The fourth
input field, `source_sequence`, supplies the hidden Sequence column; it is not
a visible table column.
+
+```text
+1 2020-02-22 1 2020-02-22 a
+1 2020-02-22 1 2020-03-23 w
+```
**Stream Load**
-In the header, use `function_column.sequence_col` to specify the
`source_sequence` mapping for the hidden column:
+Map the input fields in file order and name the Sequence field in the request
header:
```shell
-curl --location-trusted -u root \
- -H "columns: k1,k2,source_sequence,v1,v2" \
+curl --location-trusted -u root: \
+ -H "columns: k1,k2,v1,source_sequence,v2" \
-H "function_column.sequence_col: source_sequence" \
- -T testData http://host:port/api/testDb/testTbl/_stream_load
+ -T sequence_data.tsv
http://host:port/api/test/test_sequence_hidden/_stream_load
```
**Broker Load**
-Specify the `source_sequence` field for the hidden column mapping in the
`ORDER BY` clause:
+Upload the same file to HDFS and replace the path and Broker credentials
below. Use a tab separator and the same field order, then select the fourth
input field with `ORDER BY source_sequence`:
```sql
-LOAD LABEL db1.label1
+LOAD LABEL test.sequence_hidden_broker
(
- DATA INFILE("hdfs://host:port/user/data/*/test.txt")
- INTO TABLE `tbl1`
- COLUMNS TERMINATED BY ","
- (k1,k2,source_sequence,v1,v2)
+ DATA INFILE("hdfs://host:port/user/data/sequence_data.tsv")
+ INTO TABLE `test_sequence_hidden`
+ COLUMNS TERMINATED BY "\t"
+ (k1,k2,v1,source_sequence,v2)
ORDER BY source_sequence
)
WITH BROKER 'broker'
(
- "username"="user",
- "password"="pass"
+ "username" = "user",
+ "password" = "pass"
)
-PROPERTIES
-(
- "timeout" = "3600"
-);
+PROPERTIES ("timeout" = "3600");
```
**Routine Load**
-The mapping is the same as above. Example:
+Create the job with your Kafka broker and topic, then send each row above as a
separate tab-separated Kafka message:
```sql
-CREATE ROUTINE LOAD example_db.test1 ON example_tbl
- [WITH MERGE|APPEND|DELETE]
- COLUMNS(k1, k2, source_sequence, v1, v2),
- WHERE k1 100 and k2 like "%doris%"
- [ORDER BY source_sequence]
+CREATE ROUTINE LOAD test.sequence_hidden_routine ON test_sequence_hidden
+ COLUMNS TERMINATED BY "\t",
+ COLUMNS(k1,k2,v1,source_sequence,v2),
+ ORDER BY source_sequence
PROPERTIES
(
- "desired_concurrent_number"="3",
- "max_batch_interval" = "20",
- "max_batch_rows" = "300000",
- "max_batch_size" = "209715200",
- "strict_mode" = "false"
+ "desired_concurrent_number" = "1",
+ "format" = "csv"
)
FROM KAFKA
(
- "kafka_broker_list" = "broker1:9092,broker2:9092,broker3:9092",
- "kafka_topic" = "my_topic",
- "kafka_partitions" = "0,1,2,3",
- "kafka_offsets" = "101,0,0,200"
+ "kafka_broker_list" = "broker1:9092",
+ "kafka_topic" = "my_topic"
);
```
+After a load succeeds and its data becomes visible, the larger Sequence value
(`2020-03-23`) keeps the row whose `v2` is `w`. The `modify_date` examples
below continue to use the original `test_table` and its column mapping.
+
#### 4. Verify the Replacement Order Guarantee
Continuing from the previous example, import the following data (where all
Sequence values are smaller than the existing maximum `2020-03-05`):
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/update/unique-update-concurrent-control.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/update/unique-update-concurrent-control.md
index c9a5600e71d6..9857923e8573 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/update/unique-update-concurrent-control.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/update/unique-update-concurrent-control.md
@@ -173,70 +173,87 @@ MySQL> select * from test_table;
#### 3. 在不同导入方式中指定 Sequence 列
-当建表时使用 `function_column.sequence_type`（即未直接映射到表中的列）时，导入时需要显式指定 Sequence
列到数据列的映射。
+当表使用 `function_column.sequence_type` 时，Sequence
列是隐藏列，导入时需要将输入字段映射到它。以下示例使用一张**独立的表**，与上文将 Sequence 列映射到 `modify_date` 的
`test_table` 不同。
+
+```sql
+CREATE DATABASE IF NOT EXISTS test;
+CREATE TABLE test.test_sequence_hidden
+(
+ k1 BIGINT NOT NULL,
+ k2 DATE NOT NULL,
+ v1 BIGINT,
+ v2 VARCHAR(128)
+)
+UNIQUE KEY(k1, k2)
+DISTRIBUTED BY HASH(k1) BUCKETS 1
+PROPERTIES (
+ "function_column.sequence_type" = "DATE",
+ "replication_num" = "1"
+);
+```
+
+准备 `sequence_data.tsv`，内容为以下制表符分隔的数据。第四个输入字段 `source_sequence` 为隐藏 Sequence
列提供值，它不是表中的可见列。
+
+```text
+1 2020-02-22 1 2020-02-22 a
+1 2020-02-22 1 2020-03-23 w
+```
**Stream Load**
-在 header 中通过 `function_column.sequence_col` 指定隐藏列对应的 `source_sequence` 映射：
+按照文件中的字段顺序设置映射，并在请求 header 中指定 Sequence 字段：
```shell
-curl --location-trusted -u root \
- -H "columns: k1,k2,source_sequence,v1,v2" \
+curl --location-trusted -u root: \
+ -H "columns: k1,k2,v1,source_sequence,v2" \
-H "function_column.sequence_col: source_sequence" \
- -T testData http://host:port/api/testDb/testTbl/_stream_load
+ -T sequence_data.tsv
http://host:port/api/test/test_sequence_hidden/_stream_load
```
**Broker Load**
-在 `ORDER BY` 处指定隐藏列映射的 `source_sequence` 字段：
+将同一文件上传到 HDFS，并替换下方路径和 Broker 凭据。使用制表符分隔和相同的字段顺序，通过 `ORDER BY source_sequence`
指定第四个输入字段：
```sql
-LOAD LABEL db1.label1
+LOAD LABEL test.sequence_hidden_broker
(
- DATA INFILE("hdfs://host:port/user/data/*/test.txt")
- INTO TABLE `tbl1`
- COLUMNS TERMINATED BY ","
- (k1,k2,source_sequence,v1,v2)
+ DATA INFILE("hdfs://host:port/user/data/sequence_data.tsv")
+ INTO TABLE `test_sequence_hidden`
+ COLUMNS TERMINATED BY "\t"
+ (k1,k2,v1,source_sequence,v2)
ORDER BY source_sequence
)
WITH BROKER 'broker'
(
- "username"="user",
- "password"="pass"
+ "username" = "user",
+ "password" = "pass"
)
-PROPERTIES
-(
- "timeout" = "3600"
-);
+PROPERTIES ("timeout" = "3600");
```
**Routine Load**
-映射方式同上，示例如下：
+替换为实际的 Kafka broker 和 topic 创建任务，然后将上述每行数据分别作为一条制表符分隔的 Kafka 消息发送：
```sql
-CREATE ROUTINE LOAD example_db.test1 ON example_tbl
- [WITH MERGE|APPEND|DELETE]
- COLUMNS(k1, k2, source_sequence, v1, v2),
- WHERE k1 100 and k2 like "%doris%"
- [ORDER BY source_sequence]
+CREATE ROUTINE LOAD test.sequence_hidden_routine ON test_sequence_hidden
+ COLUMNS TERMINATED BY "\t",
+ COLUMNS(k1,k2,v1,source_sequence,v2),
+ ORDER BY source_sequence
PROPERTIES
(
- "desired_concurrent_number"="3",
- "max_batch_interval" = "20",
- "max_batch_rows" = "300000",
- "max_batch_size" = "209715200",
- "strict_mode" = "false"
+ "desired_concurrent_number" = "1",
+ "format" = "csv"
)
FROM KAFKA
(
- "kafka_broker_list" = "broker1:9092,broker2:9092,broker3:9092",
- "kafka_topic" = "my_topic",
- "kafka_partitions" = "0,1,2,3",
- "kafka_offsets" = "101,0,0,200"
+ "kafka_broker_list" = "broker1:9092",
+ "kafka_topic" = "my_topic"
);
```
+导入成功且数据可见后，较大的 Sequence 值（`2020-03-23`）会保留 `v2` 为 `w` 的记录。下文的 `modify_date`
示例继续使用原来的 `test_table` 及其列映射。
+
#### 4. 验证替换顺序保证
接续上文示例，再导入如下数据（Sequence 值均小于已有最大值 `2020-03-05`）：
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
index f26ea97f8da7..2b9d757715c3 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
@@ -129,67 +129,87 @@ MySQL> select * from test_table;
在这次导入中，因 sequence column 的值（也就是 modify_date 中的值）中 '2020-03-05' 为最大值，所以 keyword
列中最终保留了 c。
-如果建表时使用了 `function_column.sequence_col` 方式来指定 sequence 列，在导入时需要指定 sequence
列到其他列的映射。
+当表使用 `function_column.sequence_type` 时，Sequence
列是隐藏列，导入时需要将输入字段映射到它。以下示例使用一张**独立的表**，与上文将 Sequence 列映射到 `modify_date` 的
`test_table` 不同。
-**1. Stream Load**
+```sql
+CREATE DATABASE IF NOT EXISTS test;
+CREATE TABLE test.test_sequence_hidden
+(
+ k1 BIGINT NOT NULL,
+ k2 DATE NOT NULL,
+ v1 BIGINT,
+ v2 VARCHAR(128)
+)
+UNIQUE KEY(k1, k2)
+DISTRIBUTED BY HASH(k1) BUCKETS 1
+PROPERTIES (
+ "function_column.sequence_type" = "DATE",
+ "replication_num" = "1"
+);
+```
+
+准备 `sequence_data.tsv`，内容为以下制表符分隔的数据。第四个输入字段 `source_sequence` 为隐藏 Sequence
列提供值，它不是表中的可见列。
+
+```text
+1 2020-02-22 1 2020-02-22 a
+1 2020-02-22 1 2020-03-23 w
+```
+
+**Stream Load**
-Stream Load 的写法是在 header 中的 `function_column.sequence_col` 字段添加隐藏列对应的
source_sequence 的映射，示例如下：
+按照文件中的字段顺序设置映射，并在请求 header 中指定 Sequence 字段：
```shell
-curl --location-trusted -u root -H "columns: k1,k2,source_sequence,v1,v2" -H
"function_column.sequence_col: source_sequence" -T testData
http://host:port/api/testDb/testTbl/_stream_load
+curl --location-trusted -u root: \
+ -H "columns: k1,k2,v1,source_sequence,v2" \
+ -H "function_column.sequence_col: source_sequence" \
+ -T sequence_data.tsv
http://host:port/api/test/test_sequence_hidden/_stream_load
```
-**2. Broker Load**
+**Broker Load**
-在 `ORDER BY` 处设置隐藏列映射的 source_sequence 字段
+将同一文件上传到 HDFS，并替换下方路径和 Broker 凭据。使用制表符分隔和相同的字段顺序，通过 `ORDER BY source_sequence`
指定第四个输入字段：
```sql
-LOAD LABEL db1.label1
+LOAD LABEL test.sequence_hidden_broker
(
- DATA INFILE("hdfs://host:port/user/data/*/test.txt")
- INTO TABLE `tbl1`
- COLUMNS TERMINATED BY ","
- (k1,k2,source_sequence,v1,v2)
+ DATA INFILE("hdfs://host:port/user/data/sequence_data.tsv")
+ INTO TABLE `test_sequence_hidden`
+ COLUMNS TERMINATED BY "\t"
+ (k1,k2,v1,source_sequence,v2)
ORDER BY source_sequence
)
WITH BROKER 'broker'
(
- "username"="user",
- "password"="pass"
+ "username" = "user",
+ "password" = "pass"
)
-PROPERTIES
-(
- "timeout" = "3600"
-);
+PROPERTIES ("timeout" = "3600");
```
-**3. Routine Load**
+**Routine Load**
-映射方式同上，示例如下
+替换为实际的 Kafka broker 和 topic 创建任务，然后将上述每行数据分别作为一条制表符分隔的 Kafka 消息发送：
```sql
-CREATE ROUTINE LOAD example_db.test1 ON example_tbl
- [WITH MERGE|APPEND|DELETE]
- COLUMNS(k1, k2, source_sequence, v1, v2),
- WHERE k1 100 and k2 like "%doris%"
- [ORDER BY source_sequence]
+CREATE ROUTINE LOAD test.sequence_hidden_routine ON test_sequence_hidden
+ COLUMNS TERMINATED BY "\t",
+ COLUMNS(k1,k2,v1,source_sequence,v2),
+ ORDER BY source_sequence
PROPERTIES
(
- "desired_concurrent_number"="3",
- "max_batch_interval" = "20",
- "max_batch_rows" = "300000",
- "max_batch_size" = "209715200",
- "strict_mode" = "false"
+ "desired_concurrent_number" = "1",
+ "format" = "csv"
)
FROM KAFKA
(
- "kafka_broker_list" = "broker1:9092,broker2:9092,broker3:9092",
- "kafka_topic" = "my_topic",
- "kafka_partitions" = "0,1,2,3",
- "kafka_offsets" = "101,0,0,200"
+ "kafka_broker_list" = "broker1:9092",
+ "kafka_topic" = "my_topic"
);
```
+导入成功且数据可见后，较大的 Sequence 值（`2020-03-23`）会保留 `v2` 为 `w` 的记录。下文的 `modify_date`
示例继续使用原来的 `test_table` 及其列映射。
+
**3. 替换顺序的保证**
上述步骤完成后，接着导入如下数据
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
index 71b1dfcc9fc2..05a9eb3008eb 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
@@ -173,70 +173,87 @@ MySQL> select * from test_table;
#### 3. 在不同导入方式中指定 Sequence 列
-当建表时使用 `function_column.sequence_type`（即未直接映射到表中的列）时，导入时需要显式指定 Sequence
列到数据列的映射。
+当表使用 `function_column.sequence_type` 时，Sequence
列是隐藏列，导入时需要将输入字段映射到它。以下示例使用一张**独立的表**，与上文将 Sequence 列映射到 `modify_date` 的
`test_table` 不同。
+
+```sql
+CREATE DATABASE IF NOT EXISTS test;
+CREATE TABLE test.test_sequence_hidden
+(
+ k1 BIGINT NOT NULL,
+ k2 DATE NOT NULL,
+ v1 BIGINT,
+ v2 VARCHAR(128)
+)
+UNIQUE KEY(k1, k2)
+DISTRIBUTED BY HASH(k1) BUCKETS 1
+PROPERTIES (
+ "function_column.sequence_type" = "DATE",
+ "replication_num" = "1"
+);
+```
+
+准备 `sequence_data.tsv`，内容为以下制表符分隔的数据。第四个输入字段 `source_sequence` 为隐藏 Sequence
列提供值，它不是表中的可见列。
+
+```text
+1 2020-02-22 1 2020-02-22 a
+1 2020-02-22 1 2020-03-23 w
+```
**Stream Load**
-在 header 中通过 `function_column.sequence_col` 指定隐藏列对应的 `source_sequence` 映射：
+按照文件中的字段顺序设置映射，并在请求 header 中指定 Sequence 字段：
```shell
-curl --location-trusted -u root \
- -H "columns: k1,k2,source_sequence,v1,v2" \
+curl --location-trusted -u root: \
+ -H "columns: k1,k2,v1,source_sequence,v2" \
-H "function_column.sequence_col: source_sequence" \
- -T testData http://host:port/api/testDb/testTbl/_stream_load
+ -T sequence_data.tsv
http://host:port/api/test/test_sequence_hidden/_stream_load
```
**Broker Load**
-在 `ORDER BY` 处指定隐藏列映射的 `source_sequence` 字段：
+将同一文件上传到 HDFS，并替换下方路径和 Broker 凭据。使用制表符分隔和相同的字段顺序，通过 `ORDER BY source_sequence`
指定第四个输入字段：
```sql
-LOAD LABEL db1.label1
+LOAD LABEL test.sequence_hidden_broker
(
- DATA INFILE("hdfs://host:port/user/data/*/test.txt")
- INTO TABLE `tbl1`
- COLUMNS TERMINATED BY ","
- (k1,k2,source_sequence,v1,v2)
+ DATA INFILE("hdfs://host:port/user/data/sequence_data.tsv")
+ INTO TABLE `test_sequence_hidden`
+ COLUMNS TERMINATED BY "\t"
+ (k1,k2,v1,source_sequence,v2)
ORDER BY source_sequence
)
WITH BROKER 'broker'
(
- "username"="user",
- "password"="pass"
+ "username" = "user",
+ "password" = "pass"
)
-PROPERTIES
-(
- "timeout" = "3600"
-);
+PROPERTIES ("timeout" = "3600");
```
**Routine Load**
-映射方式同上，示例如下：
+替换为实际的 Kafka broker 和 topic 创建任务，然后将上述每行数据分别作为一条制表符分隔的 Kafka 消息发送：
```sql
-CREATE ROUTINE LOAD example_db.test1 ON example_tbl
- [WITH MERGE|APPEND|DELETE]
- COLUMNS(k1, k2, source_sequence, v1, v2),
- WHERE k1 100 and k2 like "%doris%"
- [ORDER BY source_sequence]
+CREATE ROUTINE LOAD test.sequence_hidden_routine ON test_sequence_hidden
+ COLUMNS TERMINATED BY "\t",
+ COLUMNS(k1,k2,v1,source_sequence,v2),
+ ORDER BY source_sequence
PROPERTIES
(
- "desired_concurrent_number"="3",
- "max_batch_interval" = "20",
- "max_batch_rows" = "300000",
- "max_batch_size" = "209715200",
- "strict_mode" = "false"
+ "desired_concurrent_number" = "1",
+ "format" = "csv"
)
FROM KAFKA
(
- "kafka_broker_list" = "broker1:9092,broker2:9092,broker3:9092",
- "kafka_topic" = "my_topic",
- "kafka_partitions" = "0,1,2,3",
- "kafka_offsets" = "101,0,0,200"
+ "kafka_broker_list" = "broker1:9092",
+ "kafka_topic" = "my_topic"
);
```
+导入成功且数据可见后，较大的 Sequence 值（`2020-03-23`）会保留 `v2` 为 `w` 的记录。下文的 `modify_date`
示例继续使用原来的 `test_table` 及其列映射。
+
#### 4. 验证替换顺序保证
接续上文示例，再导入如下数据（Sequence 值均小于已有最大值 `2020-03-05`）：
diff --git
a/versioned_docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
b/versioned_docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
index cb7571873524..95d15459f747 100644
---
a/versioned_docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
+++
b/versioned_docs/version-3.x/data-operate/update/unique-update-concurrent-control.md
@@ -129,67 +129,87 @@ MySQL> select * from test_table;
In this load job, the value '2020-03-05' in the sequence column (modify_date)
is the largest, so the keyword column retains 'c'.
-If the sequence column is specified using `function_column.sequence_col`
during table creation, the sequence column mapping must be specified during
load.
+When the table uses `function_column.sequence_type`, the Sequence column is
hidden and must be mapped to an input field during loading. The following
examples use a **separate table**, not the `test_table` whose Sequence column
maps to `modify_date` above.
-**1. Stream Load**
+```sql
+CREATE DATABASE IF NOT EXISTS test;
+CREATE TABLE test.test_sequence_hidden
+(
+ k1 BIGINT NOT NULL,
+ k2 DATE NOT NULL,
+ v1 BIGINT,
+ v2 VARCHAR(128)
+)
+UNIQUE KEY(k1, k2)
+DISTRIBUTED BY HASH(k1) BUCKETS 1
+PROPERTIES (
+ "function_column.sequence_type" = "DATE",
+ "replication_num" = "1"
+);
+```
+
+Prepare `sequence_data.tsv` with the following tab-separated rows. The fourth
input field, `source_sequence`, supplies the hidden Sequence column; it is not
a visible table column.
+
+```text
+1 2020-02-22 1 2020-02-22 a
+1 2020-02-22 1 2020-03-23 w
+```
+
+**Stream Load**
-In Stream Load, specify the sequence column mapping in the header:
+Map the input fields in file order and name the Sequence field in the request
header:
```shell
-curl --location-trusted -u root -H "columns: k1,k2,source_sequence,v1,v2" -H
"function_column.sequence_col: source_sequence" -T testData
http://host:port/api/testDb/testTbl/_stream_load
+curl --location-trusted -u root: \
+ -H "columns: k1,k2,v1,source_sequence,v2" \
+ -H "function_column.sequence_col: source_sequence" \
+ -T sequence_data.tsv
http://host:port/api/test/test_sequence_hidden/_stream_load
```
-**2. Broker Load**
+**Broker Load**
-Set the hidden column mapping in the `ORDER BY` clause:
+Upload the same file to HDFS and replace the path and Broker credentials
below. Use a tab separator and the same field order, then select the fourth
input field with `ORDER BY source_sequence`:
```sql
-LOAD LABEL db1.label1
+LOAD LABEL test.sequence_hidden_broker
(
- DATA INFILE("hdfs://host:port/user/data/*/test.txt")
- INTO TABLE `tbl1`
- COLUMNS TERMINATED BY ","
- (k1,k2,source_sequence,v1,v2)
+ DATA INFILE("hdfs://host:port/user/data/sequence_data.tsv")
+ INTO TABLE `test_sequence_hidden`
+ COLUMNS TERMINATED BY "\t"
+ (k1,k2,v1,source_sequence,v2)
ORDER BY source_sequence
)
WITH BROKER 'broker'
(
- "username"="user",
- "password"="pass"
+ "username" = "user",
+ "password" = "pass"
)
-PROPERTIES
-(
- "timeout" = "3600"
-);
+PROPERTIES ("timeout" = "3600");
```
-**3. Routine Load**
+**Routine Load**
-The mapping method is the same as above. Example:
+Create the job with your Kafka broker and topic, then send each row above as a
separate tab-separated Kafka message:
```sql
-CREATE ROUTINE LOAD example_db.test1 ON example_tbl
- [WITH MERGE|APPEND|DELETE]
- COLUMNS(k1, k2, source_sequence, v1, v2),
- WHERE k1 > 100 and k2 like "%doris%"
- [ORDER BY source_sequence]
+CREATE ROUTINE LOAD test.sequence_hidden_routine ON test_sequence_hidden
+ COLUMNS TERMINATED BY "\t",
+ COLUMNS(k1,k2,v1,source_sequence,v2),
+ ORDER BY source_sequence
PROPERTIES
(
- "desired_concurrent_number"="3",
- "max_batch_interval" = "20",
- "max_batch_rows" = "300000",
- "max_batch_size" = "209715200",
- "strict_mode" = "false"
+ "desired_concurrent_number" = "1",
+ "format" = "csv"
)
FROM KAFKA
(
- "kafka_broker_list" = "broker1:9092,broker2:9092,broker3:9092",
- "kafka_topic" = "my_topic",
- "kafka_partitions" = "0,1,2,3",
- "kafka_offsets" = "101,0,0,200"
+ "kafka_broker_list" = "broker1:9092",
+ "kafka_topic" = "my_topic"
);
```
+After a load succeeds and its data becomes visible, the larger Sequence value
(`2020-03-23`) keeps the row whose `v2` is `w`. The `modify_date` examples
below continue to use the original `test_table` and its column mapping.
+
**3. Ensuring Replacement Order**
After completing the above steps, load the following data:
diff --git
a/versioned_docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
b/versioned_docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
index 34a4f42bdfa5..09ca762426ab 100644
---
a/versioned_docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
+++
b/versioned_docs/version-4.x/data-operate/update/unique-update-concurrent-control.md
@@ -173,70 +173,87 @@ Because `2020-03-05` is the largest value in the Sequence
column (`modify_date`)
#### 3. Specifying the Sequence Column in Different Import Methods
-When you use `function_column.sequence_type` at table creation (that is, when
the Sequence column is not directly mapped to a column in the table), you must
explicitly specify the mapping from the Sequence column to a data column at
import time.
+When the table uses `function_column.sequence_type`, the Sequence column is
hidden and must be mapped to an input field during loading. The following
examples use a **separate table**, not the `test_table` whose Sequence column
maps to `modify_date` above.
+
+```sql
+CREATE DATABASE IF NOT EXISTS test;
+CREATE TABLE test.test_sequence_hidden
+(
+ k1 BIGINT NOT NULL,
+ k2 DATE NOT NULL,
+ v1 BIGINT,
+ v2 VARCHAR(128)
+)
+UNIQUE KEY(k1, k2)
+DISTRIBUTED BY HASH(k1) BUCKETS 1
+PROPERTIES (
+ "function_column.sequence_type" = "DATE",
+ "replication_num" = "1"
+);
+```
+
+Prepare `sequence_data.tsv` with the following tab-separated rows. The fourth
input field, `source_sequence`, supplies the hidden Sequence column; it is not
a visible table column.
+
+```text
+1 2020-02-22 1 2020-02-22 a
+1 2020-02-22 1 2020-03-23 w
+```
**Stream Load**
-In the header, use `function_column.sequence_col` to specify the
`source_sequence` mapping for the hidden column:
+Map the input fields in file order and name the Sequence field in the request
header:
```shell
-curl --location-trusted -u root \
- -H "columns: k1,k2,source_sequence,v1,v2" \
+curl --location-trusted -u root: \
+ -H "columns: k1,k2,v1,source_sequence,v2" \
-H "function_column.sequence_col: source_sequence" \
- -T testData http://host:port/api/testDb/testTbl/_stream_load
+ -T sequence_data.tsv
http://host:port/api/test/test_sequence_hidden/_stream_load
```
**Broker Load**
-Specify the `source_sequence` field for the hidden column mapping in the
`ORDER BY` clause:
+Upload the same file to HDFS and replace the path and Broker credentials
below. Use a tab separator and the same field order, then select the fourth
input field with `ORDER BY source_sequence`:
```sql
-LOAD LABEL db1.label1
+LOAD LABEL test.sequence_hidden_broker
(
- DATA INFILE("hdfs://host:port/user/data/*/test.txt")
- INTO TABLE `tbl1`
- COLUMNS TERMINATED BY ","
- (k1,k2,source_sequence,v1,v2)
+ DATA INFILE("hdfs://host:port/user/data/sequence_data.tsv")
+ INTO TABLE `test_sequence_hidden`
+ COLUMNS TERMINATED BY "\t"
+ (k1,k2,v1,source_sequence,v2)
ORDER BY source_sequence
)
WITH BROKER 'broker'
(
- "username"="user",
- "password"="pass"
+ "username" = "user",
+ "password" = "pass"
)
-PROPERTIES
-(
- "timeout" = "3600"
-);
+PROPERTIES ("timeout" = "3600");
```
**Routine Load**
-The mapping is the same as above. Example:
+Create the job with your Kafka broker and topic, then send each row above as a
separate tab-separated Kafka message:
```sql
-CREATE ROUTINE LOAD example_db.test1 ON example_tbl
- [WITH MERGE|APPEND|DELETE]
- COLUMNS(k1, k2, source_sequence, v1, v2),
- WHERE k1 100 and k2 like "%doris%"
- [ORDER BY source_sequence]
+CREATE ROUTINE LOAD test.sequence_hidden_routine ON test_sequence_hidden
+ COLUMNS TERMINATED BY "\t",
+ COLUMNS(k1,k2,v1,source_sequence,v2),
+ ORDER BY source_sequence
PROPERTIES
(
- "desired_concurrent_number"="3",
- "max_batch_interval" = "20",
- "max_batch_rows" = "300000",
- "max_batch_size" = "209715200",
- "strict_mode" = "false"
+ "desired_concurrent_number" = "1",
+ "format" = "csv"
)
FROM KAFKA
(
- "kafka_broker_list" = "broker1:9092,broker2:9092,broker3:9092",
- "kafka_topic" = "my_topic",
- "kafka_partitions" = "0,1,2,3",
- "kafka_offsets" = "101,0,0,200"
+ "kafka_broker_list" = "broker1:9092",
+ "kafka_topic" = "my_topic"
);
```
+After a load succeeds and its data becomes visible, the larger Sequence value
(`2020-03-23`) keeps the row whose `v2` is `w`. The `modify_date` examples
below continue to use the original `test_table` and its column mapping.
+
#### 4. Verify the Replacement Order Guarantee
Continuing from the previous example, import the following data (where all
Sequence values are smaller than the existing maximum `2020-03-05`):
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
