This is an automated email from the ASF dual-hosted git repository.
diwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 949562806af [improve](docs) add Flink write metrics doc (#29578)
949562806af is described below
commit 949562806afcbcb3c6a7c5c53bd1312a874ca097
Author: Petrichor <31833513+vinle...@users.noreply.github.com>
AuthorDate: Tue Jan 9 10:36:32 2024 +0800
[improve](docs) add Flink write metrics doc (#29578)
---
docs/en/docs/ecosystem/flink-doris-connector.md | 24 +++++++++++++++++++++-
docs/zh-CN/docs/ecosystem/flink-doris-connector.md | 19 +++++++++++++++++
2 files changed, 42 insertions(+), 1 deletion(-)
diff --git a/docs/en/docs/ecosystem/flink-doris-connector.md
b/docs/en/docs/ecosystem/flink-doris-connector.md
index 737f303f67e..090f515bfc9 100644
--- a/docs/en/docs/ecosystem/flink-doris-connector.md
+++ b/docs/en/docs/ecosystem/flink-doris-connector.md
@@ -390,6 +390,28 @@ ON a.city = c.city
| TIME | DOUBLE |
| HLL | Unsupported datatype |
+## Flink write Metrics
+Where the metrics value of type Counter is the cumulative value of the
imported task from the beginning to the current time, you can observe each
metric in each table in the Flink Webui metrics.
+
+| Name | Metric Type | Description
|
+| ------------------------- | ----------- |
------------------------------------------------------------ |
+| totalFlushLoadBytes | Counter | Number of bytes imported.
|
+| flushTotalNumberRows | Counter | Number of rows imported for total
processing |
+| totalFlushLoadedRows | Counter | Number of rows successfully
imported. |
+| totalFlushTimeMs | Counter | Number of Import completion time.
Unit milliseconds |
+| totalFlushSucceededNumber | Counter | Number of times that the
data-batch been successfully imported. |
+| totalFlushFailedNumber | Counter | Number of times that the
data-batch been failed. |
+| totalFlushFilteredRows | Counter | Number of rows that do not qualify
for data quality flushed |
+| totalFlushUnselectedRows | Counter | Number of rows filtered by where
condition flushed |
+| beginTxnTimeMs | Histogram | The time cost for RPC to Fe to
begin a transaction, Unit milliseconds. |
+| putDataTimeMs | Histogram | The time cost for RPC to Fe to get
a stream load plan, Unit milliseconds. |
+| readDataTimeMs | Histogram | Read data time, Unit milliseconds.
|
+| writeDataTimeMs | Histogram | Write data time, Unit
milliseconds. |
+| commitAndPublishTimeMs | Histogram | The time cost for RPC to Fe to
commit and publish a transaction, Unit milliseconds. |
+| loadTimeMs | Histogram | Import completion time
|
+
+
+
## An example of using Flink CDC to access Doris
```sql
SET 'execution.checkpointing.interval' = '10s';
@@ -759,6 +781,6 @@ You can search for the log `abort transaction response` in
TaskManager and deter
This problem is mainly caused by the conditional varchar/string type, which
needs to be quoted. The correct way to write it is xxx = ''xxx''. In this way,
the Flink SQL parser will interpret two consecutive single quotes as one single
quote character instead of The end of the string, and the concatenated string
is used as the value of the attribute.
-15. **Failed to connect to backend: http://host:webserver_port, and Be is
still alive**
+15. **Failed to connect to backend: http://host:webserver_port, and BE is
still alive**
The issue may have occurred due to configuring the IP address of `be`, which
is not reachable by the external Flink cluster.This is mainly because when
connecting to `fe`, the address of `be` is resolved through fe. For instance,
if you add a be address as '127.0.0.1', the be address obtained by the Flink
cluster through fe will be '127.0.0.1:webserver_port', and Flink will connect
to that address. When this issue arises, you can resolve it by adding the
actual corresponding external IP [...]
diff --git a/docs/zh-CN/docs/ecosystem/flink-doris-connector.md
b/docs/zh-CN/docs/ecosystem/flink-doris-connector.md
index 906f7ecd2d8..c269d0931c1 100644
--- a/docs/zh-CN/docs/ecosystem/flink-doris-connector.md
+++ b/docs/zh-CN/docs/ecosystem/flink-doris-connector.md
@@ -393,6 +393,25 @@ ON a.city = c.city
| TIME | DOUBLE |
| HLL | Unsupported datatype |
+## Flink 写入指标
+其中Counter类型的指标值为导入任务从开始到当前的累加值，可以在Flink Webui metrics中观察各表的各项指标。
+| Name | Metric Type | Description
|
+| ------------------------- | ----------- |
------------------------------------------ |
+| totalFlushLoadBytes | Counter | 已经刷新导入的总字节数 |
+| flushTotalNumberRows | Counter | 已经导入处理的总行数 |
+| totalFlushLoadedRows | Counter | 已经成功导入的总行数 |
+| totalFlushTimeMs | Counter | 已经成功导入完成的总时间 |
+| totalFlushSucceededNumber | Counter | 已经成功导入的次数 |
+| totalFlushFailedNumber | Counter | 失败导入 的次数
|
+| totalFlushFilteredRows | Counter | 数据质量不合格的总行数 |
+| totalFlushUnselectedRows | Counter | 被 where 条件过滤的总行数 |
+| beginTxnTimeMs | Histogram | 向Fe请求开始一个事务所花费的时间，单位毫秒 |
+| putDataTimeMs | Histogram | 向Fe请求获取导入数据执行计划所花费的时间 |
+| readDataTimeMs | Histogram | 读取数据所花费的时间 |
+| writeDataTimeMs | Histogram | 执行写入数据操作所花费的时间 |
+| commitAndPublishTimeMs | Histogram | 向Fe请求提交并且发布事务所花费的时间 |
+| loadTimeMs | Histogram | 导入完成的时间
|
+
## 使用FlinkSQL通过CDC接入Doris示例
```sql
-- enable checkpoint
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
