This is an automated email from the ASF dual-hosted git repository.
liaoxin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 09341d831c7 [doc](load) add routine load FAQ (#2272)
09341d831c7 is described below
commit 09341d831c73e6747ecaeeb0690f970095aa46a6
Author: hui lai <1353307...@qq.com>
AuthorDate: Wed Apr 9 14:09:23 2025 +0800
[doc](load) add routine load FAQ (#2272)
---
docs/faq/routineload-faq.md | 55 ++++++++++++++++++++++
.../current/faq/routineload-faq.md | 55 ++++++++++++++++++++++
.../version-2.1/faq/routineload-faq.md | 55 ++++++++++++++++++++++
.../version-3.0/faq/routineload-faq.md | 55 ++++++++++++++++++++++
sidebars.json | 3 +-
versioned_docs/version-2.1/faq/routineload-faq.md | 55 ++++++++++++++++++++++
versioned_docs/version-3.0/faq/routineload-faq.md | 55 ++++++++++++++++++++++
versioned_sidebars/version-2.1-sidebars.json | 3 +-
versioned_sidebars/version-3.0-sidebars.json | 3 +-
9 files changed, 336 insertions(+), 3 deletions(-)
diff --git a/docs/faq/routineload-faq.md b/docs/faq/routineload-faq.md
new file mode 100644
index 00000000000..3960f67bfe8
--- /dev/null
+++ b/docs/faq/routineload-faq.md
@@ -0,0 +1,55 @@
+---
+{
+ "title": "Routine Load FAQ",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Routine Load FAQ
+
+This document records common issues, bug fixes, and optimization improvements
related to Routine Load in Doris. It will be updated periodically.
+
+## Major Bug Fixes
+
+| Issue Description | Trigger
Conditions | Impact Scope | Temporary Solution
| Affected Versions | Fixed Versions | Fix
PR |
+| ----------------------------------------------------------- |
------------------------------------------- | ----------------- |
---------------------------------------------------------- | -----------------
| -------------- | ---------------------------------------------------------- |
+| When at least one job times out while connecting to Kafka, it affects the
import of other jobs, slowing down global Routine Load imports. | At least one
job times out while connecting to Kafka. | Shared-nothing and shared-storage |
Stop or manually pause the job to resolve the issue. | <2.1.9 <3.0.5
| 2.1.9 3.0.5 | [#47530](https://github.com/apache/doris/pull/47530) |
+| User data may be lost after restarting the FE Master. | The job's
offset is set to OFFSET_END, and the FE is restarted. | Shared-storage |
Change the consumption mode to OFFSET_BEGINNING. | 3.0.2-3.0.4 |
3.0.5 | [#46149](https://github.com/apache/doris/pull/46149) |
+| A large number of small transactions are generated during import, causing
compaction to fail and resulting in continuous -235 errors. | Doris consumes
data too quickly, or Kafka data flow is in small batches. | Shared-nothing and
shared-storage | Pause the Routine Load job and execute the following command:
`ALTER ROUTINE LOAD FOR jobname FROM kafka ("property.enable.partition.eof" =
"false");` | <2.1.8 <3.0.4 | 2.1.8 3.0.4 |
[#45528](https://github.com/apache/doris/pull/45528), [ [...]
+| Kafka third-party library destructor hangs, causing data consumption to
fail. | Kafka topic deletion (possibly other conditions). | Shared-nothing and
shared-storage | Restart all BE nodes. |
<2.1.8 <3.0.4 | 2.1.8 3.0.4 |
[#44913](https://github.com/apache/doris/pull/44913) |
+| Routine Load scheduling hangs. | Timeout occurs
when FE aborts a transaction in Meta Service. | Shared-storage | Restart
the FE node. | <3.0.2 | 3.0.2
| [#41267](https://github.com/apache/doris/pull/41267) |
+| Routine Load restart issue. | Restarting BE
nodes. | Shared-nothing and shared-storage | Manually
resume the job. | <2.1.7 <3.0.2 | 2.1.7
3.0.2 | [#3727](https://github.com/selectdb/selectdb-core/pull/3727) |
+
+## Default Configuration Optimizations
+
+| Optimization Content | Applied Versions |
Corresponding PR |
+| ------------------------------------------- | ---------------- |
---------------------------------------------------------- |
+| Increased the timeout duration for Routine Load. | 2.1.7 3.0.3 |
[#42042](https://github.com/apache/doris/pull/42042),
[#40818](https://github.com/apache/doris/pull/40818) |
+| Adjusted the default value of `max_batch_interval`. | 2.1.8 3.0.3 |
[#42491](https://github.com/apache/doris/pull/42491) |
+| Removed the restriction on `max_batch_interval`. | 2.1.5 3.0.0 |
[#29071](https://github.com/apache/doris/pull/29071) |
+| Adjusted the default values of `max_batch_rows` and `max_batch_size`. |
2.1.5 3.0.0 | [#36632](https://github.com/apache/doris/pull/36632) |
+
+## Observability Optimizations
+
+| Optimization Content | Applied Versions | Corresponding PR
|
+| ---------------------------- | ---------------- |
---------------------------------------------------------- |
+| Added observability-related metrics. | 3.0.5 |
[#48209](https://github.com/apache/doris/pull/48209),
[#48171](https://github.com/apache/doris/pull/48171),
[#48963](https://github.com/apache/doris/pull/48963) |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/faq/routineload-faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/faq/routineload-faq.md
new file mode 100644
index 00000000000..84891e72d74
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/faq/routineload-faq.md
@@ -0,0 +1,55 @@
+---
+{
+ "title": "Routine Load 常见问题",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Routine Load 常见问题
+
+本文档记录了 Doris 在使用过程中与 Routine Load 相关的常见问题、Bug 修复及优化改进，并将不定期更新。
+
+## 较严重的 Bug 修复
+
+| 问题描述 | 发生条件
| 影响范围 | 临时解决方案
| 受影响版本 | 修复版本 | 修复 PR
|
+| ---------------------------------------------------------- |
------------------------------------------ | ---------------- |
---------------------------------------------------------- | ------------- |
----------- | ---------------------------------------------------------- |
+| 当至少一个 Job 连接 Kafka 时发生超时，会影响其他 Job 的导入速度，导致全局 Routine Load 导入变慢 | 存在至少一个 Job
连接 Kafka 时发生超时 | 存算分离存算一体 | 通过停止或手动暂停该 Job 来解决。 |
<2.1.9 <3.0.5 | 2.1.9 3.0.5 |
[#47530](https://github.com/apache/doris/pull/47530) |
+| 重启 FE Master 后，用户数据可能丢失 | Job 设置的 Offset 为
OFFSET_END，重启 FE | 存算分离 | 将消费模式更改为 OFFSET_BEGINNING。
| 3.0.2-3.0.4 | 3.0.5 |
[#46149](https://github.com/apache/doris/pull/46149) |
+| 导入过程中产生大量小事务，导致 Compaction 无法及时完成，并持续报 -235 错误。 | Doris 消费速度过快，或 Kafka
数据流量呈小批量趋势 | 存算分离存算一体 | 暂停 Routine Load Job，并执行以下命令：`ALTER ROUTINE LOAD FOR
jobname FROM kafka ("property.enable.partition.eof" = "false");` | <2.1.8
<3.0.4 | 2.1.8 3.0.4 | [#45528](https://github.com/apache/doris/pull/45528),
[#44949](https://github.com/apache/doris/pull/44949),
[#39975](https://github.com/apache/doris/pull/39975) |
+| Kafka 第三方库析构卡住，导致无法正常消费数据。 | Kafka 删除 Topic（可能不止此条件） |
存算分离存算一体 | 重启所有 BE 节点。 | <2.1.8 <3.0.4
| 2.1.8 3.0.4 | [#44913](https://github.com/apache/doris/pull/44913) |
+| Routine Load 调度卡住 | 当 FE 向 Meta Service
中止事务时发生超时 | 存算分离 | 重启 FE 节点。
| <3.0.2 | 3.0.2 |
[#41267](https://github.com/apache/doris/pull/41267) |
+| Routine Load 重启问题 | 重启 BE 节点
| 存算分离存算一体 | 手动恢复 Job。
| <2.1.7 <3.0.2 | 2.1.7 3.0.2 |
[#3727](https://github.com/selectdb/selectdb-core/pull/3727) |
+
+## 默认配置优化
+
+| 优化内容 | 合入版本 | 对应 PR
|
+| ---------------------------------------- | ---------- |
---------------------------------------------------------- |
+| 增加了 Routine Load 的超时时间 | 2.1.7 3.0.3 |
[#42042](https://github.com/apache/doris/pull/42042),
[#40818](https://github.com/apache/doris/pull/40818) |
+| 调整了 max_batch_interval 的默认值 | 2.1.8 3.0.3 |
[#42491](https://github.com/apache/doris/pull/42491) |
+| 移除了 max_batch_interval 的限制 | 2.1.5 3.0.0 |
[#29071](https://github.com/apache/doris/pull/29071) |
+| 调整了 max_batch_rows 和 max_batch_size 的默认值 | 2.1.5 3.0.0 |
[#36632](https://github.com/apache/doris/pull/36632) |
+
+## 可观测优化
+
+| 优化内容 | 合入版本 | 对应 PR
|
+| ----------------------- | -------- |
---------------------------------------------------------- |
+| 增加了可观测性相关的 Metrics 指标 | 3.0.5 |
[#48209](https://github.com/apache/doris/pull/48209),
[#48171](https://github.com/apache/doris/pull/48171),
[#48963](https://github.com/apache/doris/pull/48963) |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/faq/routineload-faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/faq/routineload-faq.md
new file mode 100644
index 00000000000..84891e72d74
--- /dev/null
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/faq/routineload-faq.md
@@ -0,0 +1,55 @@
+---
+{
+ "title": "Routine Load 常见问题",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Routine Load 常见问题
+
+本文档记录了 Doris 在使用过程中与 Routine Load 相关的常见问题、Bug 修复及优化改进，并将不定期更新。
+
+## 较严重的 Bug 修复
+
+| 问题描述 | 发生条件
| 影响范围 | 临时解决方案
| 受影响版本 | 修复版本 | 修复 PR
|
+| ---------------------------------------------------------- |
------------------------------------------ | ---------------- |
---------------------------------------------------------- | ------------- |
----------- | ---------------------------------------------------------- |
+| 当至少一个 Job 连接 Kafka 时发生超时，会影响其他 Job 的导入速度，导致全局 Routine Load 导入变慢 | 存在至少一个 Job
连接 Kafka 时发生超时 | 存算分离存算一体 | 通过停止或手动暂停该 Job 来解决。 |
<2.1.9 <3.0.5 | 2.1.9 3.0.5 |
[#47530](https://github.com/apache/doris/pull/47530) |
+| 重启 FE Master 后，用户数据可能丢失 | Job 设置的 Offset 为
OFFSET_END，重启 FE | 存算分离 | 将消费模式更改为 OFFSET_BEGINNING。
| 3.0.2-3.0.4 | 3.0.5 |
[#46149](https://github.com/apache/doris/pull/46149) |
+| 导入过程中产生大量小事务，导致 Compaction 无法及时完成，并持续报 -235 错误。 | Doris 消费速度过快，或 Kafka
数据流量呈小批量趋势 | 存算分离存算一体 | 暂停 Routine Load Job，并执行以下命令：`ALTER ROUTINE LOAD FOR
jobname FROM kafka ("property.enable.partition.eof" = "false");` | <2.1.8
<3.0.4 | 2.1.8 3.0.4 | [#45528](https://github.com/apache/doris/pull/45528),
[#44949](https://github.com/apache/doris/pull/44949),
[#39975](https://github.com/apache/doris/pull/39975) |
+| Kafka 第三方库析构卡住，导致无法正常消费数据。 | Kafka 删除 Topic（可能不止此条件） |
存算分离存算一体 | 重启所有 BE 节点。 | <2.1.8 <3.0.4
| 2.1.8 3.0.4 | [#44913](https://github.com/apache/doris/pull/44913) |
+| Routine Load 调度卡住 | 当 FE 向 Meta Service
中止事务时发生超时 | 存算分离 | 重启 FE 节点。
| <3.0.2 | 3.0.2 |
[#41267](https://github.com/apache/doris/pull/41267) |
+| Routine Load 重启问题 | 重启 BE 节点
| 存算分离存算一体 | 手动恢复 Job。
| <2.1.7 <3.0.2 | 2.1.7 3.0.2 |
[#3727](https://github.com/selectdb/selectdb-core/pull/3727) |
+
+## 默认配置优化
+
+| 优化内容 | 合入版本 | 对应 PR
|
+| ---------------------------------------- | ---------- |
---------------------------------------------------------- |
+| 增加了 Routine Load 的超时时间 | 2.1.7 3.0.3 |
[#42042](https://github.com/apache/doris/pull/42042),
[#40818](https://github.com/apache/doris/pull/40818) |
+| 调整了 max_batch_interval 的默认值 | 2.1.8 3.0.3 |
[#42491](https://github.com/apache/doris/pull/42491) |
+| 移除了 max_batch_interval 的限制 | 2.1.5 3.0.0 |
[#29071](https://github.com/apache/doris/pull/29071) |
+| 调整了 max_batch_rows 和 max_batch_size 的默认值 | 2.1.5 3.0.0 |
[#36632](https://github.com/apache/doris/pull/36632) |
+
+## 可观测优化
+
+| 优化内容 | 合入版本 | 对应 PR
|
+| ----------------------- | -------- |
---------------------------------------------------------- |
+| 增加了可观测性相关的 Metrics 指标 | 3.0.5 |
[#48209](https://github.com/apache/doris/pull/48209),
[#48171](https://github.com/apache/doris/pull/48171),
[#48963](https://github.com/apache/doris/pull/48963) |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/faq/routineload-faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/faq/routineload-faq.md
new file mode 100644
index 00000000000..84891e72d74
--- /dev/null
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/faq/routineload-faq.md
@@ -0,0 +1,55 @@
+---
+{
+ "title": "Routine Load 常见问题",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Routine Load 常见问题
+
+本文档记录了 Doris 在使用过程中与 Routine Load 相关的常见问题、Bug 修复及优化改进，并将不定期更新。
+
+## 较严重的 Bug 修复
+
+| 问题描述 | 发生条件
| 影响范围 | 临时解决方案
| 受影响版本 | 修复版本 | 修复 PR
|
+| ---------------------------------------------------------- |
------------------------------------------ | ---------------- |
---------------------------------------------------------- | ------------- |
----------- | ---------------------------------------------------------- |
+| 当至少一个 Job 连接 Kafka 时发生超时，会影响其他 Job 的导入速度，导致全局 Routine Load 导入变慢 | 存在至少一个 Job
连接 Kafka 时发生超时 | 存算分离存算一体 | 通过停止或手动暂停该 Job 来解决。 |
<2.1.9 <3.0.5 | 2.1.9 3.0.5 |
[#47530](https://github.com/apache/doris/pull/47530) |
+| 重启 FE Master 后，用户数据可能丢失 | Job 设置的 Offset 为
OFFSET_END，重启 FE | 存算分离 | 将消费模式更改为 OFFSET_BEGINNING。
| 3.0.2-3.0.4 | 3.0.5 |
[#46149](https://github.com/apache/doris/pull/46149) |
+| 导入过程中产生大量小事务，导致 Compaction 无法及时完成，并持续报 -235 错误。 | Doris 消费速度过快，或 Kafka
数据流量呈小批量趋势 | 存算分离存算一体 | 暂停 Routine Load Job，并执行以下命令：`ALTER ROUTINE LOAD FOR
jobname FROM kafka ("property.enable.partition.eof" = "false");` | <2.1.8
<3.0.4 | 2.1.8 3.0.4 | [#45528](https://github.com/apache/doris/pull/45528),
[#44949](https://github.com/apache/doris/pull/44949),
[#39975](https://github.com/apache/doris/pull/39975) |
+| Kafka 第三方库析构卡住，导致无法正常消费数据。 | Kafka 删除 Topic（可能不止此条件） |
存算分离存算一体 | 重启所有 BE 节点。 | <2.1.8 <3.0.4
| 2.1.8 3.0.4 | [#44913](https://github.com/apache/doris/pull/44913) |
+| Routine Load 调度卡住 | 当 FE 向 Meta Service
中止事务时发生超时 | 存算分离 | 重启 FE 节点。
| <3.0.2 | 3.0.2 |
[#41267](https://github.com/apache/doris/pull/41267) |
+| Routine Load 重启问题 | 重启 BE 节点
| 存算分离存算一体 | 手动恢复 Job。
| <2.1.7 <3.0.2 | 2.1.7 3.0.2 |
[#3727](https://github.com/selectdb/selectdb-core/pull/3727) |
+
+## 默认配置优化
+
+| 优化内容 | 合入版本 | 对应 PR
|
+| ---------------------------------------- | ---------- |
---------------------------------------------------------- |
+| 增加了 Routine Load 的超时时间 | 2.1.7 3.0.3 |
[#42042](https://github.com/apache/doris/pull/42042),
[#40818](https://github.com/apache/doris/pull/40818) |
+| 调整了 max_batch_interval 的默认值 | 2.1.8 3.0.3 |
[#42491](https://github.com/apache/doris/pull/42491) |
+| 移除了 max_batch_interval 的限制 | 2.1.5 3.0.0 |
[#29071](https://github.com/apache/doris/pull/29071) |
+| 调整了 max_batch_rows 和 max_batch_size 的默认值 | 2.1.5 3.0.0 |
[#36632](https://github.com/apache/doris/pull/36632) |
+
+## 可观测优化
+
+| 优化内容 | 合入版本 | 对应 PR
|
+| ----------------------- | -------- |
---------------------------------------------------------- |
+| 增加了可观测性相关的 Metrics 指标 | 3.0.5 |
[#48209](https://github.com/apache/doris/pull/48209),
[#48171](https://github.com/apache/doris/pull/48171),
[#48963](https://github.com/apache/doris/pull/48963) |
diff --git a/sidebars.json b/sidebars.json
index 30c2e3863f8..6c3c82d1df6 100644
--- a/sidebars.json
+++ b/sidebars.json
@@ -814,7 +814,8 @@
"faq/sql-faq",
"faq/lakehouse-faq",
"faq/bi-faq",
- "faq/correctness-faq"
+ "faq/correctness-faq",
+ "faq/routineload-faq"
]
},
{
diff --git a/versioned_docs/version-2.1/faq/routineload-faq.md
b/versioned_docs/version-2.1/faq/routineload-faq.md
new file mode 100644
index 00000000000..3960f67bfe8
--- /dev/null
+++ b/versioned_docs/version-2.1/faq/routineload-faq.md
@@ -0,0 +1,55 @@
+---
+{
+ "title": "Routine Load FAQ",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Routine Load FAQ
+
+This document records common issues, bug fixes, and optimization improvements
related to Routine Load in Doris. It will be updated periodically.
+
+## Major Bug Fixes
+
+| Issue Description | Trigger
Conditions | Impact Scope | Temporary Solution
| Affected Versions | Fixed Versions | Fix
PR |
+| ----------------------------------------------------------- |
------------------------------------------- | ----------------- |
---------------------------------------------------------- | -----------------
| -------------- | ---------------------------------------------------------- |
+| When at least one job times out while connecting to Kafka, it affects the
import of other jobs, slowing down global Routine Load imports. | At least one
job times out while connecting to Kafka. | Shared-nothing and shared-storage |
Stop or manually pause the job to resolve the issue. | <2.1.9 <3.0.5
| 2.1.9 3.0.5 | [#47530](https://github.com/apache/doris/pull/47530) |
+| User data may be lost after restarting the FE Master. | The job's
offset is set to OFFSET_END, and the FE is restarted. | Shared-storage |
Change the consumption mode to OFFSET_BEGINNING. | 3.0.2-3.0.4 |
3.0.5 | [#46149](https://github.com/apache/doris/pull/46149) |
+| A large number of small transactions are generated during import, causing
compaction to fail and resulting in continuous -235 errors. | Doris consumes
data too quickly, or Kafka data flow is in small batches. | Shared-nothing and
shared-storage | Pause the Routine Load job and execute the following command:
`ALTER ROUTINE LOAD FOR jobname FROM kafka ("property.enable.partition.eof" =
"false");` | <2.1.8 <3.0.4 | 2.1.8 3.0.4 |
[#45528](https://github.com/apache/doris/pull/45528), [ [...]
+| Kafka third-party library destructor hangs, causing data consumption to
fail. | Kafka topic deletion (possibly other conditions). | Shared-nothing and
shared-storage | Restart all BE nodes. |
<2.1.8 <3.0.4 | 2.1.8 3.0.4 |
[#44913](https://github.com/apache/doris/pull/44913) |
+| Routine Load scheduling hangs. | Timeout occurs
when FE aborts a transaction in Meta Service. | Shared-storage | Restart
the FE node. | <3.0.2 | 3.0.2
| [#41267](https://github.com/apache/doris/pull/41267) |
+| Routine Load restart issue. | Restarting BE
nodes. | Shared-nothing and shared-storage | Manually
resume the job. | <2.1.7 <3.0.2 | 2.1.7
3.0.2 | [#3727](https://github.com/selectdb/selectdb-core/pull/3727) |
+
+## Default Configuration Optimizations
+
+| Optimization Content | Applied Versions |
Corresponding PR |
+| ------------------------------------------- | ---------------- |
---------------------------------------------------------- |
+| Increased the timeout duration for Routine Load. | 2.1.7 3.0.3 |
[#42042](https://github.com/apache/doris/pull/42042),
[#40818](https://github.com/apache/doris/pull/40818) |
+| Adjusted the default value of `max_batch_interval`. | 2.1.8 3.0.3 |
[#42491](https://github.com/apache/doris/pull/42491) |
+| Removed the restriction on `max_batch_interval`. | 2.1.5 3.0.0 |
[#29071](https://github.com/apache/doris/pull/29071) |
+| Adjusted the default values of `max_batch_rows` and `max_batch_size`. |
2.1.5 3.0.0 | [#36632](https://github.com/apache/doris/pull/36632) |
+
+## Observability Optimizations
+
+| Optimization Content | Applied Versions | Corresponding PR
|
+| ---------------------------- | ---------------- |
---------------------------------------------------------- |
+| Added observability-related metrics. | 3.0.5 |
[#48209](https://github.com/apache/doris/pull/48209),
[#48171](https://github.com/apache/doris/pull/48171),
[#48963](https://github.com/apache/doris/pull/48963) |
diff --git a/versioned_docs/version-3.0/faq/routineload-faq.md
b/versioned_docs/version-3.0/faq/routineload-faq.md
new file mode 100644
index 00000000000..3960f67bfe8
--- /dev/null
+++ b/versioned_docs/version-3.0/faq/routineload-faq.md
@@ -0,0 +1,55 @@
+---
+{
+ "title": "Routine Load FAQ",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Routine Load FAQ
+
+This document records common issues, bug fixes, and optimization improvements
related to Routine Load in Doris. It will be updated periodically.
+
+## Major Bug Fixes
+
+| Issue Description | Trigger
Conditions | Impact Scope | Temporary Solution
| Affected Versions | Fixed Versions | Fix
PR |
+| ----------------------------------------------------------- |
------------------------------------------- | ----------------- |
---------------------------------------------------------- | -----------------
| -------------- | ---------------------------------------------------------- |
+| When at least one job times out while connecting to Kafka, it affects the
import of other jobs, slowing down global Routine Load imports. | At least one
job times out while connecting to Kafka. | Shared-nothing and shared-storage |
Stop or manually pause the job to resolve the issue. | <2.1.9 <3.0.5
| 2.1.9 3.0.5 | [#47530](https://github.com/apache/doris/pull/47530) |
+| User data may be lost after restarting the FE Master. | The job's
offset is set to OFFSET_END, and the FE is restarted. | Shared-storage |
Change the consumption mode to OFFSET_BEGINNING. | 3.0.2-3.0.4 |
3.0.5 | [#46149](https://github.com/apache/doris/pull/46149) |
+| A large number of small transactions are generated during import, causing
compaction to fail and resulting in continuous -235 errors. | Doris consumes
data too quickly, or Kafka data flow is in small batches. | Shared-nothing and
shared-storage | Pause the Routine Load job and execute the following command:
`ALTER ROUTINE LOAD FOR jobname FROM kafka ("property.enable.partition.eof" =
"false");` | <2.1.8 <3.0.4 | 2.1.8 3.0.4 |
[#45528](https://github.com/apache/doris/pull/45528), [ [...]
+| Kafka third-party library destructor hangs, causing data consumption to
fail. | Kafka topic deletion (possibly other conditions). | Shared-nothing and
shared-storage | Restart all BE nodes. |
<2.1.8 <3.0.4 | 2.1.8 3.0.4 |
[#44913](https://github.com/apache/doris/pull/44913) |
+| Routine Load scheduling hangs. | Timeout occurs
when FE aborts a transaction in Meta Service. | Shared-storage | Restart
the FE node. | <3.0.2 | 3.0.2
| [#41267](https://github.com/apache/doris/pull/41267) |
+| Routine Load restart issue. | Restarting BE
nodes. | Shared-nothing and shared-storage | Manually
resume the job. | <2.1.7 <3.0.2 | 2.1.7
3.0.2 | [#3727](https://github.com/selectdb/selectdb-core/pull/3727) |
+
+## Default Configuration Optimizations
+
+| Optimization Content | Applied Versions |
Corresponding PR |
+| ------------------------------------------- | ---------------- |
---------------------------------------------------------- |
+| Increased the timeout duration for Routine Load. | 2.1.7 3.0.3 |
[#42042](https://github.com/apache/doris/pull/42042),
[#40818](https://github.com/apache/doris/pull/40818) |
+| Adjusted the default value of `max_batch_interval`. | 2.1.8 3.0.3 |
[#42491](https://github.com/apache/doris/pull/42491) |
+| Removed the restriction on `max_batch_interval`. | 2.1.5 3.0.0 |
[#29071](https://github.com/apache/doris/pull/29071) |
+| Adjusted the default values of `max_batch_rows` and `max_batch_size`. |
2.1.5 3.0.0 | [#36632](https://github.com/apache/doris/pull/36632) |
+
+## Observability Optimizations
+
+| Optimization Content | Applied Versions | Corresponding PR
|
+| ---------------------------- | ---------------- |
---------------------------------------------------------- |
+| Added observability-related metrics. | 3.0.5 |
[#48209](https://github.com/apache/doris/pull/48209),
[#48171](https://github.com/apache/doris/pull/48171),
[#48963](https://github.com/apache/doris/pull/48963) |
diff --git a/versioned_sidebars/version-2.1-sidebars.json
b/versioned_sidebars/version-2.1-sidebars.json
index 3609483ccd9..c5a2165e1c5 100644
--- a/versioned_sidebars/version-2.1-sidebars.json
+++ b/versioned_sidebars/version-2.1-sidebars.json
@@ -827,7 +827,8 @@
"faq/sql-faq",
"faq/lakehouse-faq",
"faq/bi-faq",
- "faq/correctness-faq"
+ "faq/correctness-faq",
+ "faq/routineload-faq"
]
},
{
diff --git a/versioned_sidebars/version-3.0-sidebars.json
b/versioned_sidebars/version-3.0-sidebars.json
index 347a54cc276..62fd0db1d74 100644
--- a/versioned_sidebars/version-3.0-sidebars.json
+++ b/versioned_sidebars/version-3.0-sidebars.json
@@ -873,7 +873,8 @@
"faq/sql-faq",
"faq/lakehouse-faq",
"faq/bi-faq",
- "faq/correctness-faq"
+ "faq/correctness-faq",
+ "faq/routineload-faq"
]
},
{
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
