This is an automated email from the ASF dual-hosted git repository.
diwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new d7b21d3097 [ecosystem](kettle) add kettle plugin for ecosystem (#1193)
d7b21d3097 is described below
commit d7b21d30977a7037d660e61c6efcbba3cbd49b70
Author: wudi <676366...@qq.com>
AuthorDate: Thu Nov 7 09:55:59 2024 +0800
[ecosystem](kettle) add kettle plugin for ecosystem (#1193)
# Versions
- [x] dev
- [x] 3.0
- [x] 2.1
- [x] 2.0
# Languages
- [x] Chinese
- [x] English
---
common_docs_zh/ecosystem/kettle.md | 81 ++++++++++++++++++++++++++++
ecosystem/kettle.md | 78 +++++++++++++++++++++++++++
sidebars.json | 1 +
versioned_sidebars/version-1.2-sidebars.json | 1 +
versioned_sidebars/version-2.0-sidebars.json | 1 +
versioned_sidebars/version-2.1-sidebars.json | 1 +
versioned_sidebars/version-3.0-sidebars.json | 1 +
7 files changed, 164 insertions(+)
diff --git a/common_docs_zh/ecosystem/kettle.md
b/common_docs_zh/ecosystem/kettle.md
new file mode 100644
index 0000000000..9a2130b00f
--- /dev/null
+++ b/common_docs_zh/ecosystem/kettle.md
@@ -0,0 +1,81 @@
+---
+{
+ "title": "Kettle Doris Plugin",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Kettle Doris Plugin
+
+[Kettle](https://pentaho.com/) Doris的插件，用于在Kettle中通过 Stream Load 将其他数据源的数据写入到
Doris 中。
+
+这个插件是利用 Doris 的 Stream Load 功能进行数据导入的。需要配合 Kettle 服务一起使用。
+
+## 关于 Kettle
+
+Kettle 是一款开源的ETL（Extract, Transform, Load）工具，最早由 Pentaho 公司开发，Kettle 是 Pentaho
产品套件中的核心组件之一，主要用于数据集成和数据处理，能够轻松完成从各种来源提取数据、对数据进行清洗和转换，并将其加载到目标系统中的任务。
+
+
+更多信息请参阅：`https://pentaho.com/`
+
+## 使用手册
+
+### 下载Kettle安装
+Kettle 下载地址： https://pentaho.com/download/#download-pentaho
+下载后解压，运行spoon.sh即可启动kettle
+也可以自行编译，参考[编译章节](https://github.com/pentaho/pentaho-kettle?tab=readme-ov-file#how-to-build)
+
+### 编译Kettle Doris Plugin
+```shell
+cd doris/extension/kettle
+mvn clean package -DskipTests
+```
+编译完成后，将插件包解压后拷贝到kettle的plugins目录下
+```shell
+cd assemblies/plugin/target
+unzip doris-stream-loader-plugins-9.4.0.0-343.zip
+cp -r doris-stream-loader ${KETTLE_HOME}/plugins/
+mvn clean package -DskipTests
+```
+### 构建作业
+在Kettle中的批量加载中找到Doris Stream Loader，构建作业
+
+
+
+点击开始运行作业即可完成数据同步
+
+
+
+### 参数说明
+
+| Key | Default Value | Required | Comment |
+|--------------|----------------| -------- |--------------------------------|
+| Step name | -- | Y | 步骤名称
|
+| fenodes | -- | Y | Doris FE http 地址，支持多个地址，使用逗号分隔 |
+| 数据库 | -- | Y | Doris 的写入数据库 |
+| 目标表 | -- | Y | Doris 的写入表 |
+| 用户名 | -- | Y | 访问 Doris 的用户名 |
+| 密码 | -- | N | 访问 Doris 的密码 |
+| 单次导入最大行数 | 10000 | N | 单次导入的最大行数 |
+| 单次导入最大字节 | 10485760(10MB) | N | 单次导入的最大字节大小 |
+| 导入重试次数 | 3 | N | 导入失败之后的重试次数 |
+| StreamLoad属性 | -- | N | Streamload的请求头 |
\ No newline at end of file
diff --git a/ecosystem/kettle.md b/ecosystem/kettle.md
new file mode 100644
index 0000000000..4d0753ccaa
--- /dev/null
+++ b/ecosystem/kettle.md
@@ -0,0 +1,78 @@
+---
+{
+ "title": "Kettle Doris Plugin",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Kettle Doris Plugin
+
+[Kettle](https://pentaho.com/) Doris Plugin is used to write data from other
data sources to Doris through Stream Load in Kettle.
+
+This plug-in uses the Stream Load function of Doris to import data. It needs
to be used in conjunction with the Kettle service.
+
+## About Kettle
+
+Kettle is an open source ETL (Extract, Transform, Load) tool, first developed
by Pentaho, Kettle is one of the core components of the Pentaho product suite,
mainly used for data integration and data processing, and can easily complete
the tasks of extracting data from various sources, cleaning and transforming
data, and loading it into the target system.
+
+For more information, please refer to: `https://pentaho.com/`
+
+## User Manual
+
+### Download Kettle and install
+Kettle download address: https://pentaho.com/download/#download-pentaho
+After downloading, unzip it and run spoon.sh to start kettle
+You can also compile it yourself, refer to the [Compilation
Chapter](https://github.com/pentaho/pentaho-kettle?tab=readme-ov-file#how-to-build)
+
+### Compile Kettle Doris Plugin
+```shell
+cd doris/extension/kettle
+mvn clean package -DskipTests
+```
+After compiling, unzip the plug-in package and copy it to the plugins
directory of kettle
+```shell
+cd assemblies/plugin/target
+unzip doris-stream-loader-plugins-9.4.0.0-343.zip
+cp -r doris-stream-loader ${KETTLE_HOME}/plugins/
+mvn clean package -DskipTests
+```
+### Build a job
+Find Doris Stream Loader in the batch loading in Kettle and build a job
+
+
+Click Start Running the Job to complete data synchronization
+
+
+### Parameter Description
+
+| Key | Default Value | Required | Comment |
+|--------------|----------------| -------- |--------------------------------|
+| Step name | -- | Y | Step name |
+| fenodes | -- | Y | Doris FE http address, supports multiple addresses,
separated by commas |
+| Database | -- | Y | Doris write database |
+| Target table | -- | Y | Doris's write table |
+| Username | -- | Y | Username to access Doris |
+| Password | -- | N | Password to access Doris |
+| Maximum number of rows for a single import | 10000 | N | Maximum number of
rows for a single import |
+| Maximum bytes for a single import | 10485760 (10MB) | N | Maximum byte size
for a single import |
+| Number of import retries | 3 | N | Number of retries after import failure |
+| StreamLoad properties | -- | N | Streamload request header |
\ No newline at end of file
diff --git a/sidebars.json b/sidebars.json
index e79a2dd7b5..df22e0d5e5 100644
--- a/sidebars.json
+++ b/sidebars.json
@@ -1701,6 +1701,7 @@
"ecosystem/datax",
"ecosystem/dbt-doris-adapter",
"ecosystem/seatunnel",
+ "ecosystem/kettle",
"ecosystem/kyuubi",
"ecosystem/logstash",
"ecosystem/beats",
diff --git a/versioned_sidebars/version-1.2-sidebars.json
b/versioned_sidebars/version-1.2-sidebars.json
index f97ed05e93..fd2148f8f5 100644
--- a/versioned_sidebars/version-1.2-sidebars.json
+++ b/versioned_sidebars/version-1.2-sidebars.json
@@ -1205,6 +1205,7 @@
"ecosystem/datax",
"ecosystem/dbt-doris-adapter",
"ecosystem/seatunnel",
+ "ecosystem/kettle",
"ecosystem/kyuubi",
"ecosystem/logstash",
"ecosystem/beats",
diff --git a/versioned_sidebars/version-2.0-sidebars.json
b/versioned_sidebars/version-2.0-sidebars.json
index f8d8bae2ab..959496a88f 100644
--- a/versioned_sidebars/version-2.0-sidebars.json
+++ b/versioned_sidebars/version-2.0-sidebars.json
@@ -1416,6 +1416,7 @@
"ecosystem/datax",
"ecosystem/dbt-doris-adapter",
"ecosystem/seatunnel",
+ "ecosystem/kettle",
"ecosystem/kyuubi",
"ecosystem/logstash",
"ecosystem/beats",
diff --git a/versioned_sidebars/version-2.1-sidebars.json
b/versioned_sidebars/version-2.1-sidebars.json
index 2fd67439cd..b0a476a6e8 100644
--- a/versioned_sidebars/version-2.1-sidebars.json
+++ b/versioned_sidebars/version-2.1-sidebars.json
@@ -1638,6 +1638,7 @@
"ecosystem/datax",
"ecosystem/dbt-doris-adapter",
"ecosystem/seatunnel",
+ "ecosystem/kettle",
"ecosystem/kyuubi",
"ecosystem/logstash",
"ecosystem/beats",
diff --git a/versioned_sidebars/version-3.0-sidebars.json
b/versioned_sidebars/version-3.0-sidebars.json
index be489604be..2ee0493643 100644
--- a/versioned_sidebars/version-3.0-sidebars.json
+++ b/versioned_sidebars/version-3.0-sidebars.json
@@ -1686,6 +1686,7 @@
"ecosystem/datax",
"ecosystem/dbt-doris-adapter",
"ecosystem/seatunnel",
+ "ecosystem/kettle",
"ecosystem/kyuubi",
"ecosystem/logstash",
"ecosystem/beats",
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
