This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new aa8a81f7406 [doc](load) add flink source (#1674) aa8a81f7406 is described below commit aa8a81f740697f4940b1eea6d848790d0c35904f Author: Xin Liao <liao...@selectdb.com> AuthorDate: Mon Dec 30 18:00:59 2024 +0800 [doc](load) add flink source (#1674) --- docs/data-operate/import/data-source/flink.md | 85 ++++++++++++++++++++++ .../data-operate/import/data-source/flink.md | 85 ++++++++++++++++++++++ .../data-operate/import/data-source/flink.md | 85 ++++++++++++++++++++++ .../data-operate/import/data-source/flink.md | 85 ++++++++++++++++++++++ sidebars.json | 1 + .../data-operate/import/data-source/flink.md | 85 ++++++++++++++++++++++ .../data-operate/import/data-source/flink.md | 85 ++++++++++++++++++++++ versioned_sidebars/version-2.1-sidebars.json | 1 + versioned_sidebars/version-3.0-sidebars.json | 1 + 9 files changed, 513 insertions(+) diff --git a/docs/data-operate/import/data-source/flink.md b/docs/data-operate/import/data-source/flink.md new file mode 100644 index 00000000000..9292e3e2802 --- /dev/null +++ b/docs/data-operate/import/data-source/flink.md @@ -0,0 +1,85 @@ +--- +{ + "title": "Flink", + "language": "en" +} +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +Using the Flink Doris Connector, you can load data generated by Flink (such as data read from Kafka or MySQL) into Doris in real-time. + +## Limitations + +Requires a user-deployed Flink cluster. + +## Loading Data Using Flink + +For detailed steps on loading data using Flink, please refer to [Flink-Doris-Connector](../../../ecosystem/flink-doris-connector). The following steps demonstrate how to quickly load data using Flink. + +### Step 1: Create Table + +```sql +CREATE TABLE `students` ( + `id` INT NULL, + `name` VARCHAR(256) NULL, + `age` INT NULL +) ENGINE=OLAP +UNIQUE KEY(`id`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`id`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +); +``` + +### Step 2: Load Data Using Flink + +Run bin/sql-client.sh to open the FlinkSQL console + +```sql +CREATE TABLE student_sink ( + id INT, + name STRING, + age INT + ) + WITH ( + 'connector' = 'doris', + 'fenodes' = '10.16.10.6:28737', + 'table.identifier' = 'test.students', + 'username' = 'root', + 'password' = '', + 'sink.label-prefix' = 'doris_label' +); + +INSERT INTO student_sink values(1,'zhangsan',123) +``` + +### Step 3: Verify Loaded Data + +```sql +mysql> select * from test.students; ++------+----------+------+ +| id | name | age | ++------+----------+------+ +| 1 | zhangsan | 123 | ++------+----------+------+ +1 rows in set (0.05 sec) +``` \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/data-source/flink.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/data-source/flink.md new file mode 100644 index 00000000000..c2379baa88d --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/data-source/flink.md @@ -0,0 +1,85 @@ +--- +{ + "title": "Flink", + "language": "zh-CN" +} +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +使用 Flink Doris Connector 可以实时的将 Flink 产生的数据(如:Flink 读取 Kafka,MySQL 中的数据)导入到 Doris 中。 + +## 使用限制 + +需要依赖用户部署的 Flink 集群。 + +## 使用 Flink 导入数据 + +使用 Flink 导入数据,详细步骤可以参考 [Flink-Doris-Connector](../../../ecosystem/flink-doris-connector)。在以下步骤中,演示如何通过 Flink 快速导入数据。 + +### 第 1 步:创建表 + +```sql +CREATE TABLE `students` ( + `id` INT NULL, + `name` VARCHAR(256) NULL, + `age` INT NULL +) ENGINE=OLAP +UNIQUE KEY(`id`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`id`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +); +``` + +### 第 2 步:使用 Flink 导入数据 + +运行 bin/sql-client.sh 打开 FlinkSQL 的控制台 + +```sql +CREATE TABLE student_sink ( + id INT, + name STRING, + age INT + ) + WITH ( + 'connector' = 'doris', + 'fenodes' = '10.16.10.6:28737', + 'table.identifier' = 'test.students', + 'username' = 'root', + 'password' = '', + 'sink.label-prefix' = 'doris_label' +); + +INSERT INTO student_sink values(1,'zhangsan',123) +``` + +### 第 3 步:检查导入数据 + +```sql +mysql> select * from test.students; ++------+----------+------+ +| id | name | age | ++------+----------+------+ +| 1 | zhangsan | 123 | ++------+----------+------+ +1 rows in set (0.05 sec) +``` \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/data-source/flink.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/data-source/flink.md new file mode 100644 index 00000000000..c2379baa88d --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/data-source/flink.md @@ -0,0 +1,85 @@ +--- +{ + "title": "Flink", + "language": "zh-CN" +} +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +使用 Flink Doris Connector 可以实时的将 Flink 产生的数据(如:Flink 读取 Kafka,MySQL 中的数据)导入到 Doris 中。 + +## 使用限制 + +需要依赖用户部署的 Flink 集群。 + +## 使用 Flink 导入数据 + +使用 Flink 导入数据,详细步骤可以参考 [Flink-Doris-Connector](../../../ecosystem/flink-doris-connector)。在以下步骤中,演示如何通过 Flink 快速导入数据。 + +### 第 1 步:创建表 + +```sql +CREATE TABLE `students` ( + `id` INT NULL, + `name` VARCHAR(256) NULL, + `age` INT NULL +) ENGINE=OLAP +UNIQUE KEY(`id`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`id`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +); +``` + +### 第 2 步:使用 Flink 导入数据 + +运行 bin/sql-client.sh 打开 FlinkSQL 的控制台 + +```sql +CREATE TABLE student_sink ( + id INT, + name STRING, + age INT + ) + WITH ( + 'connector' = 'doris', + 'fenodes' = '10.16.10.6:28737', + 'table.identifier' = 'test.students', + 'username' = 'root', + 'password' = '', + 'sink.label-prefix' = 'doris_label' +); + +INSERT INTO student_sink values(1,'zhangsan',123) +``` + +### 第 3 步:检查导入数据 + +```sql +mysql> select * from test.students; ++------+----------+------+ +| id | name | age | ++------+----------+------+ +| 1 | zhangsan | 123 | ++------+----------+------+ +1 rows in set (0.05 sec) +``` \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/data-source/flink.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/data-source/flink.md new file mode 100644 index 00000000000..c2379baa88d --- /dev/null +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/data-source/flink.md @@ -0,0 +1,85 @@ +--- +{ + "title": "Flink", + "language": "zh-CN" +} +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +使用 Flink Doris Connector 可以实时的将 Flink 产生的数据(如:Flink 读取 Kafka,MySQL 中的数据)导入到 Doris 中。 + +## 使用限制 + +需要依赖用户部署的 Flink 集群。 + +## 使用 Flink 导入数据 + +使用 Flink 导入数据,详细步骤可以参考 [Flink-Doris-Connector](../../../ecosystem/flink-doris-connector)。在以下步骤中,演示如何通过 Flink 快速导入数据。 + +### 第 1 步:创建表 + +```sql +CREATE TABLE `students` ( + `id` INT NULL, + `name` VARCHAR(256) NULL, + `age` INT NULL +) ENGINE=OLAP +UNIQUE KEY(`id`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`id`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +); +``` + +### 第 2 步:使用 Flink 导入数据 + +运行 bin/sql-client.sh 打开 FlinkSQL 的控制台 + +```sql +CREATE TABLE student_sink ( + id INT, + name STRING, + age INT + ) + WITH ( + 'connector' = 'doris', + 'fenodes' = '10.16.10.6:28737', + 'table.identifier' = 'test.students', + 'username' = 'root', + 'password' = '', + 'sink.label-prefix' = 'doris_label' +); + +INSERT INTO student_sink values(1,'zhangsan',123) +``` + +### 第 3 步:检查导入数据 + +```sql +mysql> select * from test.students; ++------+----------+------+ +| id | name | age | ++------+----------+------+ +| 1 | zhangsan | 123 | ++------+----------+------+ +1 rows in set (0.05 sec) +``` \ No newline at end of file diff --git a/sidebars.json b/sidebars.json index d3a99c061df..54df1f960c3 100644 --- a/sidebars.json +++ b/sidebars.json @@ -178,6 +178,7 @@ "items": [ "data-operate/import/data-source/local-file", "data-operate/import/data-source/kafka", + "data-operate/import/data-source/flink", "data-operate/import/data-source/hdfs", "data-operate/import/data-source/amazon-s3", "data-operate/import/data-source/google-cloud-storage", diff --git a/versioned_docs/version-2.1/data-operate/import/data-source/flink.md b/versioned_docs/version-2.1/data-operate/import/data-source/flink.md new file mode 100644 index 00000000000..9292e3e2802 --- /dev/null +++ b/versioned_docs/version-2.1/data-operate/import/data-source/flink.md @@ -0,0 +1,85 @@ +--- +{ + "title": "Flink", + "language": "en" +} +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +Using the Flink Doris Connector, you can load data generated by Flink (such as data read from Kafka or MySQL) into Doris in real-time. + +## Limitations + +Requires a user-deployed Flink cluster. + +## Loading Data Using Flink + +For detailed steps on loading data using Flink, please refer to [Flink-Doris-Connector](../../../ecosystem/flink-doris-connector). The following steps demonstrate how to quickly load data using Flink. + +### Step 1: Create Table + +```sql +CREATE TABLE `students` ( + `id` INT NULL, + `name` VARCHAR(256) NULL, + `age` INT NULL +) ENGINE=OLAP +UNIQUE KEY(`id`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`id`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +); +``` + +### Step 2: Load Data Using Flink + +Run bin/sql-client.sh to open the FlinkSQL console + +```sql +CREATE TABLE student_sink ( + id INT, + name STRING, + age INT + ) + WITH ( + 'connector' = 'doris', + 'fenodes' = '10.16.10.6:28737', + 'table.identifier' = 'test.students', + 'username' = 'root', + 'password' = '', + 'sink.label-prefix' = 'doris_label' +); + +INSERT INTO student_sink values(1,'zhangsan',123) +``` + +### Step 3: Verify Loaded Data + +```sql +mysql> select * from test.students; ++------+----------+------+ +| id | name | age | ++------+----------+------+ +| 1 | zhangsan | 123 | ++------+----------+------+ +1 rows in set (0.05 sec) +``` \ No newline at end of file diff --git a/versioned_docs/version-3.0/data-operate/import/data-source/flink.md b/versioned_docs/version-3.0/data-operate/import/data-source/flink.md new file mode 100644 index 00000000000..9292e3e2802 --- /dev/null +++ b/versioned_docs/version-3.0/data-operate/import/data-source/flink.md @@ -0,0 +1,85 @@ +--- +{ + "title": "Flink", + "language": "en" +} +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +Using the Flink Doris Connector, you can load data generated by Flink (such as data read from Kafka or MySQL) into Doris in real-time. + +## Limitations + +Requires a user-deployed Flink cluster. + +## Loading Data Using Flink + +For detailed steps on loading data using Flink, please refer to [Flink-Doris-Connector](../../../ecosystem/flink-doris-connector). The following steps demonstrate how to quickly load data using Flink. + +### Step 1: Create Table + +```sql +CREATE TABLE `students` ( + `id` INT NULL, + `name` VARCHAR(256) NULL, + `age` INT NULL +) ENGINE=OLAP +UNIQUE KEY(`id`) +COMMENT 'OLAP' +DISTRIBUTED BY HASH(`id`) BUCKETS 1 +PROPERTIES ( +"replication_allocation" = "tag.location.default: 1" +); +``` + +### Step 2: Load Data Using Flink + +Run bin/sql-client.sh to open the FlinkSQL console + +```sql +CREATE TABLE student_sink ( + id INT, + name STRING, + age INT + ) + WITH ( + 'connector' = 'doris', + 'fenodes' = '10.16.10.6:28737', + 'table.identifier' = 'test.students', + 'username' = 'root', + 'password' = '', + 'sink.label-prefix' = 'doris_label' +); + +INSERT INTO student_sink values(1,'zhangsan',123) +``` + +### Step 3: Verify Loaded Data + +```sql +mysql> select * from test.students; ++------+----------+------+ +| id | name | age | ++------+----------+------+ +| 1 | zhangsan | 123 | ++------+----------+------+ +1 rows in set (0.05 sec) +``` \ No newline at end of file diff --git a/versioned_sidebars/version-2.1-sidebars.json b/versioned_sidebars/version-2.1-sidebars.json index 2bc38e2b5cf..0284d0f16dc 100644 --- a/versioned_sidebars/version-2.1-sidebars.json +++ b/versioned_sidebars/version-2.1-sidebars.json @@ -164,6 +164,7 @@ "items": [ "data-operate/import/data-source/local-file", "data-operate/import/data-source/kafka", + "data-operate/import/data-source/flink", "data-operate/import/data-source/hdfs", "data-operate/import/data-source/amazon-s3", "data-operate/import/data-source/google-cloud-storage", diff --git a/versioned_sidebars/version-3.0-sidebars.json b/versioned_sidebars/version-3.0-sidebars.json index f6a94bbba9b..ec94a07f6b6 100644 --- a/versioned_sidebars/version-3.0-sidebars.json +++ b/versioned_sidebars/version-3.0-sidebars.json @@ -176,6 +176,7 @@ "items": [ "data-operate/import/data-source/local-file", "data-operate/import/data-source/kafka", + "data-operate/import/data-source/flink", "data-operate/import/data-source/hdfs", "data-operate/import/data-source/amazon-s3", "data-operate/import/data-source/google-cloud-storage", --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org