This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new 9caa2b90f3a [doc](stream load) delete the TVF section as it's an immature feature (#1352) 9caa2b90f3a is described below commit 9caa2b90f3a1e92ced0358ea6bca3e1e40a2dc85 Author: hui lai <1353307...@qq.com> AuthorDate: Thu Nov 14 20:32:32 2024 +0800 [doc](stream load) delete the TVF section as it's an immature feature (#1352) --- .../import/import-way/stream-load-manual.md | 45 ---------------------- .../import/import-way/stream-load-manual.md | 45 ---------------------- .../import/import-way/stream-load-manual.md | 42 -------------------- .../import/import-way/stream-load-manual.md | 44 --------------------- 4 files changed, 176 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/stream-load-manual.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/stream-load-manual.md index 2dfb29f12b6..c30c4c49fa3 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/stream-load-manual.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/stream-load-manual.md @@ -377,51 +377,6 @@ Stream Load 是一种同步的导入方式,导入结果会通过创建导入 通过 ErrorURL 可以查看因为数据质量不佳导致的导入失败数据。使用命令 `curl "<ErrorURL>"` 命令直接查看错误数据的信息。 -## TVF 在 Stream Load 中的应用 - http_stream 模式 - -依托 Doris 最新引入的 Table Value Function(TVF)的功能,在 Stream Load 中,可以通过使用 SQL 表达式来表达导入的参数。这个专门为 Stream Load 提供的 TVF 为 http_stream。 - -:::caution -注意 - -使用 TVF http_stream 进行 Stream Load 导入时的 Rest API URL 不同于 Stream Load 普通导入的 URL。 - -- 普通导入的 URL 为: - - http://fe_host:http_port/api/{db}/{table}/_stream_load - -- 使用 TVF http_stream 导入的 URL 为: - - http://fe_host:http_port/api/_http_stream -::: - -使用 `curl` 来使用 Stream Load 的 http stream 模式: -```shell -curl --location-trusted -u user:passwd [-H "sql: ${load_sql}"...] -T data.file -XPUT http://fe_host:http_port/api/_http_stream -``` - -在 Header 中添加一个`sql`的参数,去替代之前参数中的`column_separator`、`line_delimiter`、`where`、`columns`等参数,使用起来非常方便。 - -load_sql 举例: - -```shell -insert into db.table (col1, col2, ...) select c1, c2, ... from http_stream("property1"="value1"); -``` - -http_stream 支持的参数: - -"column_separator" = ",", "format" = "CSV", - -导入 CSV 文件时,`select ... from http_stream` 子句中的列名格式必须为 `c1, c2, c3, ...`,见下方示例 - -... - -示例: - -```Plain -curl --location-trusted -u root: -T test.csv -H "sql:insert into demo.example_tbl_1(user_id, age, cost) select c1, c4, c7 * 2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\" ) where age >= 30" http://127.0.0.1:28030/api/_http_stream -``` - ## 导入举例 ### 设置导入超时时间与最大导入 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/stream-load-manual.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/stream-load-manual.md index 130c3ac7e46..f6ba56b90fb 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/stream-load-manual.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/stream-load-manual.md @@ -377,51 +377,6 @@ Stream Load 是一种同步的导入方式,导入结果会通过创建导入 通过 ErrorURL 可以查看因为数据质量不佳导致的导入失败数据。使用命令 `curl "<ErrorURL>"` 命令直接查看错误数据的信息。 -## TVF 在 Stream Load 中的应用 - http_stream 模式 - -依托 Doris 最新引入的 Table Value Function(TVF)的功能,在 Stream Load 中,可以通过使用 SQL 表达式来表达导入的参数。这个专门为 Stream Load 提供的 TVF 为 http_stream。 - -:::caution -注意 - -使用 TVF http_stream 进行 Stream Load 导入时的 Rest API URL 不同于 Stream Load 普通导入的 URL。 - -- 普通导入的 URL 为: - - http://fe_host:http_port/api/{db}/{table}/_stream_load - -- 使用 TVF http_stream 导入的 URL 为: - - http://fe_host:http_port/api/_http_stream -::: - -使用 `curl` 来使用 Stream Load 的 http stream 模式: -```shell -curl --location-trusted -u user:passwd [-H "sql: ${load_sql}"...] -T data.file -XPUT http://fe_host:http_port/api/_http_stream -``` - -在 Header 中添加一个`sql`的参数,去替代之前参数中的`column_separator`、`line_delimiter`、`where`、`columns`等参数,使用起来非常方便。 - -load_sql 举例: - -```shell -insert into db.table (col1, col2, ...) select c1, c2, ... from http_stream("property1"="value1"); -``` - -http_stream 支持的参数: - -"column_separator" = ",", "format" = "CSV", - -导入 CSV 文件时,`select ... from http_stream` 子句中的列名格式必须为 `c1, c2, c3, ...`,见下方示例 - -... - -示例: - -```Plain -curl --location-trusted -u root: -T test.csv -H "sql:insert into demo.example_tbl_1(user_id, age, cost) select c1, c4, c7 * 2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\" ) where age >= 30" http://127.0.0.1:28030/api/_http_stream -``` - ## 导入举例 ### 设置导入超时时间与最大导入 diff --git a/versioned_docs/version-2.1/data-operate/import/import-way/stream-load-manual.md b/versioned_docs/version-2.1/data-operate/import/import-way/stream-load-manual.md index 097d6c462c8..f8fef0c084d 100644 --- a/versioned_docs/version-2.1/data-operate/import/import-way/stream-load-manual.md +++ b/versioned_docs/version-2.1/data-operate/import/import-way/stream-load-manual.md @@ -377,48 +377,6 @@ The return result parameters are explained in the following table: Users can access the ErrorURL to review data that failed to import due to issues with data quality. By executing the command `curl "<ErrorURL>"`, users can directly retrieve information about the erroneous data. -## Application of Table Value Function in Stream Load - http_stream Mode - -Leveraging the recently introduced functionality of Table Value Function (TVF) in Doris, Stream Load now allows the expression of import parameters through SQL statements. Specifically, a TVF named `http_stream` has been dedicated for Stream Load operations. - -:::tip - -When performing Stream Load using the TVF `http_stream`, the Rest API URL differs from the standard URL used for regular Stream Load imports. - -- Standard Stream Load URL: - `http://fe_host:http_port/api/{db}/{table}/_stream_load` -- URL for Stream Load using TVF `http_stream`: - `http://fe_host:http_port/api/_http_stream` - -::: - -Using curl for Stream Load in http_stream Mode: - -```shell -curl --location-trusted -u user:passwd [-H "sql: ${load_sql}"...] -T data.file -XPUT http://fe_host:http_port/api/_http_stream -``` - -Adding a SQL parameter in the header to replace the previous parameters such as `column_separator`, `line_delimiter`, `where`, `columns`, etc., makes it very convenient to use. - -Example of load SQL: - -```shell -insert into db.table (col, ...) select stream_col, ... from http_stream("property1"="value1"); -``` - -http_stream parameter: - -- "column_separator" = "," - -- "format" = "CSV" -- ... - -For example: - -```Plain -curl --location-trusted -u root: -T test.csv -H "sql:insert into demo.example_tbl_1(user_id, age, cost) select c1, c4, c7 * 2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\" ) where age >= 30" http://127.0.0.1:28030/api/_http_stream -``` - ## Load example ### Setting load timeout and maximum size diff --git a/versioned_docs/version-3.0/data-operate/import/import-way/stream-load-manual.md b/versioned_docs/version-3.0/data-operate/import/import-way/stream-load-manual.md index 3849ae1017f..dc54d5ba700 100644 --- a/versioned_docs/version-3.0/data-operate/import/import-way/stream-load-manual.md +++ b/versioned_docs/version-3.0/data-operate/import/import-way/stream-load-manual.md @@ -377,50 +377,6 @@ The return result parameters are explained in the following table: Users can access the ErrorURL to review data that failed to import due to issues with data quality. By executing the command `curl "<ErrorURL>"`, users can directly retrieve information about the erroneous data. -## Application of Table Value Function in Stream Load - http_stream Mode - -Leveraging the recently introduced functionality of Table Value Function (TVF) in Doris, Stream Load now allows the expression of import parameters through SQL statements. Specifically, a TVF named `http_stream` has been dedicated for Stream Load operations. - -:::tip - -When performing Stream Load using the TVF `http_stream`, the Rest API URL differs from the standard URL used for regular Stream Load imports. - -- Standard Stream Load URL: - `http://fe_host:http_port/api/{db}/{table}/_stream_load` -- URL for Stream Load using TVF `http_stream`: - `http://fe_host:http_port/api/_http_stream` - -::: - -Using curl for Stream Load in http_stream Mode: - -```shell -curl --location-trusted -u user:passwd [-H "sql: ${load_sql}"...] -T data.file -XPUT http://fe_host:http_port/api/_http_stream -``` - -Adding a SQL parameter in the header to replace the previous parameters such as `column_separator`, `line_delimiter`, `where`, `columns`, etc., makes it very convenient to use. - -Example of load SQL: - -```shell -insert into db.table (col1, col2, ...) select c1, c2, ... from http_stream("property1"="value1"); -``` - -http_stream parameter: - -- "column_separator" = "," -- "format" = "CSV" -- ... - -When loading CSV data from http_stream, the column name in `select ... from http_stream` must be in format of `c1, c2, ...`. -See the example below. - -For example: - -```Plain -curl --location-trusted -u root: -T test.csv -H "sql:insert into demo.example_tbl_1(user_id, age, cost) select c1, c4, c7 * 2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\" ) where age >= 30" http://127.0.0.1:28030/api/_http_stream -``` - ## Load example ### Setting load timeout and maximum size --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org