[doris-website] branch master updated: fix-doc20 (#110)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new fbf5972a77e fix-doc20 (#110) fbf5972a77e is described below commit fbf5972a77e58526c5ed6bbbc7900091188105ec Author: Liqf <109049295+lemonlit...@users.noreply.github.com> AuthorDate: Thu Sep 22 12:44:14 2022 +0800 fix-doc20 (#110) --- .../Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md | 1 + .../Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md index 7498fdaa982..e5da5e25d9b 100644 --- a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md +++ b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md @@ -54,6 +54,7 @@ illustrate: - The user needs to have`SELECT`permission for the source table and`CREATE`permission for the target database - After a table is created, data is imported. If the import fails, the table is deleted - You can specify the key type. The default key type is `Duplicate Key` +- The key column and partition bucket column will not be inherited. It is recommended to specify by yourself. If not, create a table based on the default key column and bucket column ### Example diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md index 82938f93159..8d3c6ac135d 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md @@ -54,6 +54,7 @@ CREATE TABLE table_name [( column_name_list )] - 用户需要拥有来源表的`SELECT`权限和目标库的`CREATE`权限 - 创建表成功后，会进行数据导入，如果导入失败，将会删除表 - 可以自行指定 key type，默认为`Duplicate Key` +- Key列和分区分桶列不会继承，建议自行指定，不指定的情况下按默认key列和分桶列建表 ### Example - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: Removed documentation for version 1.1 numbers function (#109)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new e947eebb9f8 Removed documentation for version 1.1 numbers function
(#109)
e947eebb9f8 is described below
commit e947eebb9f830f542234a59bf9d542495c603bac
Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com>
AuthorDate: Thu Sep 22 12:44:54 2022 +0800
Removed documentation for version 1.1 numbers function (#109)
---
.../sql-functions/table-functions/numbers.md | 65 -
.../sql-functions/table-functions/numbers.md | 68 --
sidebars.json | 3 +-
3 files changed, 1 insertion(+), 135 deletions(-)
diff --git a/docs/sql-manual/sql-functions/table-functions/numbers.md
b/docs/sql-manual/sql-functions/table-functions/numbers.md
deleted file mode 100644
index ca0f5de9153..000
--- a/docs/sql-manual/sql-functions/table-functions/numbers.md
+++ /dev/null
@@ -1,65 +0,0 @@
-{
-"title": "numbers",
-"language": "en"
-}
-
-
-
-## `numbers`
-
-### description
-
-Table-Value-Function, generate a temporary table with only one column named
'number', row values are [0,n).
-
-This function is used in FROM clauses.
-
-grammar:
-```
-FROM numbers(n[,m]);
-```
-
-parameter：
-- `n`: It means to generate rows [0, n).
-- `m`: Optional parameters. It means this function is executed simultaneously
on `m` be nodes (multiple BEs need to be deployed).
-
-### example
-```
-mysql> select * from numbers("5");
-++
-| number |
-++
-| 0 |
-| 1 |
-| 2 |
-| 3 |
-| 4 |
-| 5 |
-| 6 |
-| 7 |
-| 8 |
-| 9 |
-++
-```
-
-### keywords
-
-numbers
\ No newline at end of file
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/table-functions/numbers.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/table-functions/numbers.md
deleted file mode 100644
index 60605f72825..000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/table-functions/numbers.md
+++ /dev/null
@@ -1,68 +0,0 @@
-{
-"title": "numbers",
-"language": "zh-CN"
-}
-
-
-
-## `numbers`
-
-### description
-
-表函数，生成一张只含有一列的临时表，列名为`number`，行的值为[0,n)。
-
-该函数用于from子句中。
-
-语法：
-
-```
-FROM numbers(n[,m]);
-```
-
-参数：
-- `n`: 代表生成[0,n)的行。
-- `m`: 可选参数,代表`m`个be节点同时执行该函数（需要部署多个be）。
-
-### example
-```
-mysql> select * from numbers("10");
-++
-| number |
-++
-| 0 |
-| 1 |
-| 2 |
-| 3 |
-| 4 |
-| 5 |
-| 6 |
-| 7 |
-| 8 |
-| 9 |
-++
-```
-
-### keywords
-
-numbers
-
-
diff --git a/sidebars.json b/sidebars.json
index bac22435ae4..feae10df74f 100644
--- a/sidebars.json
+++ b/sidebars.json
@@ -489,8 +489,7 @@
"sql-manual/sql-functions/table-functions/explode-bitmap",
"sql-manual/sql-functions/table-functions/explode-split",
"sql-manual/sql-functions/table-functions/explode-json-array",
-
"sql-manual/sql-functions/table-functions/outer-combinator",
-
"sql-manual/sql-functions/table-functions/numbers"
+
"sql-manual/sql-functions/table-functions/outer-combinator"
]
},
{
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: add ADMIN-CLEAN-TRASH
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new d569a99c79c add ADMIN-CLEAN-TRASH d569a99c79c is described below commit d569a99c79c9d8dfcb759ec3072fad8023400a01 Author: jiafeng.zhang AuthorDate: Thu Sep 22 18:36:31 2022 +0800 add ADMIN-CLEAN-TRASH add ADMIN-CLEAN-TRASH --- sidebars.json | 1 + 1 file changed, 1 insertion(+) diff --git a/sidebars.json b/sidebars.json index feae10df74f..1ee53be255e 100644 --- a/sidebars.json +++ b/sidebars.json @@ -667,6 +667,7 @@ "items": [ "sql-manual/sql-reference/Database-Administration-Statements/ADMIN-CANCEL-REPAIR", "sql-manual/sql-reference/Database-Administration-Statements/ADMIN-CHECK-TABLET", + "sql-manual/sql-reference/Database-Administration-Statements/ADMIN-CLEAN-TRASH", "sql-manual/sql-reference/Database-Administration-Statements/ADMIN-COPY-TABLET", "sql-manual/sql-reference/Database-Administration-Statements/ADMIN-REPAIR-TABLE", "sql-manual/sql-reference/Database-Administration-Statements/ADMIN-SET-CONFIG", - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: Optimized date function doc order and add partial function doc (#111)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 5b3ec602538 Optimized date function doc order and add partial function
doc (#111)
5b3ec602538 is described below
commit 5b3ec60253811ec8084ed1cbe72c2de8ddd7798e
Author: zy-kkk
AuthorDate: Fri Sep 23 10:16:42 2022 +0800
Optimized date function doc order and add partial function doc (#111)
---
.../{utc_timestamp.md => days_add.md} | 30 +-
.../{utc_timestamp.md => days_diff.md} | 30 +-
.../{utc_timestamp.md => days_sub.md} | 30 +-
.../{utc_timestamp.md => extract.md} | 39 +++--
.../{utc_timestamp.md => hours_add.md} | 30 +-
.../{utc_timestamp.md => hours_diff.md}| 30 +-
.../{utc_timestamp.md => hours_sub.md} | 30 +-
.../{utc_timestamp.md => localtime.md} | 42 +++--
.../{utc_timestamp.md => minutes_add.md} | 30 +-
.../{utc_timestamp.md => minutes_diff.md} | 30 +-
.../{utc_timestamp.md => minutes_sub.md} | 30 +-
.../{utc_timestamp.md => months_add.md}| 30 +-
.../{utc_timestamp.md => months_diff.md} | 30 +-
.../{utc_timestamp.md => months_sub.md}| 30 +-
.../{utc_timestamp.md => quarter.md} | 34 +--
.../{utc_timestamp.md => seconds_add.md} | 30 +-
.../{utc_timestamp.md => seconds_diff.md} | 30 +-
.../{utc_timestamp.md => seconds_sub.md} | 30 +-
.../date-time-functions/utc_timestamp.md | 1 +
.../{utc_timestamp.md => weeks_add.md} | 30 +-
.../{utc_timestamp.md => weeks_diff.md}| 30 +-
.../{utc_timestamp.md => weeks_sub.md} | 30 +-
.../{utc_timestamp.md => year_floor.md}| 34 +--
.../{utc_timestamp.md => years_add.md} | 30 +-
.../{utc_timestamp.md => years_diff.md}| 30 +-
.../{utc_timestamp.md => years_sub.md} | 30 +-
.../sql-functions/date-time-functions/days_add.md | 32 +-
.../sql-functions/date-time-functions/days_diff.md | 32 +-
.../sql-functions/date-time-functions/days_sub.md | 32 +-
.../sql-functions/date-time-functions/extract.md | 53 +
.../sql-functions/date-time-functions/hours_add.md | 32 +-
.../date-time-functions/hours_diff.md | 32 +-
.../sql-functions/date-time-functions/hours_sub.md | 32 +-
.../sql-functions/date-time-functions/localtime.md | 44 --
.../date-time-functions/minutes_add.md | 32 +-
.../date-time-functions/minutes_diff.md| 32 +-
.../date-time-functions/minutes_sub.md | 32 +-
.../date-time-functions/months_add.md | 32 +-
.../date-time-functions/months_diff.md | 32 +-
.../date-time-functions/months_sub.md | 32 +-
.../sql-functions/date-time-functions/quarter.md | 36 ++--
.../date-time-functions/seconds_add.md | 32 +-
.../date-time-functions/seconds_diff.md| 32 +-
.../date-time-functions/seconds_sub.md | 32 +-
.../sql-functions/date-time-functions/weeks_add.md | 32 +-
.../date-time-functions/weeks_diff.md | 32 +-
.../sql-functions/date-time-functions/weeks_sub.md | 32 +-
.../date-time-functions/year_floor.md | 36 ++--
.../sql-functions/date-time-functions/years_add.md | 32 +-
.../date-time-functions/years_diff.md | 32 +-
.../sql-functions/date-time-functions/years_sub.md | 32 +-
sidebars.json | 68 +++---
52 files changed, 923 insertions(+), 766 deletions(-)
diff --git a/docs/sql-manual/sql-functions/date-time-functions/utc_timestamp.md
b/docs/sql-manual/sql-functions/date-time-functions/days_add.md
similarity index 59%
copy from docs/sql-manual/sql-functions/date-time-functions/utc_timestamp.md
copy to docs/sql-manual/sql-functions/date-time-functions/days_add.md
index 24ad8a45813..8e244e62920 100644
--- a/docs/sql-manual/sql-functions/date-time-functions/utc_timestamp.md
+++ b/docs/sql-manual/sql-functions/date-time-functions/days_add.md
@@ -1,6 +1,6 @@
---
{
-"title": "utc_timestamp",
+"title": "days_add",
"language": "en"
}
---
@@ -24,25 +24,27 @@ specific language governing permissions and limitati
[doris-website] branch master updated: Spark load import kerberos parameter modification (#113)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new a44a4cc9146 Spark load import kerberos parameter modification (#113) a44a4cc9146 is described below commit a44a4cc91468b398beb9452643fb3a6515b4936c Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com> AuthorDate: Fri Sep 23 17:54:54 2022 +0800 Spark load import kerberos parameter modification (#113) --- .../import/import-way/spark-load-manual.md | 30 +++--- .../import/import-way/spark-load-manual.md | 30 +++--- 2 files changed, 30 insertions(+), 30 deletions(-) diff --git a/docs/data-operate/import/import-way/spark-load-manual.md b/docs/data-operate/import/import-way/spark-load-manual.md index de523f89a3e..c8bfb0e7764 100644 --- a/docs/data-operate/import/import-way/spark-load-manual.md +++ b/docs/data-operate/import/import-way/spark-load-manual.md @@ -154,10 +154,10 @@ PROPERTIES working_dir = path, broker = broker_name, broker.property_key = property_value, - hadoop.security.authentication = kerberos, - kerberos_principal = do...@your.com, - kerberos_keytab = /home/doris/my.keytab - kerberos_keytab_content = ASDOWHDLAWIDJHWLDKSALDJSDIWALD + broker.hadoop.security.authentication = kerberos, + broker.kerberos_principal = do...@your.com, + broker.kerberos_keytab = /home/doris/my.keytab + broker.kerberos_keytab_content = ASDOWHDLAWIDJHWLDKSALDJSDIWALD ) -- drop spark resource @@ -194,10 +194,10 @@ REVOKE USAGE_PRIV ON RESOURCE resource_name FROM ROLE role_name - Other parameters are optional, refer to `http://spark.apache.org/docs/latest/configuration.html` - `working_dir`: directory used by ETL. Spark is required when used as an ETL resource. For example: `hdfs://host :port/tmp/doris`. -- `hadoop.security.authentication`: Specify the authentication method as kerberos. -- `kerberos_principal`: Specify the principal of kerberos. -- `kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. -- `kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `kerberos_keytab` configuration. +- `broker.hadoop.security.authentication`: Specify the authentication method as kerberos. +- `broker.kerberos_principal`: Specify the principal of kerberos. +- `broker.kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. +- `broker.kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `broker.kerberos_keytab` configuration. - `broker`: the name of the broker. Spark is required when used as an ETL resource. You need to use the 'alter system add broker' command to complete the configuration in advance. - `broker.property_key`: the authentication information that the broker needs to specify when reading the intermediate file generated by ETL. @@ -239,10 +239,10 @@ PROPERTIES If Spark load accesses Hadoop cluster resources with Kerberos authentication, we only need to specify the following parameters when creating Spark resources: -- `hadoop.security.authentication`: Specify the authentication method as kerberos. -- `kerberos_principal`: Specify the principal of kerberos. -- `kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. -- `kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `kerberos_keytab` configuration. +- `broker.hadoop.security.authentication`: Specify the authentication method as kerberos. +- `broker.kerberos_principal`: Specify the principal of kerberos. +- `broker.kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. +- `broker.kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `kerberos_keytab` configuration. Example： @@ -261,9 +261,9 @@ PROPERTIES "spark.hadoop.fs.defaultFS" = "hdfs://127.0.0.1:1", "working_dir" = "hdfs://127.0.0.1:1/tmp/doris", "broker" = "broker0", - "hadoop.security.authentication" =
[doris] branch master updated: build extension docs failed fix (#12915)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 943814a86f build extension docs failed fix (#12915) 943814a86f is described below commit 943814a86fdc250171fe000ea932e953bad2d075 Author: jiafeng.zhang AuthorDate: Fri Sep 23 21:58:02 2022 +0800 build extension docs failed fix (#12915) build extension docs fix --- .github/workflows/build-extension.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/build-extension.yml b/.github/workflows/build-extension.yml index e4c0ca731c..7f91eda546 100644 --- a/.github/workflows/build-extension.yml +++ b/.github/workflows/build-extension.yml @@ -99,6 +99,7 @@ jobs: rm -rf website/i18n/zh-CN/docusaurus-plugin-content-docs-community/* mkdir website/i18n/zh-CN/docusaurus-plugin-content-docs-community/current cp -R docs/zh-CN/community/* website/i18n/zh-CN/docusaurus-plugin-content-docs-community/current/ + cp -R docs/sidebarsCommunity.json website/ cp -R docs/sidebars.json website/ cp -R docs/images website/static/ sed '2,3d' website/versions.json > website/versions.json1 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [fix](log)Audit log status is incorrect (#12822)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
new 3f06837059 [fix](log)Audit log status is incorrect (#12822)
3f06837059 is described below
commit 3f068370598de2dc64476fe65edae2b71946b749
Author: jiafeng.zhang
AuthorDate: Sat Sep 24 09:40:29 2022 +0800
[fix](log)Audit log status is incorrect (#12822)
Audit log status is incorrect
---
.../src/main/java/org/apache/doris/qe/ConnectProcessor.java | 8 +++-
.../src/main/java/org/apache/doris/qe/MasterOpExecutor.java | 9 +
fe/fe-core/src/main/java/org/apache/doris/qe/QueryState.java | 3 ++-
fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java | 7 +++
.../main/java/org/apache/doris/service/FrontendServiceImpl.java | 7 +++
gensrc/thrift/FrontendService.thrift | 1 +
6 files changed, 33 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
index a0da3f94f0..520cd87f8a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
@@ -54,6 +54,7 @@ import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.iceberg.catalog.Catalog;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -158,7 +159,11 @@ public class ConnectProcessor {
ctx.getAuditEventBuilder().setStmt(origStmt);
}
}
-
+if(!Catalog.getCurrentCatalog().isMaster()) {
+if(ctx.executor.isForwardToMaster()) {
+
ctx.getAuditEventBuilder().setState(ctx.executor.getProxyStatus());
+}
+}
Catalog.getCurrentAuditEventProcessor().handleAuditEvent(ctx.getAuditEventBuilder().build());
}
@@ -506,6 +511,7 @@ public class ConnectProcessor {
}
result.setMaxJournalId(Catalog.getCurrentCatalog().getMaxJournalId().longValue());
result.setPacket(getResultPacket());
+result.setStatus(ctx.getState().toString());
if (executor != null && executor.getProxyResultSet() != null) {
result.setResultSet(executor.getProxyResultSet().tothrift());
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
index 36b86f2588..d960b52fca 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/MasterOpExecutor.java
@@ -141,6 +141,15 @@ public class MasterOpExecutor {
}
}
+public String getProxyStatus() {
+if (result == null) {
+return QueryState.MysqlStateType.UNKNOWN.name();
+}
+if (!result.isSetStatus()) {
+return QueryState.MysqlStateType.UNKNOWN.name();
+}
+return result.getStatus();
+}
public ShowResultSet getProxyResultSet() {
if (result == null) {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/QueryState.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/QueryState.java
index cf0376ab4a..fd24d9a1ad 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/QueryState.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/QueryState.java
@@ -29,7 +29,8 @@ public class QueryState {
NOOP, // send nothing to remote
OK, // send OK packet to remote
EOF,// send EOF packet to remote
-ERR // send ERROR packet to remote
+ERR, // send ERROR packet to remote
+UNKNOWN // send UNKNOWN packet to remote
}
public enum ErrType {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
index 5acf31d0b2..6302268452 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
@@ -296,6 +296,13 @@ public class StmtExecutor implements ProfileWriter {
}
}
+public String getProxyStatus() {
+if (masterOpExecutor == null) {
+return QueryState.MysqlStateType.UNKNOWN.name();
+}
+return masterOpExecutor.getProxyStatus();
+}
+
public boolean isQueryStmt() {
return parsedStmt != null && parsedStmt instanceof QueryStmt;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
index 56d0cfd1fd..aac464cb99
[doris-website] branch master updated: [doc] Add blog 'principle of Doris Stream Load' (#88)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new cdffc330088 [doc] Add blog 'principle of Doris Stream Load' (#88)
cdffc330088 is described below
commit cdffc330088a4b33f1c54c6e5f0d7282e2b24ea3
Author: TaoZex <45089228+tao...@users.noreply.github.com>
AuthorDate: Sun Sep 25 10:32:41 2022 +0800
[doc] Add blog 'principle of Doris Stream Load' (#88)
* principle-of-Doris-Stream-Load
---
blog/principle-of-Doris-Stream-Load.md | 155 +
.../principle-of-Doris-Stream-Load.md | 155 +
.../principle-of-Doris-Stream-Load/Figure_1_cn.png | Bin 0 -> 58221 bytes
.../principle-of-Doris-Stream-Load/Figure_1_en.png | Bin 0 -> 58221 bytes
.../principle-of-Doris-Stream-Load/Figure_2_cn.png | Bin 0 -> 160362 bytes
.../principle-of-Doris-Stream-Load/Figure_2_en.png | Bin 0 -> 309493 bytes
.../principle-of-Doris-Stream-Load/Figure_3_cn.png | Bin 0 -> 16237 bytes
.../principle-of-Doris-Stream-Load/Figure_3_en.png | Bin 0 -> 16237 bytes
.../principle-of-Doris-Stream-Load/Figure_4_cn.png | Bin 0 -> 134398 bytes
.../principle-of-Doris-Stream-Load/Figure_4_en.png | Bin 0 -> 134398 bytes
.../principle-of-Doris-Stream-Load/Figure_5_cn.png | Bin 0 -> 85638 bytes
.../principle-of-Doris-Stream-Load/Figure_5_en.png | Bin 0 -> 85638 bytes
.../principle-of-Doris-Stream-Load/Figure_6_cn.png | Bin 0 -> 96350 bytes
.../principle-of-Doris-Stream-Load/Figure_6_en.png | Bin 0 -> 96350 bytes
.../principle-of-Doris-Stream-Load/Figure_7_cn.png | Bin 0 -> 109718 bytes
.../principle-of-Doris-Stream-Load/Figure_7_en.png | Bin 0 -> 109718 bytes
.../principle-of-Doris-Stream-Load/Figure_8_cn.png | Bin 0 -> 238659 bytes
.../principle-of-Doris-Stream-Load/Figure_8_en.png | Bin 0 -> 238659 bytes
18 files changed, 310 insertions(+)
diff --git a/blog/principle-of-Doris-Stream-Load.md
b/blog/principle-of-Doris-Stream-Load.md
new file mode 100644
index 000..1ccde7c0d69
--- /dev/null
+++ b/blog/principle-of-Doris-Stream-Load.md
@@ -0,0 +1,155 @@
+---
+{
+'title': 'Doris Stream Load Principle Analysis',
+'summary': "Stream Load, one of the most commonly used data import methods for
Doris users, is a synchronous import method. It allows users to import data
into Doris in batch through HTTP access and returns the results of data
import.",
+'date': '2022-09-08',
+'author': 'Apache Doris',
+'tags': ['Tech Sharing'],
+}
+---
+
+
+
+**Lead：**
+
+Stream Load, one of the most commonly used data import methods for Doris
users, is a synchronous import method. It allows users to import data into
Doris in batch through HTTP access and returns the results of data import. The
user can not only directly judge whether the data import is successful through
the return body of the HTTP request, but also query the results of historical
tasks by executing query SQL on the client.
+
+# **Introduction to Stream Load**
+
+The Doris import (Load) function is to import the user's original data into
the Doris table. And Doris realizes a unified streaming import framework at the
bottom. On this basis, Doris provides a very rich import mode to adapt to
different data sources and data import requirements. Stream Load is one of the
most commonly used data import methods for Doris users. It is a synchronous
import method that allows users to import data in CSV format or JSON format
into Doris in batch through HTT [...]
+
+# 1 Implementation Process
+
+The user submits the HTTP request of Stream Load to the FE, and the FE will
forward the data import request to a BE node through HTTP Redirect, which will
be the Coordinator of this Stream Load task. In this process, the FE node
receiving the request only provides forwarding service. The BE node as the
Coordinator is actually responsible for the entire import job, such as sending
transaction requests to the Master FE, obtaining import execution plans from
the FE, receiving real-time data [...]
+
+In the Coordinator BE, all HTTP requests, including Stream Load requests, are
processed through a thread pool. A Stream Load task is uniquely identified by
the imported Label. The principle block diagram of Stream Load is shown in
Figure 1.
+
+
+
+
+Figure 1 The principle block diagram of Stream Load
+
+The complete execution process of Stream Load is shown in Figure 2:
+
+(1)The user submits the HTTP request of Stream Load to the FE (the user can
also directly submit the HTTP request of Stream Load to the Coordinator BE).
+
+(2)FE, after receiving the Stream Load request submitted by the user, will
perform HTTP Header pa
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 6d51d889466 fix 6d51d889466 is described below commit 6d51d889466811522750cb453a1e7b879f599528 Author: jiafeng.zhang AuthorDate: Mon Sep 26 09:30:59 2022 +0800 fix --- blog/principle-of-Doris-Stream-Load.md | 42 +- .../principle-of-Doris-Stream-Load.md | 40 + 2 files changed, 17 insertions(+), 65 deletions(-) diff --git a/blog/principle-of-Doris-Stream-Load.md b/blog/principle-of-Doris-Stream-Load.md index 1ccde7c0d69..7c37fcec8bb 100644 --- a/blog/principle-of-Doris-Stream-Load.md +++ b/blog/principle-of-Doris-Stream-Load.md @@ -41,10 +41,8 @@ The user submits the HTTP request of Stream Load to the FE, and the FE will forw In the Coordinator BE, all HTTP requests, including Stream Load requests, are processed through a thread pool. A Stream Load task is uniquely identified by the imported Label. The principle block diagram of Stream Load is shown in Figure 1. - - - -Figure 1 The principle block diagram of Stream Load + + The complete execution process of Stream Load is shown in Figure 2: @@ -76,19 +74,12 @@ The complete execution process of Stream Load is shown in Figure 2: (14)The Coordinator BE returns the final result of Stream Load to the user. - - - -Figure 2 The complete execution process of Stream Load - + # 2 Transaction Management Doris ensures the atomicity of data import through Transaction. One Stream Load task corresponds to one transaction. The FE is responsible for the transaction management of Stream Load. The FE receives the Thrift RPC transaction request sent by the Coordinator BE node through the FrontendService. Transaction request types include Begin Transaction, Commit Transaction and Rollback Transaction. The transaction states of Doris include PREPARE, COMMITTED, VISIBLE, and ABORTED. The status flo [...] - - - -Figure 3 The status flow process of the Stream Load transaction + The Coordinator BE node will send a Begin Transaction request to the FE before data import. The FE will check whether the label requested by the Begin Transaction already exists. If the label does not exist in the system, it will open a new transaction for the current label, assign a Transaction ID to the transaction, and set the transaction status to PREPARE, then returns the Transaction ID and the success information of the Begin Transaction to the Coordinator BE. Otherwise, this trans [...] @@ -105,40 +96,25 @@ After importing the execution plan and submitting it to the thread pool of Fragm The PlanFragmentExecutor executes a specific import plan process, which consists of three stages: Prepare, Open, and Close. In the Prepare stage, the import execution plan from the FE is mainly analyzed; In the Open stage, BrokerScanNode and OlapTableSink will be opened. BrokerScanNode is responsible for reading the real-time data of one Batch at a time, and OlapTableSink is responsible for calling BRPC to send the data of each Batch to other Executor BE nodes; In the Close stage, it is [...] - - - -Figure 4 The import execution plan of Stream Load + OlapTableSink is responsible for the data distribution of the Stream Load task. Tables in Doris may have Rollup or Materialized view. Each Table and its Rollup and Materialized view are called an Index. In the process of data distribution, the IndexChannel will maintain a data distribution channel of the Index. The Tablet under the Index may have multiple replicas and are distributed on different BE nodes. The NodeChannel will maintain the data distribution channel of an Executor BE node [...] - - - -Figure 5 The Data distribution channel for Stream Load task + When OlapTableSink distributes data, it will read the data Batch obtained by BrokerScanNode row by row, and add the data row to the IndexChannel of each Index. The Partition and Tablet of the data row can be determined according to the PartitionKey and DistributionKey, and then the corresponding Tablet of the data row in other Index can be calculated according to the order of the Tablet in the Partition. Each Tablet may have multiple replicas distributed on different BE nodes. Therefore, [...] - - - -Figure 6 The data distribution process of the Stream Load task + # 4 **Data
[doris] branch master updated (978dae267e -> 9c03deb150)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 978dae267e [typo](docs)Optimized string and date function doc (#12949) add 9c03deb150 [fix](log)Audit log status is incorrect (#12824) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/doris/qe/ConnectProcessor.java| 7 ++- .../src/main/java/org/apache/doris/qe/MasterOpExecutor.java| 10 ++ fe/fe-core/src/main/java/org/apache/doris/qe/QueryState.java | 3 ++- fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java | 7 +++ .../java/org/apache/doris/service/FrontendServiceImpl.java | 6 ++ gensrc/thrift/FrontendService.thrift | 1 + 6 files changed, 32 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: Spark load import kerberos parameter modification (#12924)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 18433d7105 Spark load import kerberos parameter modification (#12924) 18433d7105 is described below commit 18433d710549ff8ba54afed74e6be6795770a8b3 Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com> AuthorDate: Mon Sep 26 12:24:43 2022 +0800 Spark load import kerberos parameter modification (#12924) Spark load import kerberos parameter modification --- .../import/import-way/spark-load-manual.md | 30 +++--- .../import/import-way/spark-load-manual.md | 30 +++--- 2 files changed, 30 insertions(+), 30 deletions(-) diff --git a/docs/en/docs/data-operate/import/import-way/spark-load-manual.md b/docs/en/docs/data-operate/import/import-way/spark-load-manual.md index 071d0c3401..bd66db6b6f 100644 --- a/docs/en/docs/data-operate/import/import-way/spark-load-manual.md +++ b/docs/en/docs/data-operate/import/import-way/spark-load-manual.md @@ -154,10 +154,10 @@ PROPERTIES working_dir = path, broker = broker_name, broker.property_key = property_value, - hadoop.security.authentication = kerberos, - kerberos_principal = do...@your.com, - kerberos_keytab = /home/doris/my.keytab - kerberos_keytab_content = ASDOWHDLAWIDJHWLDKSALDJSDIWALD + broker.hadoop.security.authentication = kerberos, + broker.kerberos_principal = do...@your.com, + broker.kerberos_keytab = /home/doris/my.keytab + broker.kerberos_keytab_content = ASDOWHDLAWIDJHWLDKSALDJSDIWALD ) -- drop spark resource @@ -194,10 +194,10 @@ REVOKE USAGE_PRIV ON RESOURCE resource_name FROM ROLE role_name - Other parameters are optional, refer to `http://spark.apache.org/docs/latest/configuration.html` - `working_dir`: directory used by ETL. Spark is required when used as an ETL resource. For example: `hdfs://host :port/tmp/doris`. -- `hadoop.security.authentication`: Specify the authentication method as kerberos. -- `kerberos_principal`: Specify the principal of kerberos. -- `kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. -- `kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `kerberos_keytab` configuration. +- `broker.hadoop.security.authentication`: Specify the authentication method as kerberos. +- `broker.kerberos_principal`: Specify the principal of kerberos. +- `broker.kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. +- `broker.kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `broker.kerberos_keytab` configuration. - `broker`: the name of the broker. Spark is required when used as an ETL resource. You need to use the 'alter system add broker' command to complete the configuration in advance. - `broker.property_key`: the authentication information that the broker needs to specify when reading the intermediate file generated by ETL. - `env`: Specify the spark environment variable and support dynamic setting. For example, when the authentication mode of Hadoop is simple, set the Hadoop user name and password @@ -242,10 +242,10 @@ PROPERTIES If Spark load accesses Hadoop cluster resources with Kerberos authentication, we only need to specify the following parameters when creating Spark resources: -- `hadoop.security.authentication`: Specify the authentication method as kerberos. -- `kerberos_principal`: Specify the principal of kerberos. -- `kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. -- `kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `kerberos_keytab` configuration. +- `broker.hadoop.security.authentication`: Specify the authentication method as kerberos. +- `broker.kerberos_principal`: Specify the principal of kerberos. +- `broker.kerberos_keytab`: Specify the path to the keytab file for kerberos. The file must be an absolute path to a file on the server where the broker process is located. And can be accessed by the Broker process. +- `broker.kerberos_keytab_content`: Specify the content of the keytab file in kerberos after base64 encoding. You can choose one of these with `kerberos_keytab` configuration. Example： @@ -264,9 +264
[doris] branch master updated: Optimized materialized view documentation (#12798)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 9afa3cdb19 Optimized materialized view documentation (#12798)
9afa3cdb19 is described below
commit 9afa3cdb1971a292603587104e78608bd911c33d
Author: zy-kkk
AuthorDate: Mon Sep 26 12:25:20 2022 +0800
Optimized materialized view documentation (#12798)
Optimized materialized view documentation
---
.../Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md | 2 +-
.../Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
index 643638870f..2c2a6036da 100644
---
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
+++
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
@@ -113,7 +113,7 @@ duplicate key (k1,k2,k3,k4)
distributed BY hash(k4) buckets 3
properties("replication_num" = "1");
```
-attention：The partition and distributed columns must be key column in mv
+attention：If the materialized view contains partitioned and distributed
columns of the Base table, these columns must be used as key columns in the
materialized view
1. Create a materialized view that contains only the columns of the original
table (k1, k2)
diff --git
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
index 2b552930c7..aa0ae7ed1f 100644
---
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
+++
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
@@ -113,7 +113,7 @@ duplicate key (k1,k2,k3,k4)
distributed BY hash(k4) buckets 3
properties("replication_num" = "1");
```
-注意：分区列和分桶列必须作为物化视图中的key列
+注意：如果物化视图包含了base表的分区列和分桶列,那么这些列必须作为物化视图中的key列
1. 创建一个仅包含原始表 （k1, k2）列的物化视图
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [docs](function)Optimized string function doc (#114)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new fd2b8840b8d [docs](function)Optimized string function doc (#114)
fd2b8840b8d is described below
commit fd2b8840b8d22475493b9be7bc46a688dfad78bd
Author: zy-kkk
AuthorDate: Mon Sep 26 12:45:31 2022 +0800
[docs](function)Optimized string function doc (#114)
* Optimized string function doc
---
.../append_trailing_char_if_absent.md | 2 +-
.../sql-functions/string-functions/ascii.md| 2 +-
.../sql-functions/string-functions/bit_length.md | 2 +-
.../sql-functions/string-functions/char_length.md | 2 +-
.../sql-functions/string-functions/concat.md | 2 +-
.../sql-functions/string-functions/concat_ws.md| 2 +-
.../sql-functions/string-functions/ends_with.md| 2 +-
.../sql-functions/string-functions/find_in_set.md | 2 +-
.../sql-functions/string-functions/hex.md | 2 +-
.../sql-functions/string-functions/instr.md| 2 +-
.../sql-functions/string-functions/lcase.md| 2 +-
.../sql-functions/string-functions/left.md | 2 +-
.../sql-functions/string-functions/length.md | 2 +-
.../sql-functions/string-functions/like/like.md| 2 +-
.../string-functions/like/not_like.md | 2 +-
.../sql-functions/string-functions/locate.md | 2 +-
.../sql-functions/string-functions/lower.md| 2 +-
.../sql-functions/string-functions/lpad.md | 2 +-
.../sql-functions/string-functions/ltrim.md| 2 +-
.../sql-functions/string-functions/money_format.md | 2 +-
.../string-functions/null_or_empty.md | 2 +-
.../string-functions/{reverse.md => parse_url.md} | 32 +++---
.../string-functions/regexp/not_regexp.md | 2 +-
.../string-functions/regexp/regexp.md | 2 +-
.../string-functions/regexp/regexp_extract.md | 2 +-
.../string-functions/regexp/regexp_replace.md | 2 +-
.../sql-functions/string-functions/repeat.md | 2 +-
.../sql-functions/string-functions/replace.md | 2 +-
.../sql-functions/string-functions/reverse.md | 2 +-
.../sql-functions/string-functions/right.md| 2 +-
.../sql-functions/string-functions/rpad.md | 2 +-
.../sql-functions/string-functions/rtrim.md| 24 ++---
.../sql-functions/string-functions/split_part.md | 2 +-
.../sql-functions/string-functions/starts_with.md | 2 +-
.../sql-functions/string-functions/strleft.md | 2 +-
.../sql-functions/string-functions/strright.md | 2 +-
.../sql-functions/string-functions/substring.md| 2 +-
.../sql-functions/string-functions/trim.md | 22 ++--
.../string-functions/{lcase.md => ucase.md}| 12 +++
.../sql-functions/string-functions/unhex.md| 2 +-
.../string-functions/{lower.md => upper.md}| 18 +-
.../append_trailing_char_if_absent.md | 2 +-
.../sql-functions/string-functions/ascii.md| 2 +-
.../sql-functions/string-functions/bit_length.md | 2 +-
.../sql-functions/string-functions/char_length.md | 4 +--
.../sql-functions/string-functions/concat.md | 2 +-
.../sql-functions/string-functions/concat_ws.md| 2 +-
.../sql-functions/string-functions/ends_with.md| 2 +-
.../sql-functions/string-functions/find_in_set.md | 2 +-
.../sql-functions/string-functions/hex.md | 2 +-
.../sql-functions/string-functions/instr.md| 2 +-
.../sql-functions/string-functions/lcase.md| 2 +-
.../sql-functions/string-functions/left.md | 2 +-
.../sql-functions/string-functions/length.md | 2 +-
.../sql-functions/string-functions/like/like.md| 2 +-
.../string-functions/like/not_like.md | 2 +-
.../sql-functions/string-functions/locate.md | 2 +-
.../sql-functions/string-functions/lower.md| 2 +-
.../sql-functions/string-functions/lpad.md | 2 +-
.../sql-functions/string-functions/ltrim.md| 2 +-
.../sql-functions/string-functions/money_format.md | 2 +-
.../string-functions/null_or_empty.md | 2 +-
.../{find_in_set.md => parse_url.md} | 25 +++---
.../string-functions/regexp/not_regexp.md | 2 +-
.../string-functions/regexp/regexp.md | 2 +-
.../string-functions/regexp/regexp_extract.md | 2 +-
.../string-functions/regexp/regexp_replace.md | 2 +-
.../sql-functions/string-functions/repeat.md | 2 +-
.../sql-functions/string-functions/replace.md | 2 +-
.../sql-functions/string-functions/reverse.md | 4 +--
.../sql-functions/string-functions/right.md| 2 +-
.../sql-functions/string-functions/rpad.md | 2 +-
.../string-functions/{ltrim.md
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 427441b6e3c fix 427441b6e3c is described below commit 427441b6e3cbc243803a6334f14140f093b2cc72 Author: jiafeng.zhang AuthorDate: Mon Sep 26 13:56:40 2022 +0800 fix --- .../current/admin-manual/config/fe-config.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/fe-config.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/fe-config.md index f1faea6c609..be0ce9cee95 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/fe-config.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/fe-config.md @@ -29,7 +29,7 @@ under the License. 该文档主要介绍 FE 的相关配置项。 -FE 的配置文件 `fe.conf` 通常存放在 FE 部署路径的 `conf/` 目录下。 而在 0.14 版本中会引入另一个配置文件 `fe_custom.conf`。该配置文件用于记录用户在运行是动态配置并持久化的配置项。 +FE 的配置文件 `fe.conf` 通常存放在 FE 部署路径的 `conf/` 目录下。 而在 0.14 版本中会引入另一个配置文件 `fe_custom.conf`。该配置文件用于记录用户在运行时动态配置并持久化的配置项。 FE 进程启动后，会先读取 `fe.conf` 中的配置项，之后再读取 `fe_custom.conf` 中的配置项。`fe_custom.conf` 中的配置项会覆盖 `fe.conf` 中相同的配置项。 @@ -837,7 +837,7 @@ tablet 状态更新间隔 默认值： storage_flood_stage_usage_percent : 95 (95%) - + storage_flood_stage_left_capacity_bytes : 1 * 1024 * 1024 * 1024 (1GB) 是否可以动态配置：true - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix-doc22 (#115)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new b74558d064c fix-doc22 (#115) b74558d064c is described below commit b74558d064c44bd5b65fe392ea5d4a327a11936b Author: Liqf <109049295+lemonlit...@users.noreply.github.com> AuthorDate: Tue Sep 27 09:34:41 2022 +0800 fix-doc22 (#115) --- sidebars.json | 79 +++ 1 file changed, 41 insertions(+), 38 deletions(-) diff --git a/sidebars.json b/sidebars.json index 87874d7281b..d6d0429d88a 100644 --- a/sidebars.json +++ b/sidebars.json @@ -396,34 +396,35 @@ "type": "category", "label": "Bitmap Functions", "items": [ - "sql-manual/sql-functions/bitmap-functions/bitmap_and", - "sql-manual/sql-functions/bitmap-functions/bitmap_contains", - "sql-manual/sql-functions/bitmap-functions/bitmap_empty", - "sql-manual/sql-functions/bitmap-functions/bitmap_from_string", - "sql-manual/sql-functions/bitmap-functions/bitmap_has_any", - "sql-manual/sql-functions/bitmap-functions/bitmap_has_all", + "sql-manual/sql-functions/bitmap-functions/to_bitmap", "sql-manual/sql-functions/bitmap-functions/bitmap_hash", - "sql-manual/sql-functions/bitmap-functions/bitmap_intersect", + "sql-manual/sql-functions/bitmap-functions/bitmap_from_string", + "sql-manual/sql-functions/bitmap-functions/bitmap_to_string", + "sql-manual/sql-functions/bitmap-functions/bitmap_empty", "sql-manual/sql-functions/bitmap-functions/bitmap_or", - "sql-manual/sql-functions/bitmap-functions/bitmap_count", - "sql-manual/sql-functions/bitmap-functions/bitmap_and_count", - "sql-manual/sql-functions/bitmap-functions/bitmap_or_count", - "sql-manual/sql-functions/bitmap-functions/bitmap_xor_count", + "sql-manual/sql-functions/bitmap-functions/bitmap_and", + "sql-manual/sql-functions/bitmap-functions/bitmap_union", + "sql-manual/sql-functions/bitmap-functions/bitmap_xor", "sql-manual/sql-functions/bitmap-functions/bitmap_not", "sql-manual/sql-functions/bitmap-functions/bitmap_and_not", - "sql-manual/sql-functions/bitmap-functions/bitmap_and_not_count", - "sql-manual/sql-functions/bitmap-functions/bitmap_subset_in_range", "sql-manual/sql-functions/bitmap-functions/bitmap_subset_limit", + "sql-manual/sql-functions/bitmap-functions/bitmap_subset_in_range", "sql-manual/sql-functions/bitmap-functions/sub_bitmap", - "sql-manual/sql-functions/bitmap-functions/bitmap_to_string", - "sql-manual/sql-functions/bitmap-functions/bitmap_union", - "sql-manual/sql-functions/bitmap-functions/bitmap_xor", - "sql-manual/sql-functions/bitmap-functions/to_bitmap", + "sql-manual/sql-functions/bitmap-functions/bitmap_count", + "sql-manual/sql-functions/bitmap-functions/bitmap_and_count", + "sql-manual/sql-functions/bitmap-functions/bitmap_and_not_count", + "sql-manual/sql-functions/bitmap-functions/orthogonal_bitmap_union_count", + "sql-manual/sql-functions/bitmap-functions/bitmap_xor_count", + "sql-manual/sql-functions/bitmap-functions/bitmap_or_count", + "sql-manual/sql-functions/bitmap-functions/bitmap_contains", +
[doris-website] branch master updated (b74558d064c -> 6e1d53a924f)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from b74558d064c fix-doc22 (#115) add 6e1d53a924f add build check No new revisions were added by this update. Summary of changes: .github/workflows/build-check.yml | 63 +++ 1 file changed, 63 insertions(+) create mode 100644 .github/workflows/build-check.yml - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: 1
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 0639286d6a7 1 0639286d6a7 is described below commit 0639286d6a7d27f0decfa738ec31c39d9aff59ba Author: jiafeng.zhang AuthorDate: Tue Sep 27 11:40:05 2022 +0800 1 --- .github/workflows/build-check.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/build-check.yml b/.github/workflows/build-check.yml index 346239a4337..b0de0b78a93 100644 --- a/.github/workflows/build-check.yml +++ b/.github/workflows/build-check.yml @@ -51,13 +51,16 @@ jobs: mkdir i18n/zh-CN/docusaurus-plugin-content-docs/version-dev cp -R doris/docs/zh-CN/docs/* i18n/zh-CN/docusaurus-plugin-content-docs/version-dev/ cp -R doris/docs/dev.json i18n/zh-CN/docusaurus-plugin-content-docs/version-dev.json + mkdir -p community cp -R doris/docs/en/community/* community/ mkdir -p i18n/zh-CN/docusaurus-plugin-content-docs-community/current/ cp -R doris/docs/zh-CN/community/* i18n/zh-CN/docusaurus-plugin-content-docs-community/current/ cp -R doris/docs/sidebarsCommunity.json . + cp -R doris/docs/images static/ npm install -g yarn yarn cache clean yarn && yarn build + rm -rf build \ No newline at end of file - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated (0639286d6a7 -> b9f799383bb)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from 0639286d6a7 1 new e46d56c9e3d add local build docs new b9f799383bb add local docs build The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: i18n/zh-CN/local_build_docs.sh | 51 ++ local_build_docs.sh| 51 ++ 2 files changed, 102 insertions(+) create mode 100644 i18n/zh-CN/local_build_docs.sh create mode 100644 local_build_docs.sh - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] 02/02: add local docs build
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git commit b9f799383bb55817bdf6b4d1bdeaba12b16b9321 Author: jiafeng.zhang AuthorDate: Tue Sep 27 14:33:05 2022 +0800 add local docs build --- local_build_docs.sh | 51 +++ 1 file changed, 51 insertions(+) diff --git a/local_build_docs.sh b/local_build_docs.sh new file mode 100644 index 000..f94ea8406f2 --- /dev/null +++ b/local_build_docs.sh @@ -0,0 +1,51 @@ + + +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +## +# This is a native document format compilation check script +## + +#!/bin/bash + +set -eo pipefail + +git clone https://github.com/apache/doris.git +mkdir versioned_docs/version-dev/ +cp -R doris/docs/en/docs/* versioned_docs/version-dev/ +cp -R doris/docs/sidebars.json versioned_sidebars/version-dev-sidebars.json +mkdir i18n/zh-CN/docusaurus-plugin-content-docs/version-dev +cp -R doris/docs/zh-CN/docs/* i18n/zh-CN/docusaurus-plugin-content-docs/version-dev/ +cp -R doris/docs/dev.json i18n/zh-CN/docusaurus-plugin-content-docs/version-dev.json + +mkdir -p community +cp -R doris/docs/en/community/* community/ +mkdir -p i18n/zh-CN/docusaurus-plugin-content-docs-community/current/ +cp -R doris/docs/zh-CN/community/* i18n/zh-CN/docusaurus-plugin-content-docs-community/current/ +cp -R doris/docs/sidebarsCommunity.json . + +cp -R doris/docs/images static/ +npm install -g yarn +yarn cache clean +yarn && yarn build + + +echo "***" +echo "Docs build success" +echo "***" \ No newline at end of file - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] 01/02: add local build docs
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git commit e46d56c9e3dc42e1c293a266cf432a8d715fb498 Author: jiafeng.zhang AuthorDate: Tue Sep 27 13:52:08 2022 +0800 add local build docs --- i18n/zh-CN/local_build_docs.sh | 51 ++ 1 file changed, 51 insertions(+) diff --git a/i18n/zh-CN/local_build_docs.sh b/i18n/zh-CN/local_build_docs.sh new file mode 100644 index 000..f94ea8406f2 --- /dev/null +++ b/i18n/zh-CN/local_build_docs.sh @@ -0,0 +1,51 @@ + + +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +## +# This is a native document format compilation check script +## + +#!/bin/bash + +set -eo pipefail + +git clone https://github.com/apache/doris.git +mkdir versioned_docs/version-dev/ +cp -R doris/docs/en/docs/* versioned_docs/version-dev/ +cp -R doris/docs/sidebars.json versioned_sidebars/version-dev-sidebars.json +mkdir i18n/zh-CN/docusaurus-plugin-content-docs/version-dev +cp -R doris/docs/zh-CN/docs/* i18n/zh-CN/docusaurus-plugin-content-docs/version-dev/ +cp -R doris/docs/dev.json i18n/zh-CN/docusaurus-plugin-content-docs/version-dev.json + +mkdir -p community +cp -R doris/docs/en/community/* community/ +mkdir -p i18n/zh-CN/docusaurus-plugin-content-docs-community/current/ +cp -R doris/docs/zh-CN/community/* i18n/zh-CN/docusaurus-plugin-content-docs-community/current/ +cp -R doris/docs/sidebarsCommunity.json . + +cp -R doris/docs/images static/ +npm install -g yarn +yarn cache clean +yarn && yarn build + + +echo "***" +echo "Docs build success" +echo "***" \ No newline at end of file - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-flink-connector] branch schemachange-1.14 updated: [Feature] Support Light Schema change for flink1.14 (#64)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch schemachange-1.14
in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git
The following commit(s) were added to refs/heads/schemachange-1.14 by this push:
new b407905 [Feature] Support Light Schema change for flink1.14 (#64)
b407905 is described below
commit b40790537b0fe89d2c4c0aac75ff7476c0345ee2
Author: wudi <676366...@qq.com>
AuthorDate: Wed Sep 28 08:59:55 2022 +0800
[Feature] Support Light Schema change for flink1.14 (#64)
Support Light Schema change for flink1.14
---
flink-doris-connector/pom.xml | 45 ++--
.../doris/flink/cfg/DorisExecutionOptions.java | 2 +-
.../org/apache/doris/flink/rest/RestService.java | 2 +-
.../apache/doris/flink/sink/HttpGetWithEntity.java | 36 +++
.../apache/doris/flink/sink/HttpPutBuilder.java| 12 +-
.../doris/flink/sink/committer/DorisCommitter.java | 1 +
.../doris/flink/sink/writer/DorisStreamLoad.java | 3 +
.../doris/flink/sink/writer/DorisWriter.java | 7 +-
.../sink/writer/JsonDebeziumSchemaSerializer.java | 264 +
.../flink/table/DorisDynamicTableFactory.java | 2 +-
.../apache/doris/flink/CDCSchemaChangeExample.java | 87 +++
.../org/apache/doris/flink/DorisSinkExample.java | 12 +-
.../apache/doris/flink/DorisSinkSQLExample.java| 7 +-
.../apache/doris/flink/DorisSourceSinkExample.java | 40 ++--
.../writer/TestJsonDebeziumSchemaSerializer.java | 151
.../doris/flink/source/DorisSourceExampleTest.java | 1 -
.../flink/source/reader/DorisSourceReaderTest.java | 2 +
.../doris/flink/utils/DateToStringConverter.java | 147
18 files changed, 759 insertions(+), 62 deletions(-)
diff --git a/flink-doris-connector/pom.xml b/flink-doris-connector/pom.xml
index 6206b4a..bde8d37 100644
--- a/flink-doris-connector/pom.xml
+++ b/flink-doris-connector/pom.xml
@@ -62,9 +62,9 @@ under the License.
-${env.scala.version}
-${env.flink.version}
-${env.flink.minor.version}
+2.12
+1.14.4
+1.14
0.13.0
5.0.0
3.8.1
@@ -251,33 +251,20 @@ under the License.
2.13.3
-org.apache.logging.log4j
-log4j-web
-${log4j2.version}
-
-
-
-org.apache.logging.log4j
-log4j-api
-${log4j2.version}
-
-
-
-org.apache.logging.log4j
-log4j-core
-${log4j2.version}
-
-
-
-org.apache.logging.log4j
-log4j-slf4j-impl
-${log4j2.version}
+org.slf4j
+slf4j-api
+1.7.25
-
org.slf4j
slf4j-log4j12
-1.7.9
+1.7.25
+test
+
+
+log4j
+log4j
+1.2.17
@@ -310,6 +297,12 @@ under the License.
test
+
+com.ververica
+flink-connector-mysql-cdc
+2.2.1
+test
+
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/DorisExecutionOptions.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/DorisExecutionOptions.java
index 2daf5e1..102a7ee 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/DorisExecutionOptions.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/DorisExecutionOptions.java
@@ -73,7 +73,7 @@ public class DorisExecutionOptions implements Serializable {
public static DorisExecutionOptions defaults() {
Properties properties = new Properties();
properties.setProperty("format", "json");
-properties.setProperty("strip_outer_array", "true");
+properties.setProperty("read_json_by_line", "true");
return new Builder().setStreamLoadProp(properties).build();
}
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/RestService.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/RestService.java
index 734bfdb..5732dc8 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/RestService.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/RestService.java
@@ -442,7 +442,7 @@ public class RestService implements Serializable {
public static boolean isUniqueKeyType(DorisOptions options,
DorisReadOptions readOptions, Logger logger)
throws DorisRuntimeException {
try {
-return "UNIQUE_KEYS_TYPE".equals(getSchema(options, readOptions,
logger).g
[doris-flink-connector] branch branch-for-flink-before-1.13 updated: update (#67)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch branch-for-flink-before-1.13
in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git
The following commit(s) were added to refs/heads/branch-for-flink-before-1.13
by this push:
new 154bcf4 update (#67)
154bcf4 is described below
commit 154bcf4161ba29a90d9d0603c241dda85ff85901
Author: wudi <676366...@qq.com>
AuthorDate: Wed Sep 28 08:59:12 2022 +0800
update (#67)
Co-authored-by: wudi <>
---
.../java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java| 2 +-
.../java/org/apache/doris/flink/table/DorisDynamicTableSink.java | 5 +++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
index 0ea11d0..a055195 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/cfg/GenericDorisSinkFunction.java
@@ -59,7 +59,7 @@ public class GenericDorisSinkFunction extends
RichSinkFunction
@Override
public void snapshotState(FunctionSnapshotContext context) throws
Exception {
-
+outputFormat.flush();
}
@Override
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicTableSink.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicTableSink.java
index cccdb45..813669b 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicTableSink.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/table/DorisDynamicTableSink.java
@@ -19,10 +19,11 @@ package org.apache.doris.flink.table;
import org.apache.doris.flink.cfg.DorisExecutionOptions;
import org.apache.doris.flink.cfg.DorisOptions;
import org.apache.doris.flink.cfg.DorisReadOptions;
+import org.apache.doris.flink.cfg.GenericDorisSinkFunction;
import org.apache.flink.table.api.TableSchema;
import org.apache.flink.table.connector.ChangelogMode;
import org.apache.flink.table.connector.sink.DynamicTableSink;
-import org.apache.flink.table.connector.sink.OutputFormatProvider;
+import org.apache.flink.table.connector.sink.SinkFunctionProvider;
import org.apache.flink.types.RowKind;
/**
@@ -65,7 +66,7 @@ public class DorisDynamicTableSink implements
DynamicTableSink {
.setExecutionOptions(executionOptions)
.setFieldDataTypes(tableSchema.getFieldDataTypes())
.setFieldNames(tableSchema.getFieldNames());
-return OutputFormatProvider.of(builder.build());
+return SinkFunctionProvider.of(new
GenericDorisSinkFunction(builder.build()));
}
@Override
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Bug](Datax)Fix bug that the dataxwriter will drop column when convert map to json (#13042)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 29fc167548 [Bug](Datax)Fix bug that the dataxwriter will drop column
when convert map to json (#13042)
29fc167548 is described below
commit 29fc167548718c96d59e0a67ec06d32db763bad2
Author: Wilson-β <40427110+wilson...@users.noreply.github.com>
AuthorDate: Thu Sep 29 11:37:10 2022 +0800
[Bug](Datax)Fix bug that the dataxwriter will drop column when convert map
to json (#13042)
* fix bug that when value is null，toJSONString will drop this key value.
---
docs/zh-CN/docs/ecosystem/datax.md | 2 ++
.../com/alibaba/datax/plugin/writer/doriswriter/DorisJsonCodec.java| 3 ++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/docs/zh-CN/docs/ecosystem/datax.md
b/docs/zh-CN/docs/ecosystem/datax.md
index feb8429d3f..46006122b8 100644
--- a/docs/zh-CN/docs/ecosystem/datax.md
+++ b/docs/zh-CN/docs/ecosystem/datax.md
@@ -180,6 +180,7 @@ PROPERTIES (
"password": "xxx",
"postSql": [],
"preSql": [],
+"format":"csv|json",
"loadProps": {
},
"maxBatchRows" : 30,
@@ -191,5 +192,6 @@ PROPERTIES (
}
}
```
+> 注意：```format``` 参数默认值：```csv```，如果想要使用```json```格式，需要在配置中指定```format```
为```json```.
4.执行datax任务，具体参考
[datax官网](https://github.com/alibaba/DataX/blob/master/userGuid.md)
diff --git
a/extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisJsonCodec.java
b/extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisJsonCodec.java
index 664b9d3629..063503bbb3 100644
---
a/extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisJsonCodec.java
+++
b/extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisJsonCodec.java
@@ -19,6 +19,7 @@ package com.alibaba.datax.plugin.writer.doriswriter;
import com.alibaba.datax.common.element.Record;
import com.alibaba.fastjson.JSON;
+import com.alibaba.fastjson.serializer.SerializerFeature;
import java.util.HashMap;
import java.util.List;
@@ -45,6 +46,6 @@ public class DorisJsonCodec extends DorisCodec {
rowMap.put(fieldName, this.convertColumn(row.getColumn(idx)));
++idx;
}
-return JSON.toJSONString(rowMap);
+return JSON.toJSONString(rowMap, SerializerFeature.WriteMapNullValue);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](doc)add orthogonal bitmap function Note. (#118)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new deb92781dec [typo](doc)add orthogonal bitmap function Note. (#118)
deb92781dec is described below
commit deb92781decb4d3deb791cf2d5af7c622f4eeb98
Author: Liqf <109049295+lemonlit...@users.noreply.github.com>
AuthorDate: Fri Sep 30 15:23:37 2022 +0800
[typo](doc)add orthogonal bitmap function Note. (#118)
---
docs/advanced/orthogonal-bitmap-manual.md | 2 +
.../sql-functions/math-functions/cbrt.md | 57 ++
.../current/advanced/orthogonal-bitmap-manual.md | 2 +
.../sql-functions/math-functions/cbrt.md | 57 ++
4 files changed, 118 insertions(+)
diff --git a/docs/advanced/orthogonal-bitmap-manual.md
b/docs/advanced/orthogonal-bitmap-manual.md
index d08307e7490..42750ffa00e 100644
--- a/docs/advanced/orthogonal-bitmap-manual.md
+++ b/docs/advanced/orthogonal-bitmap-manual.md
@@ -87,6 +87,8 @@ Note: the first column represents the user tags, which have
been converted from
When loading data, vertically cut the bitmap value range of the user. For
example, the hid value of the user ID in the range of 1-500 is the same,
and the row with the same HID value will be allocated into a sub-bucket, so
that the bitmap value in each sub-bucket is orthogonal. On the UDAF
implementation of bitmap, the orthogonal feature of bitmap value in the bucket
can be used to perform intersection union calculation, and the calculation
results will be shuffled to the top node fo [...]
+Note: The orthogonal bitmap function cannot be used in the partitioned table.
Because the partitions of the partitioned table are orthogonal, the data
between partitions cannot be guaranteed to be orthogonal, so the calculation
result cannot be estimated.
+
orthogonal_bitmap_intersect
The bitmap intersection function
diff --git a/docs/sql-manual/sql-functions/math-functions/cbrt.md
b/docs/sql-manual/sql-functions/math-functions/cbrt.md
new file mode 100644
index 000..24fcba137e3
--- /dev/null
+++ b/docs/sql-manual/sql-functions/math-functions/cbrt.md
@@ -0,0 +1,57 @@
+---
+{
+"title": "cbrt",
+"language": "en"
+}
+---
+
+
+
+## cbrt
+
+### description
+ Syntax
+
+`DOUBLE cbrt(DOUBLE x)`
+Returns the cube root of x.
+
+### example
+
+```
+mysql> select cbrt(8);
++---+
+| cbrt(8.0) |
++---+
+| 2 |
++---+
+mysql> select cbrt(2.0);
+++
+| cbrt(2.0) |
+++
+| 1.2599210498948734 |
+++
+mysql> select cbrt(-1000.0);
++---+
+| cbrt(-1000.0) |
++---+
+| -10 |
++---+
+```
+
+### keywords
+ CBRT
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/orthogonal-bitmap-manual.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/orthogonal-bitmap-manual.md
index 7db218cdc2d..a1cc19afa86 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/orthogonal-bitmap-manual.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/orthogonal-bitmap-manual.md
@@ -89,6 +89,8 @@ user_id = to_bitmap(tmp_user_id)
load数据时，对用户bitmap值range范围纵向切割，例如，用户id在1-500范围内的hid值相同，hid值相同的行会分配到一个分桶内，如此每个分桶内到的bitmap都是正交的。可以利用桶内bitmap值正交特性，进行交并集计算，计算结果会被shuffle至top节点聚合。
+注：正交bitmap函数不能用在分区表，因为分区表分区内正交，分区之间的数据是无法保证正交的，则计算结果也是无法预估的。
+
bitmap_orthogonal_intersect
求bitmap交集函数
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/math-functions/cbrt.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/math-functions/cbrt.md
new file mode 100644
index 000..73178f2003d
--- /dev/null
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/math-functions/cbrt.md
@@ -0,0 +1,57 @@
+---
+{
+"title": "cbrt",
+"language": "zh-CN"
+}
+---
+
+
+
+## cbrt
+
+### description
+ Syntax
+
+`DOUBLE cbrt(DOUBLE x)`
+返回`x`的立方根.
+
+### example
+
+```
+mysql> select cbrt(8);
++---+
+| cbrt(8.0) |
++---+
+| 2 |
++---+
+mysql> select cbrt(2.0);
+++
+| cbrt(2.0) |
+++
+| 1.2599210498948734 |
+++
+mysql> select cbrt(-1000.0);
++---+
+| cbrt(-1000.0) |
++---+
+| -10 |
++---+
+```
+
+### keywords
+ CBRT
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new bbcead68e91 fix bbcead68e91 is described below commit bbcead68e910a15b9758816ce72367b57995f353 Author: jiafeng.zhang AuthorDate: Fri Sep 30 15:36:13 2022 +0800 fix --- docs/admin-manual/maint-monitor/metadata-operation.md | 2 +- .../current/admin-manual/maint-monitor/metadata-operation.md| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin-manual/maint-monitor/metadata-operation.md b/docs/admin-manual/maint-monitor/metadata-operation.md index 6af164ab4f4..fdf33872090 100644 --- a/docs/admin-manual/maint-monitor/metadata-operation.md +++ b/docs/admin-manual/maint-monitor/metadata-operation.md @@ -28,7 +28,7 @@ under the License. This document focuses on how to manage Doris metadata in a real production environment. It includes the proposed deployment of FE nodes, some commonly used operational methods, and common error resolution methods. -For the time being, read the [Doris metadata design document](../../../community/design/metadata-design) to understand how Doris metadata works. +For the time being, read the [Doris metadata design document](/community/design/metadata-design) to understand how Doris metadata works. ## Important tips diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md index 57ba71c5d3c..86dda4da225 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md @@ -28,7 +28,7 @@ under the License. 本文档主要介绍在实际生产环境中，如何对 Doris 的元数据进行管理。包括 FE 节点建议的部署方式、一些常用的操作方法、以及常见错误的解决方法。 -在阅读本文当前，请先阅读 [Doris 元数据设计文档](../../../../docusaurus-plugin-content-docs-community/current/design/metadata-design) 了解 Doris 元数据的工作原理。 +在阅读本文当前，请先阅读 [Doris 元数据设计文档](/community/design/metadata-design) 了解 Doris 元数据的工作原理。 ## 重要提示 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4872af0bd78 fix 4872af0bd78 is described below commit 4872af0bd780eed311b04e83f0418c7f6742eb0e Author: jiafeng.zhang AuthorDate: Fri Sep 30 15:41:22 2022 +0800 fix --- docs/admin-manual/maint-monitor/metadata-operation.md | 2 +- .../current/admin-manual/maint-monitor/metadata-operation.md| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin-manual/maint-monitor/metadata-operation.md b/docs/admin-manual/maint-monitor/metadata-operation.md index fdf33872090..bc2439ff586 100644 --- a/docs/admin-manual/maint-monitor/metadata-operation.md +++ b/docs/admin-manual/maint-monitor/metadata-operation.md @@ -32,7 +32,7 @@ For the time being, read the [Doris metadata design document](/community/design/ ## Important tips -* Current metadata design is not backward compatible. That is, if the new version has a new metadata structure change (you can see whether there is a new VERSION in the `FeMetaVersion.java` file in the FE code), it is usually impossible to roll back to the old version after upgrading to the new version. Therefore, before upgrading FE, be sure to test metadata compatibility according to the operations in the [Upgrade Document](../upgrade.md). +* Current metadata design is not backward compatible. That is, if the new version has a new metadata structure change (you can see whether there is a new VERSION in the `FeMetaVersion.java` file in the FE code), it is usually impossible to roll back to the old version after upgrading to the new version. Therefore, before upgrading FE, be sure to test metadata compatibility according to the operations in the [Upgrade Document](../../admin-manual/cluster-management/upgrade). ## Metadata catalog structure diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md index 86dda4da225..aa1752c93d4 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metadata-operation.md @@ -32,7 +32,7 @@ under the License. ## 重要提示 -* 当前元数据的设计是无法向后兼容的。即如果新版本有新增的元数据结构变动（可以查看 FE 代码中的 `FeMetaVersion.java` 文件中是否有新增的 VERSION），那么在升级到新版本后，通常是无法再回滚到旧版本的。所以，在升级 FE 之前，请务必按照 [升级文档](../../admin-manual/cluster-management/upgrade.md) 中的操作，测试元数据兼容性。 +* 当前元数据的设计是无法向后兼容的。即如果新版本有新增的元数据结构变动（可以查看 FE 代码中的 `FeMetaVersion.java` 文件中是否有新增的 VERSION），那么在升级到新版本后，通常是无法再回滚到旧版本的。所以，在升级 FE 之前，请务必按照 [升级文档](../../admin-manual/cluster-management/upgrade) 中的操作，测试元数据兼容性。 ## 元数据目录结构 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: slack link update
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 8343504deec slack link update
8343504deec is described below
commit 8343504deecde2ce2382ba625bbb11c6e738b86b
Author: jiafeng.zhang
AuthorDate: Fri Sep 30 15:52:26 2022 +0800
slack link update
slack link update
---
src/pages/index.tsx| 2 +-
src/theme/Footer/index.tsx | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/pages/index.tsx b/src/pages/index.tsx
index 206b83eda9d..81419ff24b3 100644
--- a/src/pages/index.tsx
+++ b/src/pages/index.tsx
@@ -26,7 +26,7 @@ export default function Home(): JSX.Element {
},
{
label: 'Slack',
-link:
'https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-1co4h0c90-Kfu5MUgaTi4n~lAwNXfbuQ',
+link:
'https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-1h153f1ar-sTJB_QahY1SHvZdtPFoIOQ',
type: 'default',
},
];
diff --git a/src/theme/Footer/index.tsx b/src/theme/Footer/index.tsx
index d2d043c0825..c07fbcf2d2b 100644
--- a/src/theme/Footer/index.tsx
+++ b/src/theme/Footer/index.tsx
@@ -46,7 +46,7 @@ function Footer() {
https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-1co4h0c90-Kfu5MUgaTi4n~lAwNXfbuQ";
+
href="https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-1h153f1ar-sTJB_QahY1SHvZdtPFoIOQ";
title="slack"
className="item"
>
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](docs)Metadata Operations and Maintenance link error (#13090)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 90512ebd59 [typo](docs)Metadata Operations and Maintenance link error (#13090) 90512ebd59 is described below commit 90512ebd59dba499f4de4912b26423a23fbce01b Author: jiafeng.zhang AuthorDate: Wed Oct 5 22:58:24 2022 +0800 [typo](docs)Metadata Operations and Maintenance link error (#13090) * Metadata Operations and Maintenance link error --- README.md| 2 +- docs/en/docs/admin-manual/maint-monitor/metadata-operation.md| 4 ++-- docs/zh-CN/docs/admin-manual/maint-monitor/metadata-operation.md | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 3cd99e377b..6e25912b4e 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ under the License. [](https://ci-builds.apache.org/job/Doris/job/doris_daily_disable_vectorized) [](https://ci-builds.apache.org/job/Doris/job/doris_daily_enable_vectorized) [](https://github.com/apache/doris) -[](https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-1h153f1ar-sTJB_QahY1SHvZdtPFoIOQ) +[ [](https://gitter.im/apache-doris/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [](https://doris.apache.org/docs/get-starting/) [](https://doris.apache.org/zh-CN/docs/get-starting/) diff --git a/docs/en/docs/admin-manual/maint-monitor/metadata-operation.md b/docs/en/docs/admin-manual/maint-monitor/metadata-operation.md index db047000b1..bc2439ff58 100644 --- a/docs/en/docs/admin-manual/maint-monitor/metadata-operation.md +++ b/docs/en/docs/admin-manual/maint-monitor/metadata-operation.md @@ -28,11 +28,11 @@ under the License. This document focuses on how to manage Doris metadata in a real production environment. It includes the proposed deployment of FE nodes, some commonly used operational methods, and common error resolution methods. -For the time being, read the [Doris metadata design document](../../design/metadata-design.md) to understand how Doris metadata works. +For the time being, read the [Doris metadata design document](/community/design/metadata-design) to understand how Doris metadata works. ## Important tips -* Current metadata design is not backward compatible. That is, if the new version has a new metadata structure change (you can see whether there is a new VERSION in the `FeMetaVersion.java` file in the FE code), it is usually impossible to roll back to the old version after upgrading to the new version. Therefore, before upgrading FE, be sure to test metadata compatibility according to the operations in the [Upgrade Document](../upgrade.md). +* Current metadata design is not backward compatible. That is, if the new version has a new metadata structure change (you can see whether there is a new VERSION in the `FeMetaVersion.java` file in the FE code), it is usually impossible to roll back to the old version after upgrading to the new version. Therefore, before upgrading FE, be sure to test metadata compatibility according to the operations in the [Upgrade Document](../../admin-manual/cluster-management/upgrade). ## Metadata catalog structure diff --git a/docs/zh-CN/docs/admin-manual/maint-monitor/metadata-operation.md b/docs/zh-CN/docs/admin-manual/maint-monitor/metadata-operation.md index f0008a781a..aa1752c93d 100644 --- a/docs/zh-CN/docs/admin-manual/maint-monitor/metadata-operation.md +++ b/docs/zh-CN/docs/admin-manual/maint-monitor/metadata-operation.md @@ -28,11 +28,11 @@ under the License. 本文档主要介绍在实际生产环境中，如何对 Doris 的元数据进行管理。包括 FE 节点建议的部署方式、一些常用的操作方法、以及常见错误的解决方法。 -在阅读本文当前，请先阅读 [Doris 元数据设计文档](../../design/metadata-design.md) 了解 Doris 元数据的工作原理。 +在阅读本文当前，请先阅读 [Doris 元数据设计文档](/community/design/metadata-design) 了解 Doris 元数据的工作原理。 ## 重要提示 -* 当前元数据的设计是无法向后兼容的。即如果新版本有新增的元数据结构变动（可以查看 FE 代码中的 `FeMetaVersion.java` 文件中是否有新增的 VERSION），那么在
[doris] 01/01: fix slack link
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch hf200012-patch-3 in repository https://gitbox.apache.org/repos/asf/doris.git commit af090af07dc26f10d5a7a8cfbc6b6106aa7cb950 Author: jiafeng.zhang AuthorDate: Wed Oct 5 23:02:34 2022 +0800 fix slack link --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6e25912b4e..3cd99e377b 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ under the License. [](https://ci-builds.apache.org/job/Doris/job/doris_daily_disable_vectorized) [](https://ci-builds.apache.org/job/Doris/job/doris_daily_enable_vectorized) [](https://github.com/apache/doris) -[ +[](https://join.slack.com/t/apachedoriscommunity/shared_invite/zt-1h153f1ar-sTJB_QahY1SHvZdtPFoIOQ) [](https://gitter.im/apache-doris/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [](https://doris.apache.org/docs/get-starting/) [](https://doris.apache.org/zh-CN/docs/get-starting/) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch hf200012-patch-3 created (now af090af07d)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch hf200012-patch-3 in repository https://gitbox.apache.org/repos/asf/doris.git at af090af07d fix slack link This branch includes the following new commits: new af090af07d fix slack link The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix doris on es link
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 148a44fd5e7 fix doris on es link
148a44fd5e7 is described below
commit 148a44fd5e7b8761c6894514f1dfe17fe8be7038
Author: jiafeng.zhang
AuthorDate: Fri Oct 7 17:32:24 2022 +0800
fix doris on es link
---
src/pages/learning/index.tsx | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/pages/learning/index.tsx b/src/pages/learning/index.tsx
index b7161022e50..115b617d7be 100644
--- a/src/pages/learning/index.tsx
+++ b/src/pages/learning/index.tsx
@@ -182,7 +182,7 @@ const sitemapList = [
list: [
{
title: Doris on ES,
-link:
'https://doris.apache.orgdocs/dev/ecosystem/external-table/doris-on-es',
+link: '/docs/dev/ecosystem/external-table/doris-on-es',
},
{
title: Doris on Hudi,
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix data cache sidebar error (#119)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new b3308b78d27 fix data cache sidebar error (#119)
b3308b78d27 is described below
commit b3308b78d27014f98e0002837d6fd0c45b471eec
Author: zy-kkk
AuthorDate: Fri Oct 7 17:33:58 2022 +0800
fix data cache sidebar error (#119)
fix data cache sidebar error
---
i18n/zh-CN/docusaurus-plugin-content-docs/current.json | 2 +-
sidebars.json | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
b/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
index b1d9cde0022..c3aa673266f 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
@@ -67,7 +67,7 @@
"message": "Join 优化",
"description": "The label for category Join Optimization in sidebar docs"
},
- "sidebar.docs.category.Date Cache": {
+ "sidebar.docs.category.Data Cache": {
"message": "数据缓存",
"description": "The label for category Date Cache in sidebar docs"
},
diff --git a/sidebars.json b/sidebars.json
index d6d0429d88a..881e38ff728 100644
--- a/sidebars.json
+++ b/sidebars.json
@@ -132,7 +132,7 @@
},
{
"type": "category",
-"label": "Date Cache",
+"label": "Data Cache",
"items": [
"advanced/cache/partition-cache"
]
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: fix data cache sidebar error (#13137)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 71399ed771 fix data cache sidebar error (#13137)
71399ed771 is described below
commit 71399ed771e5429db8771e6873c8aa09ef46816b
Author: zy-kkk
AuthorDate: Fri Oct 7 17:45:21 2022 +0800
fix data cache sidebar error (#13137)
fix data cache sidebar error
---
docs/dev.json | 2 +-
docs/sidebars.json | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/dev.json b/docs/dev.json
index fda8436369..07bee9fc6c 100644
--- a/docs/dev.json
+++ b/docs/dev.json
@@ -67,7 +67,7 @@
"message": "Join 优化",
"description": "The label for category Join Optimization in sidebar docs"
},
- "sidebar.docs.category.Date Cache": {
+ "sidebar.docs.category.Data Cache": {
"message": "数据缓存",
"description": "The label for category Date Cache in sidebar docs"
},
diff --git a/docs/sidebars.json b/docs/sidebars.json
index 4ac1763263..c0971d7d9e 100644
--- a/docs/sidebars.json
+++ b/docs/sidebars.json
@@ -132,7 +132,7 @@
},
{
"type": "category",
-"label": "Date Cache",
+"label": "Data Cache",
"items": [
"advanced/cache/partition-cache"
]
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix doc link
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 99b376a60d9 fix doc link 99b376a60d9 is described below commit 99b376a60d9f615be90453c36073fe850c893415 Author: jiafeng.zhang AuthorDate: Sat Oct 8 11:35:16 2022 +0800 fix doc link --- docs/ecosystem/logstash.md | 2 +- docs/ecosystem/udf/contribute-udf.md | 2 +- docs/faq/install-faq.md| 5 + docs/get-starting/get-starting.md | 2 +- .../Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- .../Create/CREATE-MATERIALIZED-VIEW.md | 2 +- .../Drop/DROP-DATABASE.md | 2 +- .../Load/BROKER-LOAD.md| 22 +++--- .../Load/CREATE-SYNC-JOB.md| 2 +- .../sql-reference/Show-Statements/SHOW-STATUS.md | 2 +- .../Show-Statements/SHOW-STREAM-LOAD.md| 2 +- 12 files changed, 23 insertions(+), 26 deletions(-) diff --git a/docs/ecosystem/logstash.md b/docs/ecosystem/logstash.md index d5dfe17a41c..dbf7901625e 100644 --- a/docs/ecosystem/logstash.md +++ b/docs/ecosystem/logstash.md @@ -28,7 +28,7 @@ under the License. This plugin is used to output data to Doris for logstash, use the HTTP protocol to interact with the Doris FE Http interface, and import data through Doris's stream load. -[Learn more about Doris Stream Load ](../data-operate/import/import-way/stream-load-manual.html) +[Learn more about Doris Stream Load ](../../data-operate/import/import-way/stream-load-manual) [Learn more about Doris](../) diff --git a/docs/ecosystem/udf/contribute-udf.md b/docs/ecosystem/udf/contribute-udf.md index 075bd41a5fe..1db7a8b9642 100644 --- a/docs/ecosystem/udf/contribute-udf.md +++ b/docs/ecosystem/udf/contribute-udf.md @@ -121,4 +121,4 @@ The user manual needs to include: UDF function definition description, applicabl When you meet the conditions and prepare the code, you can contribute UDF to the Doris community after the document. Simply submit the request (PR) on [Github](https://github.com/apache/incubator-doris). See the specific submission method: [Pull Request (PR)](https://help.github.com/articles/about-pull-requests/). -Finally, when the PR assessment is passed and merged. Congratulations, your UDF becomes a third-party UDF supported by Doris. You can check it out in the ecological expansion section of [Doris official website](/en)~. +Finally, when the PR assessment is passed and merged. Congratulations, your UDF becomes a third-party UDF supported by Doris. You can check it out in the ecological expansion section of [Doris official website](/)~. diff --git a/docs/faq/install-faq.md b/docs/faq/install-faq.md index dd3fb581ef6..d877c6adc0e 100644 --- a/docs/faq/install-faq.md +++ b/docs/faq/install-faq.md @@ -83,7 +83,7 @@ Here we provide 3 ways to solve this problem: 3. Manually migrate data using the API - Doris provides [HTTP API](../admin-manual/http-actions/tablet-migration-action.md), which can manually specify the migration of data shards on one disk to another disk. + Doris provides [HTTP API](../admin-manual/http-actions/tablet-migration-action), which can manually specify the migration of data shards on one disk to another disk. ### Q5. How to read FE/BE logs correctly? @@ -155,9 +155,6 @@ In many cases, we need to troubleshoot problems through logs. The format and vie Logs starting with F are Fatal logs. For example, F0916 , indicating the Fatal log on September 16th. Fatal logs usually indicate a program assertion error, and an assertion error will directly cause the process to exit (indicating a bug in the program). Welcome to the WeChat group, github discussion or dev mail group for help. - 4. Minidump(removed) - - Mindump is a function added after Doris version 0.15. For details, please refer to [document](https://doris.apache.org/zh-CN/developer-guide/minidump.html). 2. FE diff --git a/docs/get-starting/get-starting.md b/docs/get-starting/get-starting.md index 221f7b88fbe..1ae61dbea67 100644 --- a/docs/get-starting/get-starting.md +++ b/docs/get-starting/get-starting.md @@ -122,7 +122,7 @@ mysql -uroot -P9030 -h127.0.0.1 >Note: > ->1. The root user used here is the default user built into doris, and is also the super administrator user, see [Rights Management](...) /admin-manual/privilege-ldap/user-privilege) +>1. The root user used here is the default user built into doris, and is also the super administrator user, see [Rights Management](../admin-manual/privilege-ldap/user-privilege) >2. -P: Here is our query port to connect to Doris, the default po
[doris-website] branch master updated (99b376a60d9 -> 59606726aa0)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from 99b376a60d9 fix doc link add fb18b0c181a fix add 59606726aa0 fix No new revisions were added by this update. Summary of changes: docs/get-starting/get-starting.md | 6 +++--- .../Data-Manipulation-Statements/Load/CREATE-SYNC-JOB.md| 2 +- .../Data-Manipulation-Statements/Load/CREATE-SYNC-JOB.md| 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix link (#124)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new ef42cb667bb fix link (#124) ef42cb667bb is described below commit ef42cb667bb04d1f4bc386ae2787a17dcedd795d Author: FreeOnePlus <54164178+freeonep...@users.noreply.github.com> AuthorDate: Sat Oct 8 15:46:19 2022 +0800 fix link (#124) fix link --- docs/releasenotes/release-1.1.0.md| 4 +--- .../sql-reference/Data-Definition-Statements/Drop/DROP-DATABASE.md| 2 +- .../current/ecosystem/doris-manager/space-list.md | 2 +- .../current/releasenotes/release-1.1.0.md | 4 +--- .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-REPLACE.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../sql-reference/Data-Definition-Statements/Drop/DROP-DATABASE.md| 2 +- .../Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- 9 files changed, 9 insertions(+), 13 deletions(-) diff --git a/docs/releasenotes/release-1.1.0.md b/docs/releasenotes/release-1.1.0.md index c58a1ff46ff..e9e517d7c00 100644 --- a/docs/releasenotes/release-1.1.0.md +++ b/docs/releasenotes/release-1.1.0.md @@ -42,9 +42,7 @@ BE binary file has been renamed from palo_be to doris_be . Please pay attention The storage format of earlier versions of Apache Doris was Segment V1. In version 0.12, we had implemented Segment V2 as a new storage format, which introduced Bitmap indexes, memory tables, page cache, dictionary compression, delayed materialization and many other features. Starting from version 0.13, the default storage format for newly created tables is Segment V2, while maintaining compatibility with the Segment V1 format. -In order to ensure the maintainability of the code structure and reduce the additional learning and development costs caused by redundant historical codes, we have decided to no longer support the Segment v1 storage format from the next version. It is expected that this part of the code will be deleted in the Apache Doris 1.2 version, and all users who are still using the Segment V1 storage format must complete the data format conversion in version 1.1. Please refer to the following link [...] - -[https://doris.apache.org/zh-CN/docs/1.0/administrator-guide/segment-v2-usage](https://doris.apache.org/zh-CN/docs/1.0/administrator-guide/segment-v2-usage) +In order to ensure the maintainability of the code structure and reduce the additional learning and development costs caused by redundant historical codes, we have decided to no longer support the Segment v1 storage format from the next version. It is expected that this part of the code will be deleted in the Apache Doris 1.2 version. ### Normal Upgrade diff --git a/docs/sql-manual/sql-reference/Data-Definition-Statements/Drop/DROP-DATABASE.md b/docs/sql-manual/sql-reference/Data-Definition-Statements/Drop/DROP-DATABASE.md index 05fdaed73ae..ef0ab3273d1 100644 --- a/docs/sql-manual/sql-reference/Data-Definition-Statements/Drop/DROP-DATABASE.md +++ b/docs/sql-manual/sql-reference/Data-Definition-Statements/Drop/DROP-DATABASE.md @@ -41,7 +41,7 @@ DROP DATABASE [IF EXISTS] db_name [FORCE]; illustrate: -- During the execution of DROP DATABASE, the deleted database can be recovered through the RECOVER statement. See the [RECOVER](../../Data-Definition-Statements/Backup-and-Restore/RECOVER) statement for details +- During the execution of DROP DATABASE, the deleted database can be recovered through the RECOVER statement. See the [RECOVER](../../Database-Administration-Statements/RECOVER) statement for details - If you execute DROP DATABASE FORCE, the system will not check the database for unfinished transactions, the database will be deleted directly and cannot be recovered, this operation is generally not recommended ### Example diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/doris-manager/space-list.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/doris-manager/space-list.md index 78abf4e980c..fdbb578f8e5 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/doris-manager/space-list.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/doris-manager/space-list.md @@ -104,7 +104,7 @@ ssh agen...@xx.xxx.xx.xx 1. 代码包路径 - 通过Doris Manager 进行集群部署时，需要提供已编译好的 Doris 安装包，您可以通过 Doris 源码自行编译，或使用官方提供的[二进制版本](https://doris.apache.org/zh-CN/downloads/downloads.html)。 + 通过Doris Manager 进行集群部署时，需要提供已编译好的 Doris 安装包，您可以通过 Doris 源码自行编译，或使用官方提供的[二进制版本](https://doris.apache.org/zh-CN/download)。 `Doris Manager 将通过 http 方式拉取Doris安装包，若您需要自建 http 服务，请参考文档底部-自建
[doris-website] branch master updated: fix docs 404 url (#122)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 96324cdb5ab fix docs 404 url (#122) 96324cdb5ab is described below commit 96324cdb5abc3dbc13cc2d83a0c66d0c48681a00 Author: zy-kkk AuthorDate: Sat Oct 8 17:12:11 2022 +0800 fix docs 404 url (#122) Update SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md --- docs/ecosystem/logstash.md | 4 +- docs/ecosystem/udf/contribute-udf.md | 10 +- docs/faq/install-faq.md| 1 - .../Load/CREATE-SYNC-JOB.md| 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 120 + .../sql-reference/Show-Statements/SHOW-STATUS.md | 86 +-- .../current/ecosystem/logstash.md | 4 +- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Load/CREATE-SYNC-JOB.md| 2 +- 9 files changed, 136 insertions(+), 95 deletions(-) diff --git a/docs/ecosystem/logstash.md b/docs/ecosystem/logstash.md index dbf7901625e..c6ab151d1f4 100644 --- a/docs/ecosystem/logstash.md +++ b/docs/ecosystem/logstash.md @@ -28,9 +28,9 @@ under the License. This plugin is used to output data to Doris for logstash, use the HTTP protocol to interact with the Doris FE Http interface, and import data through Doris's stream load. -[Learn more about Doris Stream Load ](../../data-operate/import/import-way/stream-load-manual) +[Learn more about Doris Stream Load ](../data-operate/import/import-way/stream-load-manual) -[Learn more about Doris](../) +[Learn more about Doris](/) ## Install and compile diff --git a/docs/ecosystem/udf/contribute-udf.md b/docs/ecosystem/udf/contribute-udf.md index 1db7a8b9642..0492c7e0139 100644 --- a/docs/ecosystem/udf/contribute-udf.md +++ b/docs/ecosystem/udf/contribute-udf.md @@ -14,7 +14,7 @@ to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an @@ -73,8 +73,8 @@ The user manual needs to include: UDF function definition description, applicabl ``` ├── docs -│ └── zh-CN -│ └──extending-doris +│ └── zh-CN +│ └──extending-doris │ └──udf │└──contrib │ ├── udf-simple-manual.md @@ -83,8 +83,8 @@ The user manual needs to include: UDF function definition description, applicabl ``` ├── docs -│ └── en -│ └──extending-doris +│ └── en +│ └──extending-doris │ └──udf │└──contrib │ ├── udf-simple-manual.md diff --git a/docs/faq/install-faq.md b/docs/faq/install-faq.md index d877c6adc0e..e945ceb6c79 100644 --- a/docs/faq/install-faq.md +++ b/docs/faq/install-faq.md @@ -155,7 +155,6 @@ In many cases, we need to troubleshoot problems through logs. The format and vie Logs starting with F are Fatal logs. For example, F0916 , indicating the Fatal log on September 16th. Fatal logs usually indicate a program assertion error, and an assertion error will directly cause the process to exit (indicating a bug in the program). Welcome to the WeChat group, github discussion or dev mail group for help. - 2. FE FE is a java process, and the robustness is due to the C/C++ program. Usually the reason for FE to hang up may be OOM (Out-of-Memory) or metadata write failure. These errors usually have an error stack in fe.log or fe.out. Further investigation is required based on the error stack information. diff --git a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-SYNC-JOB.md b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-SYNC-JOB.md index 09302c36e4e..99b592c7619 100644 --- a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-SYNC-JOB.md +++ b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-SYNC-JOB.md @@ -36,7 +36,7 @@ The data synchronization (Sync Job) function supports users to submit a resident Currently, the data synchronization job only supports connecting to Canal, obtaining the parsed Binlog data from the Canal Server and importing it into Doris. -Users can view the data synchronization job status through [SHOW SYNC JOB](../../Show-Statements/SHOW-SYNC-JOB). +Users can view the data synchronization job status through [SHOW SYNC JOB](../../../../sql-manual/sql-reference/Show-Statements/SHOW-SYNC-JOB). grammar: diff --
[doris] branch master updated (ece4a6c194 -> e0044e5a5f)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from ece4a6c194 [doc][fix](multi-catalog) add doc for multi catalog and fix refresh bug (#13097) add e0044e5a5f [typo](docs)Sql doc link fix (#13151) No new revisions were added by this update. Summary of changes: docs/en/docs/get-starting/get-starting.md | 6 +++--- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Create/CREATE-MATERIALIZED-VIEW.md | 2 +- .../Load/BROKER-LOAD.md| 18 .../Load/CREATE-SYNC-JOB.md| 2 +- .../sql-reference/Show-Statements/SHOW-STATUS.md | 2 +- .../Show-Statements/SHOW-STREAM-LOAD.md| 2 +- .../Load/BROKER-LOAD.md| 24 +++--- .../Load/CREATE-SYNC-JOB.md| 2 +- .../Show-Statements/SHOW-STREAM-LOAD.md| 2 +- 10 files changed, 31 insertions(+), 31 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix link 404 error (#135)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new bfdbe0cca55 fix link 404 error (#135) bfdbe0cca55 is described below commit bfdbe0cca5530ca47b6eef01635f8e7e8ee7d9cb Author: lsy3993 <110876560+lsy3...@users.noreply.github.com> AuthorDate: Sun Oct 9 09:28:48 2022 +0800 fix link 404 error (#135) --- .../administrator-guide/alter-table/alter-table-rollup.md | 2 +- versioned_docs/version-0.15/administrator-guide/config/be_config.md | 2 +- .../administrator-guide/load-data/routine-load-manual.md| 2 +- .../version-0.15/administrator-guide/operation/disk-capacity.md | 2 +- .../administrator-guide/operation/metadata-operation.md | 6 +++--- versioned_docs/version-0.15/administrator-guide/outfile.md | 2 +- .../version-0.15/administrator-guide/resource-management.md | 2 +- versioned_docs/version-0.15/best-practices/star-schema-benchmark.md | 2 +- 8 files changed, 10 insertions(+), 10 deletions(-) diff --git a/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-rollup.md b/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-rollup.md index a3df4c8dc26..f8ceca3bef4 100644 --- a/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-rollup.md +++ b/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-rollup.md @@ -27,7 +27,7 @@ under the License. # Rollup Users can speed up queries by creating rollup tables. For the concept and usage of Rollup, please refer to [Data - Model, ROLLUP and Prefix Index](../../getting-started/data-model-rollup_EN.md) and + Model, ROLLUP and Prefix Index](../../getting-started/data-model-rollup) and [Rollup and query](../../getting-started/hit-the-rollup_EN.md). This document focuses on how to create a Rollup job, as well as some considerations and frequently asked questions about creating a Rollup. diff --git a/versioned_docs/version-0.15/administrator-guide/config/be_config.md b/versioned_docs/version-0.15/administrator-guide/config/be_config.md index aec6bfa1c8b..47eed01cb81 100644 --- a/versioned_docs/version-0.15/administrator-guide/config/be_config.md +++ b/versioned_docs/version-0.15/administrator-guide/config/be_config.md @@ -456,7 +456,7 @@ Cgroups assigned to doris ### `doris_max_scan_key_num` * Type: int -* Description: Used to limit the maximum number of scan keys that a scan node can split in a query request. When a conditional query request reaches the scan node, the scan node will try to split the conditions related to the key column in the query condition into multiple scan key ranges. After that, these scan key ranges will be assigned to multiple scanner threads for data scanning. A larger value usually means that more scanner threads can be used to increase the parallelism of the s [...] +* Description: Used to limit the maximum number of scan keys that a scan node can split in a query request. When a conditional query request reaches the scan node, the scan node will try to split the conditions related to the key column in the query condition into multiple scan key ranges. After that, these scan key ranges will be assigned to multiple scanner threads for data scanning. A larger value usually means that more scanner threads can be used to increase the parallelism of the s [...] * Default value: 1024 When the concurrency cannot be improved in high concurrency scenarios, try to reduce this value and observe the impact. diff --git a/versioned_docs/version-0.15/administrator-guide/load-data/routine-load-manual.md b/versioned_docs/version-0.15/administrator-guide/load-data/routine-load-manual.md index 35b3cf8a6c1..5fe5558626c 100644 --- a/versioned_docs/version-0.15/administrator-guide/load-data/routine-load-manual.md +++ b/versioned_docs/version-0.15/administrator-guide/load-data/routine-load-manual.md @@ -243,7 +243,7 @@ You can only view tasks that are currently running, and tasks that have ended an ### Alter job -Users can modify jobs that have been created. Specific instructions can be viewed through the `HELP ALTER ROUTINE LOAD;` command. Or refer to [ALTER ROUTINE LOAD](../../sql-reference/sql-statements/Data%20Manipulation/alter-routine-load.md). +Users can modify jobs that have been created. Specific instructions can be viewed through the `HELP ALTER ROUTINE LOAD;` command. Or refer to [ALTER ROUTINE LOAD](../../sql-reference/sql-statements/Data-Manipulation/alter-routine-load). ### Job Control diff --git a/versioned_docs/version-0.15/administrator-guide/operation/disk-capacity.md b/versioned_docs/version-0.15/administrator-guide/operation/disk-capacity.md index 71027d9f6e3..515ce4b1737 100644 --- a/versioned_docs/version-0.15/administrator
[doris-website] branch master updated: fix 404 links (#125)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new b3726741d56 fix 404 links (#125)
b3726741d56 is described below
commit b3726741d563cf750eae4f75928f5b650b77be03
Author: DongLiang-0 <46414265+donglian...@users.noreply.github.com>
AuthorDate: Sun Oct 9 09:33:24 2022 +0800
fix 404 links (#125)
* fix 404 links
---
.../current/data-operate/import/import-scenes/external-storage-load.md | 2 +-
.../current/data-operate/import/import-scenes/jdbc-load.md | 2 +-
.../docusaurus-plugin-content-docs/current/data-table/basic-usage.md| 2 +-
.../docusaurus-plugin-content-docs/current/data-table/hit-the-rollup.md | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/external-storage-load.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/external-storage-load.md
index c6051ecd504..1dd619cde04 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/external-storage-load.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/external-storage-load.md
@@ -86,7 +86,7 @@ Hdfs load 创建导入语句，导入方式和[Broker Load](../../../data-operat
3. 查看导入状态
- Broker load 是一个异步的导入方式，具体导入结果可以通过[SHOW
LOAD](../../../sql-manual/sql-reference/Show-Statements/SHOW-LOAD.html#show-load)命令查看
+ Broker load 是一个异步的导入方式，具体导入结果可以通过[SHOW
LOAD](../../../sql-manual/sql-reference/Show-Statements/SHOW-LOAD)命令查看
```
mysql> show load order by createtime desc limit 1\G;
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/jdbc-load.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/jdbc-load.md
index c0012300fda..dfb67028400 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/jdbc-load.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-scenes/jdbc-load.md
@@ -160,4 +160,4 @@ public class DorisJDBCDemo {
前面提到，我们建议在使用 INSERT 导入数据时，采用 ”批“ 的方式进行导入，而不是单条插入。
- 同时，我们可以为每次 INSERT 操作设置一个 Label。通过 [Label
机制](./load-atomicity.html#label-机制) 可以保证操作的幂等性和原子性，最终做到数据的不丢不重。关于 INSERT 中
Label 的具体用法，可以参阅
[INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT.md)
文档。
+ 同时，我们可以为每次 INSERT 操作设置一个 Label。通过 [Label 机制](./load-atomicity)
可以保证操作的幂等性和原子性，最终做到数据的不丢不重。关于 INSERT 中 Label 的具体用法，可以参阅
[INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT)
文档。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/basic-usage.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/basic-usage.md
index 3e2447a35f3..5b658252870 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/basic-usage.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/basic-usage.md
@@ -130,7 +130,7 @@ CREATE DATABASE example_db;
>SHOW CREATE ROUTINE LOAD
> ```
-数据库创建完成之后，可以通过 [SHOW
DATABASES](../sql-manual/sql-reference/Show-Statements/SHOW-DATABASES.html#show-databases)
查看数据库信息。
+数据库创建完成之后，可以通过 [SHOW
DATABASES](../sql-manual/sql-reference/Show-Statements/SHOW-DATABASES) 查看数据库信息。
```sql
mysql> SHOW DATABASES;
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/hit-the-rollup.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/hit-the-rollup.md
index 68d5ebba9ee..79eb4529ff3 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/hit-the-rollup.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/hit-the-rollup.md
@@ -44,7 +44,7 @@ ROLLUP 表的基本作用，在于在 Base 表的基础上，获得更粗粒度
1. 示例1：获得每个用户的总消费
-接 **[数据模型Aggregate 模型](./data-model.html#Aggregate模型)**小节的**示例2**，Base 表结构如下：
+接 **[数据模型Aggregate 模型](./data-model)**小节的**示例2**，Base 表结构如下：
| ColumnName | Type| AggregationType | Comment|
| --- | --- | --- | -- |
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs)Fix jump link 404 in jdbc load.md (#128)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 700f31c2b5b [typo](docs)Fix jump link 404 in jdbc load.md (#128) 700f31c2b5b is described below commit 700f31c2b5b14368938aa00f0d23b0c2130b3c2b Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Sun Oct 9 09:34:52 2022 +0800 [typo](docs)Fix jump link 404 in jdbc load.md (#128) * [typo](docs)Fix jump link 404 --- docs/data-operate/import/import-scenes/jdbc-load.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/data-operate/import/import-scenes/jdbc-load.md b/docs/data-operate/import/import-scenes/jdbc-load.md index 5dc6a2fd5c6..a74f1c9a6b7 100644 --- a/docs/data-operate/import/import-scenes/jdbc-load.md +++ b/docs/data-operate/import/import-scenes/jdbc-load.md @@ -160,4 +160,4 @@ Please note the following: As mentioned earlier, we recommend that when using INSERT to import data, use the "batch" method to import, rather than a single insert. - At the same time, we can set a Label for each INSERT operation. Through the [Label mechanism](./load-atomicity.html#label-mechanism), the idempotency and atomicity of operations can be guaranteed, and the data will not be lost or heavy in the end. For the specific usage of Label in INSERT, you can refer to the [INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT.md) document. + At the same time, we can set a Label for each INSERT operation. Through the [Label mechanism](./load-atomicity.md#label-mechanism), the idempotency and atomicity of operations can be guaranteed, and the data will not be lost or heavy in the end. For the specific usage of Label in INSERT, you can refer to the [INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT.md) document. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs)fix error url (#129)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 48e751853e7 [typo](docs)fix error url (#129) 48e751853e7 is described below commit 48e751853e77bc08b8f08e4490bbdacf05ec2bf0 Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com> AuthorDate: Sun Oct 9 09:35:34 2022 +0800 [typo](docs)fix error url (#129) * fix error url * fix error url Co-authored-by: jiafeng.zhang --- docs/advanced/alter-table/replace-table.md| 2 +- docs/advanced/resource.md | 2 +- docs/data-operate/import/import-scenes/external-storage-load.md | 2 +- docs/data-operate/import/import-scenes/jdbc-load.md | 3 ++- docs/data-table/basic-usage.md| 8 .../admin-manual/maint-monitor/tablet-repair-and-balance.md | 2 +- .../current/advanced/alter-table/replace-table.md | 2 +- .../docusaurus-plugin-content-docs/current/advanced/resource.md | 2 +- .../current/data-operate/import/import-scenes/jdbc-load.md| 2 +- .../current/data-table/basic-usage.md | 6 +++--- 10 files changed, 16 insertions(+), 15 deletions(-) diff --git a/docs/advanced/alter-table/replace-table.md b/docs/advanced/alter-table/replace-table.md index ad1fc22b319..204b395a44d 100644 --- a/docs/advanced/alter-table/replace-table.md +++ b/docs/advanced/alter-table/replace-table.md @@ -69,4 +69,4 @@ If `swap` is `false`, the operation is as follows: 1. Atomic Overwrite Operation -In some cases, the user wants to be able to rewrite the data of a certain table, but if it is dropped and then imported, there will be a period of time in which the data cannot be viewed. At this time, the user can first use the `CREATE TABLE LIKE` statement to create a new table with the same structure, import the new data into the new table, and replace the old table atomically through the replacement operation to achieve the goal. For partition level atomic overwrite operation, pl [...] +In some cases, the user wants to be able to rewrite the data of a certain table, but if it is dropped and then imported, there will be a period of time in which the data cannot be viewed. At this time, the user can first use the `CREATE TABLE LIKE` statement to create a new table with the same structure, import the new data into the new table, and replace the old table atomically through the replacement operation to achieve the goal. For partition level atomic overwrite operation, pl [...] diff --git a/docs/advanced/resource.md b/docs/advanced/resource.md index 494ed7b2008..885240d1903 100644 --- a/docs/advanced/resource.md +++ b/docs/advanced/resource.md @@ -74,7 +74,7 @@ The following shows how the two resources are used. `spark.hadoop.fs.defaultFS`: required when master is yarn. -Other parameters are optional, refer to: http://spark.apache.org/docs/latest/configuration.html. +Other parameters are optional, refer to: http://spark.apache.org/docs/latest/configuration.html # If spark is used for ETL, also need to specify the following parameters: diff --git a/docs/data-operate/import/import-scenes/external-storage-load.md b/docs/data-operate/import/import-scenes/external-storage-load.md index 4909e0cd03e..9c9358d12fd 100644 --- a/docs/data-operate/import/import-scenes/external-storage-load.md +++ b/docs/data-operate/import/import-scenes/external-storage-load.md @@ -82,7 +82,7 @@ Hdfs load creates an import statement. The import method is basically the same a 3. Check import status - Broker load is an asynchronous import method. The specific import results can be accessed through [SHOW LOAD](../../../sql-manual/sql-reference/Show-Statements/SHOW-LOAD.html#show-load) command to view + Broker load is an asynchronous import method. The specific import results can be accessed through [SHOW LOAD](../../../sql-manual/sql-reference/Show-Statements/SHOW-LOAD) command to view ``` mysql> show load order by createtime desc limit 1\G; diff --git a/docs/data-operate/import/import-scenes/jdbc-load.md b/docs/data-operate/import/import-scenes/jdbc-load.md index a74f1c9a6b7..f38c7771ba8 100644 --- a/docs/data-operate/import/import-scenes/jdbc-load.md +++ b/docs/data-operate/import/import-scenes/jdbc-load.md @@ -35,7 +35,7 @@ The INSERT statement is used in a similar way to the INSERT statement used in da * INSERT INTO table VALUES(...) -Here we only introduce the second way. For a detailed description of the INSERT command, see the [INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT.md) command documentation. +Here we only introduce the second way. For a
[doris-website] branch master updated: doc_fix_404_221008 (#131)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new ec9f09e1617 doc_fix_404_221008 (#131) ec9f09e1617 is described below commit ec9f09e1617de9fba413ba2fac05409688b8a69f Author: catpineapple <42031973+catpineap...@users.noreply.github.com> AuthorDate: Sun Oct 9 09:38:07 2022 +0800 doc_fix_404_221008 (#131) --- docs/install/install-deploy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/install/install-deploy.md b/docs/install/install-deploy.md index 627f7fd2bb0..344adbf167e 100644 --- a/docs/install/install-deploy.md +++ b/docs/install/install-deploy.md @@ -158,7 +158,7 @@ BROKER does not currently have, nor does it need, priority\_networks. Broker's s By default, doris is case-sensitive. If there is a need for case-insensitive table names, you need to set it before cluster initialization. The table name case sensitivity cannot be changed after cluster initialization is completed. -See the section on `lower_case_table_names` variables in [Variables](../advanced/variables.md) for details. +See the section on `lower_case_table_names` variables in [Variables](../advanced/variables) for details. ## Cluster deployment - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix error url in version 0.15 (#134)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new d1d1bb9aed1 fix error url in version 0.15 (#134) d1d1bb9aed1 is described below commit d1d1bb9aed16038ac392c3520bf267c76af49949 Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com> AuthorDate: Sun Oct 9 09:51:44 2022 +0800 fix error url in version 0.15 (#134) Co-authored-by: jiafeng.zhang --- .../alter-table/alter-table-bitmap-index.md | 20 ++-- .../alter-table/alter-table-bitmap-index.md | 18 +- .../alter-table/alter-table-rollup.md| 3 ++- 3 files changed, 21 insertions(+), 20 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md index 5e7853c0df6..7ddf145f557 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md @@ -32,34 +32,34 @@ under the License. * bitmap index：位图索引，是一种快速数据结构，能够加快查询速度 ## 原理介绍 -创建和删除本质上是一个 schema change 的作业，具体细节可以参照 [Schema Change](alter-table-schema-change)。 +创建和删除本质上是一个 schema change 的作业，具体细节可以参照 [Schema Change](./alter-table-schema-change)。 ## 语法 index 创建和修改相关语法有两种形式，一种集成与 alter table 语句中，另一种是使用单独的 create/drop index 语法 1. 创建索引 -创建索引的的语法可以参见 [CREATE INDEX](../../sql-reference/sql-statements/Data%20Definition/CREATE%20INDEX.html) -或 [ALTER TABLE](../../sql-reference/sql-statements/Data%20Definition/ALTER%20TABLE.html) 中bitmap 索引相关的操作, -也可以通过在创建表时指定bitmap 索引，参见[CREATE TABLE](../../sql-reference/sql-statements/Data%20Definition/CREATE%20TABLE.html) +创建索引的的语法可以参见 [CREATE INDEX](../../sql-reference/sql-statements/Data-Definition/CREATE-INDEX) +或 [ALTER TABLE](../../sql-reference/sql-statements/Data-Definition/ALTER-TABLE) 中bitmap 索引相关的操作, +也可以通过在创建表时指定bitmap 索引，参见[CREATE TABLE](../../sql-reference/sql-statements/Data-Definition/CREATE-TABLE) 2. 查看索引 -参照[SHOW INDEX](../../sql-reference/sql-statements/Administration/SHOW%20INDEX.html) +参照[SHOW INDEX](../../sql-reference/sql-statements/Administration/SHOW-INDEX) 3. 删除索引 -参照[DROP INDEX](../../sql-reference/sql-statements/Data%20Definition/DROP%20INDEX.html) -或者 [ALTER TABLE](../../sql-reference/sql-statements/Data%20Definition/ALTER%20TABLE.html) 中bitmap 索引相关的操作 +参照[DROP INDEX](../../sql-reference/sql-statements/Data-Definition/DROP-INDEX) +或者 [ALTER TABLE](../../sql-reference/sql-statements/Data-Definition/ALTER-TABLE) 中bitmap 索引相关的操作 ## 创建作业 -参照 schema change 文档 [Schema Change](alter-table-schema-change.html) +参照 schema change 文档 [Schema Change](./alter-table-schema-change) ## 查看作业 -参照 schema change 文档 [Schema Change](alter-table-schema-change.html) +参照 schema change 文档 [Schema Change](./alter-table-schema-change) ## 取消作业 -参照 schema change 文档 [Schema Change](alter-table-schema-change.html) +参照 schema change 文档 [Schema Change](./alter-table-schema-change) ## 注意事项 * 目前索引仅支持 bitmap 类型的索引。 diff --git a/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md b/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md index 34506e5e339..95c1da79f28 100644 --- a/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md +++ b/versioned_docs/version-0.15/administrator-guide/alter-table/alter-table-bitmap-index.md @@ -33,32 +33,32 @@ This document focuses on how to create an index job, as well as some considerati ## Basic Principles Creating and dropping index is essentially a schema change job. For details, please refer to -[Schema Change](alter-table-schema-change.html). +[Schema Change](./alter-table-schema-change). ## Syntax There are two forms of index creation and modification related syntax, one is integrated with alter table statement, and the other is using separate create/drop index syntax 1. Create Index -Please refer to [CREATE INDEX](../../sql-reference/sql-statements/Data%20Definition/CREATE%20INDEX.html) -or [ALTER TABLE](../../sql-reference/sql-statements/Data%20Definition/ALTER%20TABLE.html), -You can also specify a bitmap index when creating a table, Please refer to [CREATE TABLE](../../sql-reference/sql-statements/Data%20Definition/CREATE%20TABLE.html) +Please refer to [CREATE INDEX](../../sql-reference/sql-statements/Data-Definition/CREATE-INDEX) +or [ALTER TABLE](../../sql-reference/sql-statements/Data-Definition/ALTER
[doris-website] branch master updated (d1d1bb9aed1 -> 908db203c14)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git from d1d1bb9aed1 fix error url in version 0.15 (#134) add 908db203c14 [typo](docs)Fix jump link 404 in external storage load.md (#130) No new revisions were added by this update. Summary of changes: docs/data-operate/import/import-scenes/external-storage-load.md | 6 +++--- docs/data-operate/import/import-scenes/jdbc-load.md | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs) Fix jump link 404 (#127)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 6f04f8b0227 [typo](docs) Fix jump link 404 (#127) 6f04f8b0227 is described below commit 6f04f8b0227c664e86bdb8aa619ea588a774cd8b Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Sun Oct 9 09:57:14 2022 +0800 [typo](docs) Fix jump link 404 (#127) Fix jump link 404 in kafka load.md --- docs/data-operate/import/import-scenes/kafka-load.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/data-operate/import/import-scenes/kafka-load.md b/docs/data-operate/import/import-scenes/kafka-load.md index d2bf8901887..e43a20c8100 100644 --- a/docs/data-operate/import/import-scenes/kafka-load.md +++ b/docs/data-operate/import/import-scenes/kafka-load.md @@ -62,7 +62,7 @@ After the upload is complete, you can view the uploaded files through the [SHOW ### Create a routine import job -For specific commands to create routine import tasks, see [ROUTINE LOAD](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-ROUTINE-LOAD.html ) command manual. Here is an example: +For specific commands to create routine import tasks, see [ROUTINE LOAD](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-ROUTINE-LOAD.md ) command manual. Here is an example: 1. Access the Kafka cluster without authentication - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (e0044e5a5f -> c53d2d6a8b)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from e0044e5a5f [typo](docs)Sql doc link fix (#13151) add c53d2d6a8b install deploy doc fix (#13177) No new revisions were added by this update. Summary of changes: docs/en/docs/install/install-deploy.md | 125 + 1 file changed, 1 insertion(+), 124 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (c53d2d6a8b -> 555f9520e3)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from c53d2d6a8b install deploy doc fix (#13177) add 555f9520e3 fix community module error url (#13182) No new revisions were added by this update. Summary of changes: docs/en/community/how-to-contribute/how-to-contribute.md | 2 +- docs/en/community/release-and-verify/release-doris-manager.md | 2 +- docs/en/community/release-and-verify/release-verify.md| 6 +++--- docs/zh-CN/community/release-and-verify/release-verify.md | 6 +++--- 4 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: be config fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new fb0dbf7f6ab be config fix fb0dbf7f6ab is described below commit fb0dbf7f6ab0ce462bf3e5a2aca7beaeb70f4cbf Author: jiafeng.zhang AuthorDate: Sun Oct 9 11:47:25 2022 +0800 be config fix --- docs/admin-manual/config/be-config.md | 8 ++-- .../current/admin-manual/config/be-config.md | 4 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/admin-manual/config/be-config.md b/docs/admin-manual/config/be-config.md index e5316c03235..66282b1acfb 100644 --- a/docs/admin-manual/config/be-config.md +++ b/docs/admin-manual/config/be-config.md @@ -1171,9 +1171,9 @@ Shard size of StoragePageCache, the value must be power of two. It's recommended * /home/disk1/doris.HDD : The storage medium is HDD; * /home/disk2/doris.SSD : The storage medium is SSD; * /home/disk2/doris : The storage medium is HDD HDD(default) - + eg.2: `storage_root_path=/home/disk1/doris,medium:hdd;/home/disk2/doris,medium:ssd` - + * /home/disk1/doris,medium:hdd The storage medium is HDD; * /home/disk2/doris,medium:ssdThe storage medium is SSD; @@ -1294,6 +1294,8 @@ Default: 300 Update interval of tablet state cache, unit: second +The RPC timeout for sending a Batch (1024 lines) during import. The default is 60 seconds. Since this RPC may involve writing multiple batches of memory, the RPC timeout may be caused by writing batches, so this timeout can be adjusted to reduce timeout errors (such as send batch fail errors). Also, if you increase the write_buffer_size configuration, you need to increase this parameter as well. + ### `tablet_writer_ignore_eovercrowded` * Type: bool @@ -1421,6 +1423,8 @@ Default: 104857600 The size of the buffer before flashing +Imported data is first written to a memory block on the BE, and only written back to disk when this memory block reaches the threshold. The default size is 100MB. too small a threshold may result in a large number of small files on the BE. This threshold can be increased to reduce the number of files. However, too large a threshold may cause RPC timeouts, see the configuration notes below. + ### `zone_map_row_num_threshold` * Type: int32 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md index 04efab651d2..47b85bff9c1 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/config/be-config.md @@ -1316,6 +1316,8 @@ tablet状态缓存的更新间隔，单位：秒 在远程BE 中打开tablet writer的 rpc 超时。 操作时间短，可设置短超时时间 +导入过程中，发送一个 Batch（1024行）的 RPC 超时时间。默认 60 秒。因为该 RPC 可能涉及多个 分片内存块的写盘操作，所以可能会因为写盘导致 RPC 超时，可以适当调整这个超时时间来减少超时错误（如 send batch fail 错误）。同时，如果调大 write_buffer_size 配置，也需要适当调大这个参数。 + ### `tablet_writer_ignore_eovercrowded` * 类型：bool @@ -1442,6 +1444,8 @@ webserver默认工作线程数 刷写前缓冲区的大小 +导入数据在 BE 上会先写入到一个内存块，当这个内存块达到阈值后才会写回磁盘。默认大小是 100MB。过小的阈值可能导致 BE 上存在大量的小文件。可以适当提高这个阈值减少文件数量。但过大的阈值可能导致 RPC 超时。 + ### `zone_map_row_num_threshold` * 类型： int32 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (33fe389d62 -> 8f36f8b83a)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 33fe389d62 [regression](datev2) Add regression tests for datev2 (#13040) add 8f36f8b83a Add be Parameter Description(#13201) No new revisions were added by this update. Summary of changes: docs/en/docs/admin-manual/config/be-config.md| 8 ++-- docs/zh-CN/docs/admin-manual/config/be-config.md | 4 2 files changed, 10 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (9c64fde8f5 -> 207e913b55)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 9c64fde8f5 [tools](banchmark) upgrade date type (#13197) add 207e913b55 fix the bad link fo delete-recover.md (#13203) No new revisions were added by this update. Summary of changes: docs/zh-CN/docs/admin-manual/data-admin/delete-recover.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](docs)Fix the jump link 404 in delete recover.md (#13156)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new e5fbecc621 [typo](docs)Fix the jump link 404 in delete recover.md (#13156) e5fbecc621 is described below commit e5fbecc621bef7ce786724240e279d7a050dfef5 Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Sun Oct 9 16:12:34 2022 +0800 [typo](docs)Fix the jump link 404 in delete recover.md (#13156) * [typo](docs)Fix the jump link 404 in delete-recover.md --- docs/en/docs/admin-manual/data-admin/delete-recover.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/docs/admin-manual/data-admin/delete-recover.md b/docs/en/docs/admin-manual/data-admin/delete-recover.md index f02c4ada15..e9944ae9cd 100644 --- a/docs/en/docs/admin-manual/data-admin/delete-recover.md +++ b/docs/en/docs/admin-manual/data-admin/delete-recover.md @@ -50,4 +50,4 @@ RECOVER PARTITION p1 FROM example_tbl; ## More Help -For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RECOVER.md) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help. +For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-reference/Database-Administration-Statements/RECOVER) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](docs)fix docs 404 url (#13157)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new b9516b50c1 [typo](docs)fix docs 404 url (#13157) b9516b50c1 is described below commit b9516b50c11d11ba934140cb20b75e555a0468d6 Author: zy-kkk AuthorDate: Sun Oct 9 20:02:48 2022 +0800 [typo](docs)fix docs 404 url (#13157) * fix docs 404 url --- docs/en/docs/ecosystem/logstash.md | 4 +- docs/en/docs/ecosystem/udf/contribute-udf.md | 4 +- docs/en/docs/faq/install-faq.md| 4 - .../Alter/ALTER-TABLE-ROLLUP.md| 1 - .../Drop/DROP-DATABASE.md | 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 120 + .../sql-reference/Show-Statements/SHOW-STATUS.md | 17 ++- docs/zh-CN/docs/ecosystem/logstash.md | 2 +- .../Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Drop/DROP-DATABASE.md | 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- 12 files changed, 137 insertions(+), 25 deletions(-) diff --git a/docs/en/docs/ecosystem/logstash.md b/docs/en/docs/ecosystem/logstash.md index d5dfe17a41..c6ab151d1f 100644 --- a/docs/en/docs/ecosystem/logstash.md +++ b/docs/en/docs/ecosystem/logstash.md @@ -28,9 +28,9 @@ under the License. This plugin is used to output data to Doris for logstash, use the HTTP protocol to interact with the Doris FE Http interface, and import data through Doris's stream load. -[Learn more about Doris Stream Load ](../data-operate/import/import-way/stream-load-manual.html) +[Learn more about Doris Stream Load ](../data-operate/import/import-way/stream-load-manual) -[Learn more about Doris](../) +[Learn more about Doris](/) ## Install and compile diff --git a/docs/en/docs/ecosystem/udf/contribute-udf.md b/docs/en/docs/ecosystem/udf/contribute-udf.md index 075bd41a5f..c7c01035ae 100644 --- a/docs/en/docs/ecosystem/udf/contribute-udf.md +++ b/docs/en/docs/ecosystem/udf/contribute-udf.md @@ -14,7 +14,7 @@ to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an @@ -121,4 +121,4 @@ The user manual needs to include: UDF function definition description, applicabl When you meet the conditions and prepare the code, you can contribute UDF to the Doris community after the document. Simply submit the request (PR) on [Github](https://github.com/apache/incubator-doris). See the specific submission method: [Pull Request (PR)](https://help.github.com/articles/about-pull-requests/). -Finally, when the PR assessment is passed and merged. Congratulations, your UDF becomes a third-party UDF supported by Doris. You can check it out in the ecological expansion section of [Doris official website](/en)~. +Finally, when the PR assessment is passed and merged. Congratulations, your UDF becomes a third-party UDF supported by Doris. You can check it out in the ecological expansion section of [Doris official website](/)~. diff --git a/docs/en/docs/faq/install-faq.md b/docs/en/docs/faq/install-faq.md index a8f805e9c2..b8f5f88537 100644 --- a/docs/en/docs/faq/install-faq.md +++ b/docs/en/docs/faq/install-faq.md @@ -155,10 +155,6 @@ In many cases, we need to troubleshoot problems through logs. The format and vie Logs starting with F are Fatal logs. For example, F0916 , indicating the Fatal log on September 16th. Fatal logs usually indicate a program assertion error, and an assertion error will directly cause the process to exit (indicating a bug in the program). Welcome to the WeChat group, github discussion or dev mail group for help. - 4. Minidump(removed) - - Mindump is a function added after Doris version 0.15. For details, please refer to [document](https://doris.apache.org/zh-CN/developer-guide/minidump.html). - 2. FE FE is a java process, and the robustness is better than the C/C++ program. Usually the reason for FE to hang up may be OOM (Out-of-Memory) or metadata write failure. These errors usually have an error stack in fe.log or fe.out. Further investigation is required based on the error stack information. diff --git a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md index 2991aa24b4..596e7cfcc5 100644 --- a/docs/en/docs/sql-manual/sql-reference/
[doris-website] branch master updated: [typo](doc)add hive-bitmap package des (#137)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 1cf8557ff35 [typo](doc)add hive-bitmap package des (#137) 1cf8557ff35 is described below commit 1cf8557ff352621b1d219291eae75025cd49bd8c Author: Liqf <109049295+lemonlit...@users.noreply.github.com> AuthorDate: Mon Oct 10 12:48:56 2022 +0800 [typo](doc)add hive-bitmap package des (#137) --- .../import/import-way/spark-load-manual.md | 4 +-- docs/ecosystem/external-table/hive-bitmap-udf.md | 39 +- .../import/import-way/spark-load-manual.md | 2 +- .../ecosystem/external-table/hive-bitmap-udf.md| 30 - 4 files changed, 56 insertions(+), 19 deletions(-) diff --git a/docs/data-operate/import/import-way/spark-load-manual.md b/docs/data-operate/import/import-way/spark-load-manual.md index c8bfb0e7764..d712deb9f1b 100644 --- a/docs/data-operate/import/import-way/spark-load-manual.md +++ b/docs/data-operate/import/import-way/spark-load-manual.md @@ -441,7 +441,7 @@ PROPERTIES ``` -Example 3: when the upstream data source is hive binary type table +Example 3: when the upstream data source is hive binary type table . ```sql step 1: create hive external table @@ -600,7 +600,7 @@ The data type applicable to the aggregate column of the doris table is bitmap ty There is no need to build a global dictionary, just specify the corresponding field in the load command, the format is: ```doris field name=binary_bitmap (hive table field name)``` -Similarly, the binary (bitmap) type of data import is currently only supported when the upstream data source is a hive table. +Similarly, the binary (bitmap) type of data import is currently only supported when the upstream data source is a hive table,You can refer to the use of hive bitmap [hive-bitmap-udf](../../../ecosystem/external-table/hive-bitmap-udf) ### Show Load diff --git a/docs/ecosystem/external-table/hive-bitmap-udf.md b/docs/ecosystem/external-table/hive-bitmap-udf.md index 1402cf1ef69..7f37c791f5b 100644 --- a/docs/ecosystem/external-table/hive-bitmap-udf.md +++ b/docs/ecosystem/external-table/hive-bitmap-udf.md @@ -43,23 +43,46 @@ CREATE TABLE IF NOT EXISTS `hive_bitmap_table`( `k3` StringCOMMENT '', `uuid` binaryCOMMENT 'bitmap' ) comment 'comment' + +-- Example：Create Hive Table +CREATE TABLE IF NOT EXISTS `hive_table`( + `k1` int COMMENT '', + `k2` StringCOMMENT '', + `k3` StringCOMMENT '', + `uuid` int COMMENT '' +) comment 'comment' ``` ### Hive Bitmap UDF Usage： - Hive Bitmap UDF used in Hive/Spark + Hive Bitmap UDF used in Hive/Spark,First, you need to compile fe to get hive-udf-jar-with-dependencies.jar. + Compilation preparation:If you have compiled the ldb source code, you can directly compile fe,If you have compiled the ldb source code, you can compile it directly. If you have not compiled the ldb source code, you need to manually install thrift， + Reference:[Setting Up dev env for FE](https://doris.apache.org/community/developer-guide/fe-idea-dev). + +```sql +--clone doris code +git clone https://github.com/apache/doris.git +--install thrift +--Enter the fe directory +cd fe +--Execute the maven packaging command（All sub modules of fe will be packaged） +mvn package -Dmaven.test.skip=true +--You can also just package the hive-udf module +mvn package -pl hive-udf -am -Dmaven.test.skip=true +``` +After packaging and compiling, enter the hive-udf directory and there will be a target directory,There will be hive-udf-jar-with-dependencies.jar package ```sql -- Load the Hive Bitmap Udf jar package (Upload the compiled hive-udf jar package to HDFS) add jar hdfs://node:9001/hive-udf-jar-with-dependencies.jar; -- Create Hive Bitmap UDAF function -create temporary function to_bitmap as 'org.apache.doris.udf.ToBitmapUDAF'; -create temporary function bitmap_union as 'org.apache.doris.udf.BitmapUnionUDAF'; +create temporary function to_bitmap as 'org.apache.doris.udf.ToBitmapUDAF' USING JAR 'hdfs://node:9001/hive-udf-jar-with-dependencies.jar'; +create temporary function bitmap_union as 'org.apache.doris.udf.BitmapUnionUDAF' USING JAR 'hdfs://node:9001/hive-udf-jar-with-dependencies.jar'; -- Create Hive Bitmap UDF function -create temporary function bitmap_count as 'org.apache.doris.udf.BitmapCountUDF'; -create temporary function bitmap_and as 'org.apache.doris.udf.BitmapAndUDF'; -create temporary function bitmap_or as 'org.apache.doris.udf.BitmapOrUDF'; -create temporary function bitmap_xor as 'org.apache.doris.udf.BitmapXorUDF'; +c
[doris] branch master updated: [improvement](mysql-to-doris)Fully resolved Mysql external table issues (#13229)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 54e6f12110 [improvement](mysql-to-doris)Fully resolved Mysql external
table issues (#13229)
54e6f12110 is described below
commit 54e6f12110e0aae1caf45b777f05d4912962b764
Author: toms <94617906+toms1...@users.noreply.github.com>
AuthorDate: Mon Oct 10 16:48:52 2022 +0800
[improvement](mysql-to-doris)Fully resolved Mysql external table issues
(#13229)
Fully resolved Mysql external table issues
---
extension/mysql_to_doris/README.md | 12 +-
extension/mysql_to_doris/all_tables.sh | 154 -
extension/mysql_to_doris/bin/e_auto.sh | 52 +++
extension/mysql_to_doris/bin/e_mysql_to_doris.sh | 97 +
.../mysql_to_doris/conf/{tables => doris_tables} | 28 +---
.../mysql_to_doris/conf/{tables => mysql_tables} | 25 +---
extension/mysql_to_doris/lib/mysql_to_doris.sh | 61
extension/mysql_to_doris/user_define_tables.sh | 154 -
8 files changed, 225 insertions(+), 358 deletions(-)
diff --git a/extension/mysql_to_doris/README.md
b/extension/mysql_to_doris/README.md
index 049dafcdf3..351f2b439f 100644
--- a/extension/mysql_to_doris/README.md
+++ b/extension/mysql_to_doris/README.md
@@ -22,11 +22,13 @@ How to do?
1、To configure mysql.conf and doris.conf in the conf directory,the conf
including
host、port and password
-2、sh all_tables.sh and give it two args,it is mysql database and doris database
+2、To configure mysql_tables and doris_tables in the conf directory,the conf is
user need to synchronization tables and want to get table name
-In addition
+3、To execute e_mysql_to_doris.sh by sh e_mysql_to_doris.sh
-user can add mysql tables to the conf directory tables by sh
user_define_tables.sh
-can also be export mysql tables to doris
+4、To execute e_auto.sh by nohup sh e_auto.sh &
+
+What do you get?
+
+A simple configuration synchronizes all configured tables and Monitor Mysql
metadata changes in real time
-just do it
diff --git a/extension/mysql_to_doris/all_tables.sh
b/extension/mysql_to_doris/all_tables.sh
deleted file mode 100644
index 634ece0f71..00
--- a/extension/mysql_to_doris/all_tables.sh
+++ /dev/null
@@ -1,154 +0,0 @@
-#!/bin/bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-
-
-# This script is used to will mysql databases import doris by external
-
-
-#reference configuration file
-source ./conf/mysql.conf
-source ./conf/doris.conf
-
-#define mysql database and doris database
-d_mysql=$1
-d_doris=$2
-
-#check args
-if [ ! -n "$1" ];then
-echo "please check source database"
-exit
-fi
-if [ ! -n "$2" ];then
-echo "please check sink database"
-exit
-fi
-
-#mkdir files to store tables and tables.sql
-mkdir -p files
-rm -rf ./files/tables
-rm -rf ./files/tables.sql
-#get tables from mysql databases
-echo "use $d_mysql; show tables;" |mysql -h$mysql_host -uroot
-p$mysql_password 2>/dev/null >> ./files/tables
-
-#delete tables first line
-sed -i '1d' ./files/tables
-
-#reference tables to create tables.sql
-for table in $(awk -F '\n' '{print $1}' ./files/tables)
-do
-sed -i "/${table}view/d" ./files/tables
-echo "use $d_mysql; show create table ${table};" |mysql -h$mysql_host
-uroot -p$mysql_password 2>/dev/null >> ./files/tables.sql
-echo "print ${table} sql to tables.sql in the file dir"
-
-done
-
-echo '==start to transform mysql table for doris
extral table=='
-#adjust sql
-awk -F '\t' '{print $2}' ./files/tables.sql |awk '!(NR%2)' |awk '{print $0
";"}' > ./files/tables1.sql
-sed -i
[doris] branch master updated (7c0695c793 -> 6ee150755a)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 7c0695c793 [regression](load)Open broker load regression test (#13163)
add 6ee150755a [refactor](datax)Refactoring doris writer code (#13226)
No new revisions were added by this update.
Summary of changes:
docs/en/docs/ecosystem/datax.md| 63 +++--
docs/zh-CN/docs/ecosystem/datax.md | 64 +++--
extension/DataX/doriswriter/doc/doriswriter.md | 171 ++--
extension/DataX/doriswriter/doc/mysql2doris.json | 19 +-
extension/DataX/doriswriter/pom.xml| 1 -
.../plugin/writer/doriswriter/DelimiterParser.java | 71 +
.../{EscapeHandler.java => DorisBaseCodec.java}| 34 ++-
.../plugin/writer/doriswriter/DorisCodec.java | 55 +---
.../{EscapeHandler.java => DorisCodecFactory.java} | 30 +--
.../plugin/writer/doriswriter/DorisCsvCodec.java | 31 +--
.../plugin/writer/doriswriter/DorisFlushBatch.java | 66 -
.../plugin/writer/doriswriter/DorisJsonCodec.java | 29 +-
.../doriswriter/DorisStreamLoadObserver.java | 250 ++
.../datax/plugin/writer/doriswriter/DorisUtil.java | 122 +
.../plugin/writer/doriswriter/DorisWriter.java | 251 ++
.../writer/doriswriter/DorisWriterEmitter.java | 292 -
.../writer/doriswriter/DorisWriterExcetion.java| 36 ++-
.../writer/doriswriter/DorisWriterManager.java | 209 +++
.../datax/plugin/writer/doriswriter/Key.java | 191 --
.../datax/plugin/writer/doriswriter/Keys.java | 194 ++
.../plugin/writer/doriswriter/WriterTuple.java | 26 +-
.../doriswriter/src/main/resources/plugin.json | 4 +-
.../src/main/resources/plugin_job_template.json| 14 +-
.../writer/doriswriter/TestDorisWriterLoad.java| 88 +++
24 files changed, 1281 insertions(+), 1030 deletions(-)
create mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DelimiterParser.java
copy
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/{EscapeHandler.java
=> DorisBaseCodec.java} (54%)
rename
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/{EscapeHandler.java
=> DorisCodecFactory.java} (53%)
delete mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisFlushBatch.java
create mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisStreamLoadObserver.java
create mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisUtil.java
delete mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisWriterEmitter.java
copy fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/Timestamp.java =>
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisWriterExcetion.java
(54%)
create mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisWriterManager.java
delete mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/Key.java
create mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/Keys.java
copy fe/fe-core/src/main/java/org/apache/doris/analysis/ImportColumnsStmt.java
=>
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/WriterTuple.java
(61%)
create mode 100644
extension/DataX/doriswriter/src/test/java/com/alibaba/datax/plugin/writer/doriswriter/TestDorisWriterLoad.java
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (2b5db24d82 -> 0b9e9ac209)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 2b5db24d82 remove unstable q21 (#13271) add 0b9e9ac209 metadata operation fix 404 error url (#13215) No new revisions were added by this update. Summary of changes: docs/en/docs/admin-manual/maint-monitor/metadata-operation.md | 2 +- docs/en/docs/advanced/alter-table/schema-change.md| 2 +- docs/en/docs/ecosystem/logstash.md| 2 +- docs/zh-CN/docs/data-operate/export/outfile.md| 2 +- docs/zh-CN/docs/ecosystem/logstash.md | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs)Modification instructions and examples for adding schema change key columns
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 68e1cfa381c [typo](docs)Modification instructions and examples for adding schema change key columns 68e1cfa381c is described below commit 68e1cfa381c5868b5fd9c662e6169bb9f9bef939 Author: jiafeng.zhang AuthorDate: Tue Oct 11 14:45:51 2022 +0800 [typo](docs)Modification instructions and examples for adding schema change key columns [typo](docs)Modification instructions and examples for adding schema change key columns --- docs/advanced/alter-table/schema-change.md | 39 ++ .../Alter/ALTER-TABLE-COLUMN.md| 6 .../current/advanced/alter-table/schema-change.md | 39 ++ .../Alter/ALTER-TABLE-COLUMN.md| 6 4 files changed, 90 insertions(+) diff --git a/docs/advanced/alter-table/schema-change.md b/docs/advanced/alter-table/schema-change.md index e5e4a6465f5..43488665a24 100644 --- a/docs/advanced/alter-table/schema-change.md +++ b/docs/advanced/alter-table/schema-change.md @@ -171,6 +171,45 @@ When completion, the Schema becomes: As you can see, the base table tbl1 also automatically added k4, k5 columns. That is, columns added to any rollup are automatically added to the Base table. At the same time, columns that already exist in the Base table are not allowed to be added to Rollup. If you need to do this, you can re-create a Rollup with the new columns and then delete the original Rollup. +### Modify Key column + +Modifying the Key column of a table is done through the `key` keyword. Let's take a look at an example below. + +Source Schema : + +```text ++---+---+-+--+--+-+---+ +| IndexName | Field | Type| Null | Key | Default | Extra | ++---+---+-+--+--+-+---+ +| tbl1 | k1| INT | No | true | N/A | | +| | k2| INT | No | true | N/A | | +| | k3| varchar(20) | No | true | N/A | | +| | k4| INT | No | false| N/A | | ++---+---+-+--+--+-+---+ +``` + +The modification statement is as follows, we will change the degree of the k3 column to 50 + + +```sql +alter table example_tbl modify column k3 varchar(50) key null comment 'to 50' + + +When done, the Schema becomes: + +```text ++---+---+-+--+--+-+---+ +| IndexName | Field | Type| Null | Key | Default | Extra | ++---+---+-+--+--+-+---+ +| tbl1 | k1| INT | No | true | N/A | | +| | k2| INT | No | true | N/A | | +| | k3| varchar(50) | No | true | N/A | | +| | k4| INT | No | false| N/A | | ++---+---+-+--+--+-+---+ +``` + + + ## Notice * Only one Schema Change job can be running on a table at a time. diff --git a/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md b/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md index 697ef2a912b..308ede9f649 100644 --- a/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md +++ b/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md @@ -215,6 +215,12 @@ ADD COLUMN v2 INT MAX DEFAULT "0" AFTER k2 TO example_rollup_index, ORDER BY (k3,k1,k2,v2,v1) FROM example_rollup_index; ``` +11. Modify the length of a field in the Key column + +```sql +alter table example_tbl modify column k3 varchar(50) key null comment 'to 50' + + ### Keywords ```text diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md index dfee8198417..bbf6614ff67 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md @@ -181,6 +181,45 @@ ADD COLUMN k5 INT default "1" to rollup2; 同时，不允许向 Rollup 中加入 Base 表已经存在的列。如果用户需要这样做，可以重新建立一个包含新增列的 Rollup，之后再删除原 Rollup。 +### 修改 Key 列 + +修改表的 Key 列是通过 `key` 关键字完成，下面我们通过一个例子来看。 + +源 Schema : + +```text ++---+---+-+--+--+-+---+ +| IndexName | Field | Type| Null | Key | Default | Extra | ++---+---+-+--+--+-+---+ +| tbl1 | k1| INT | No | true |
[doris-website] branch master updated: [typo](docs)delete duplicate document (#138)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 3c98c47d67a [typo](docs)delete duplicate document (#138)
3c98c47d67a is described below
commit 3c98c47d67a6370d20abc0a4cda98cf4cbe6dd22
Author: Liqf <109049295+lemonlit...@users.noreply.github.com>
AuthorDate: Tue Oct 11 14:47:45 2022 +0800
[typo](docs)delete duplicate document (#138)
---
.../date-time-functions/year_floor.md | 48 --
.../date-time-functions/year_floor.md | 48 --
sidebars.json | 1 -
3 files changed, 97 deletions(-)
diff --git a/docs/sql-manual/sql-functions/date-time-functions/year_floor.md
b/docs/sql-manual/sql-functions/date-time-functions/year_floor.md
deleted file mode 100644
index 169655ba20c..000
--- a/docs/sql-manual/sql-functions/date-time-functions/year_floor.md
+++ /dev/null
@@ -1,48 +0,0 @@
-{
-"title": "year_floor",
-"language": "en"
-}
-
-
-
-## year_floor
-### description
- Syntax
-
-`DATETIME year_floor(DATETIME date)`
-
-Rounding down the specified time, reserving the field level to years, returns
the DATETIME type
-
-### Example
-
-```
-mysql> select year_floor('2022-09-22 17:00:00');
-+---+
-| year_floor('2022-09-22 17:00:00') |
-+---+
-| 2022-01-01 00:00:00 |
-+---+
-```
-
-### keywords
-
-year_floor
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/date-time-functions/year_floor.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/date-time-functions/year_floor.md
deleted file mode 100644
index df7d6851bd6..000
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-functions/date-time-functions/year_floor.md
+++ /dev/null
@@ -1,48 +0,0 @@
-{
-"title": "year_floor",
-"language": "zh-CN"
-}
-
-
-
-## year_floor
-### description
- Syntax
-
-`DATETIME year_floor(DATETIME date)`
-
-指定时间向下舍入，保留字段级别到年,返回DATETIME类型
-
-### Example
-
-```
-mysql> select year_floor('2022-09-22 17:00:00');
-+---+
-| year_floor('2022-09-22 17:00:00') |
-+---+
-| 2022-01-01 00:00:00 |
-+---+
-```
-
-### keywords
-
-year_floor
diff --git a/sidebars.json b/sidebars.json
index 881e38ff728..4b540048810 100644
--- a/sidebars.json
+++ b/sidebars.json
@@ -249,7 +249,6 @@
"sql-manual/sql-functions/date-time-functions/yearweek",
"sql-manual/sql-functions/date-time-functions/dayname",
"sql-manual/sql-functions/date-time-functions/monthname",
-
"sql-manual/sql-functions/date-time-functions/year_floor",
"sql-manual/sql-functions/date-time-functions/hour",
"sql-manual/sql-functions/date-time-functions/minute",
"sql-manual/sql-functions/date-time-functions/second",
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix 404 error url in logstash (#136)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 095e133ee0d fix 404 error url in logstash (#136)
095e133ee0d is described below
commit 095e133ee0da2e68e77de86a79bf12b2134cdd5c
Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com>
AuthorDate: Tue Oct 11 14:48:02 2022 +0800
fix 404 error url in logstash (#136)
---
docs/ecosystem/logstash.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/ecosystem/logstash.md b/docs/ecosystem/logstash.md
index c6ab151d1f4..11ec6163db7 100644
--- a/docs/ecosystem/logstash.md
+++ b/docs/ecosystem/logstash.md
@@ -85,7 +85,7 @@ Configuration | Explanation
`label_prefix` | Import the identification prefix, the final generated ID is
*{label\_prefix}\_{db}\_{table}\_{time_stamp}*
-Load configuration:([Reference
documents](http://doris.apache.org/master/zh-CN/administrator-guide/load-data/stream-load-manual.html))
+Load configuration:([Reference
documents](../data-operate/import/import-way/stream-load-manual))
Configuration | Explanation
--- | ---
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: update url (#133)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 787ef492f8e update url (#133) 787ef492f8e is described below commit 787ef492f8e3fb59ab3d1712d9ddd070aa5e846d Author: wudi <676366...@qq.com> AuthorDate: Tue Oct 11 14:48:16 2022 +0800 update url (#133) Co-authored-by: wudi <> --- .../version-0.15/administrator-guide/config/fe_config.md | 2 +- .../version-0.15/administrator-guide/load-data/broker-load-manual.md | 2 +- .../version-0.15/administrator-guide/load-data/routine-load-manual.md | 2 +- .../version-0.15/administrator-guide/operation/disk-capacity.md | 4 ++-- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/config/fe_config.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/config/fe_config.md index d67952c8197..0ef6f996b15 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/config/fe_config.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/config/fe_config.md @@ -82,7 +82,7 @@ FE 的配置项有两种方式进行配置： 3. 通过 HTTP 协议动态配置 - 具体请参阅 [Set Config Action](http://doris.apache.org/master/zh-CN/administrator-guide/http-actions/fe/set-config-action.html) + 具体请参阅 [Set Config Action](../http-actions/fe/set-config-action) 该方式也可以持久化修改后的配置项。配置项将持久化在 `fe_custom.conf` 文件中，在 FE 重启后仍会生效。 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/broker-load-manual.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/broker-load-manual.md index 8d85457301d..69ba5255c26 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/broker-load-manual.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/broker-load-manual.md @@ -513,7 +513,7 @@ LoadFinishTime: 2019-07-27 11:50:16 * 导入报错：`failed to send batch` 或 `TabletWriter add batch with unknown id` -请参照 [导入手册](./load-manual.md) 中 **通用系统配置** 中 **BE 配置**，适当修改 `query_timeout` 和 `streaming_load_rpc_max_alive_time_sec`。 +请参照 [导入手册](./load-manual) 中 **通用系统配置** 中 **BE 配置**，适当修改 `query_timeout` 和 `streaming_load_rpc_max_alive_time_sec`。 * 导入报错：`LOAD_RUN_FAIL; msg:Invalid Column Name:xxx` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/routine-load-manual.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/routine-load-manual.md index 65ab1d63297..aa150ec1d70 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/routine-load-manual.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/load-data/routine-load-manual.md @@ -242,7 +242,7 @@ FE 中的 JobScheduler 根据汇报结果，继续生成后续新的 Task，或 ### 修改作业属性 -用户可以修改已经创建的作业。具体说明可以通过 `HELP ALTER ROUTINE LOAD;` 命令查看。或参阅 [ALTER ROUTINE LOAD](../../sql-reference/sql-statements/Data%20Manipulation/alter-routine-load.md)。 +用户可以修改已经创建的作业。具体说明可以通过 `HELP ALTER ROUTINE LOAD;` 命令查看。或参阅 [ALTER ROUTINE LOAD](../../sql-reference/sql-statements/Data-Manipulation/alter-routine-load)。 ### 作业控制 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/operation/disk-capacity.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/operation/disk-capacity.md index 8616f63575d..aa3ca1a79d8 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/operation/disk-capacity.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-0.15/administrator-guide/operation/disk-capacity.md @@ -127,7 +127,7 @@ capacity_min_left_bytes_flood_stage 默认 1GB。 * snapshot/: 快照目录下的快照文件。 * trash/：回收站中的文件。 -**这种操作会对 [从 BE 回收站中恢复数据](./tablet-restore-tool.md) 产生影响。** +**这种操作会对 [从 BE 回收站中恢复数据](./tablet-restore-tool) 产生影响。** 如果BE还能够启动，则可以使用`ADMIN CLEAN TRASH ON(BackendHost:BackendHeartBeatPort);`来主动清理临时文件，会清理 **所有** trash文件和过期snapshot文件，**这将影响从回收站恢复数据的操作** 。 @@ -158,6 +158,6 @@ capacity_min_left_bytes_flood_stage 默认 1GB。 ```rm -rf data/0/12345/``` -* 删除 Tablet 元数据（具体参考 [Tablet 元数据管理工具](./tablet-meta-tool.md)） +* 删除 Tablet 元数据（具体参考 [Tablet 元数据管理工具](./tablet-meta-tool)） ```./lib/meta_tool --operation=delete_header --root_path=/path/to/root_path --tablet_id=12345 --schema_hash= 35278``` \ No newline at end of file - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: Update install-deploy 404 repair (#132)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 732057e20d5 Update install-deploy 404 repair (#132) 732057e20d5 is described below commit 732057e20d58858c14c047044508b543e50a3c9a Author: SeekingYang <100211303+seekingy...@users.noreply.github.com> AuthorDate: Tue Oct 11 14:49:02 2022 +0800 Update install-deploy 404 repair (#132) --- versioned_docs/version-0.15/best-practices/star-schema-benchmark.md | 2 +- versioned_docs/version-0.15/extending-doris/logstash.md | 2 +- .../version-0.15/extending-doris/udf/user-defined-function.md | 2 +- versioned_docs/version-0.15/installing/install-deploy.md| 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/versioned_docs/version-0.15/best-practices/star-schema-benchmark.md b/versioned_docs/version-0.15/best-practices/star-schema-benchmark.md index 17288bbcc0f..99174569059 100644 --- a/versioned_docs/version-0.15/best-practices/star-schema-benchmark.md +++ b/versioned_docs/version-0.15/best-practices/star-schema-benchmark.md @@ -162,4 +162,4 @@ The following test report is based on Doris [branch-0.15](https://github.com/apa > > Note 4: Parallelism means query concurrency, which is set by `set parallel_fragment_exec_instance_num=8`. > -> Note 5: Runtime Filter Mode is the type of Runtime Filter, set by `set runtime_filter_type="BLOOM_FILTER"`. ([Runtime Filter](http://doris.incubator.apache.org/master/en/administrator-guide/runtime-filter.html) function has a significant effect on the SSB test set. Because in this test set, The data from the right table of Join can filter the left table very well. You can try to turn off this function through `set runtime_filter_mode=off` to see the change in query latency.) +> Note 5: Runtime Filter Mode is the type of Runtime Filter, set by `set runtime_filter_type="BLOOM_FILTER"`. ([Runtime Filter](../administrator-guide/runtime-filter/) function has a significant effect on the SSB test set. Because in this test set, The data from the right table of Join can filter the left table very well. You can try to turn off this function through `set runtime_filter_mode=off` to see the change in query latency.) diff --git a/versioned_docs/version-0.15/extending-doris/logstash.md b/versioned_docs/version-0.15/extending-doris/logstash.md index 39177207e6f..0bcc143633c 100644 --- a/versioned_docs/version-0.15/extending-doris/logstash.md +++ b/versioned_docs/version-0.15/extending-doris/logstash.md @@ -28,7 +28,7 @@ under the License. This plugin is used to output data to Doris for logstash, use the HTTP protocol to interact with the Doris FE Http interface, and import data through Doris's stream load. -[Learn more about Doris Stream Load ](http://doris.apache.org/master/zh-CN/administrator-guide/load-data/stream-load-manual.html) +[Learn more about Doris Stream Load ](../administrator-guide/load-data/stream-load-manual/) [Learn more about Doris](http://doris.apache.org/master/zh-CN/) diff --git a/versioned_docs/version-0.15/extending-doris/udf/user-defined-function.md b/versioned_docs/version-0.15/extending-doris/udf/user-defined-function.md index f1ebcced6a6..304760b40b4 100644 --- a/versioned_docs/version-0.15/extending-doris/udf/user-defined-function.md +++ b/versioned_docs/version-0.15/extending-doris/udf/user-defined-function.md @@ -34,7 +34,7 @@ There are two types of analysis requirements that UDF can meet: UDF and UDAF. UD This document mainly describes how to write a custom UDF function and how to use it in Doris. -If users use the UDF function and extend Doris' function analysis, and want to contribute their own UDF functions back to the Doris community for other users, please see the document [Contribute UDF](./contribute_udf.md). +If users use the UDF function and extend Doris' function analysis, and want to contribute their own UDF functions back to the Doris community for other users, please see the document [Contribute UDF](../contribute_udf). ## Writing UDF functions diff --git a/versioned_docs/version-0.15/installing/install-deploy.md b/versioned_docs/version-0.15/installing/install-deploy.md index 867d79ec671..37cd6545f0e 100644 --- a/versioned_docs/version-0.15/installing/install-deploy.md +++ b/versioned_docs/version-0.15/installing/install-deploy.md @@ -74,7 +74,7 @@ Doris, as an open source MPP architecture OLAP database, can run on most mainstr > Note 2: Number of FE nodes > -> 1. FE roles are divided into Follower and Observer. (Leader is an elected role in the Follower group, hereinafter referred to as Follower, for the specific meaning, see [Metadata Design Document](./internal/metadata-design)
[doris-website] branch master updated: [typo](docs) Fix jump link 404 in elastic expansion. md (#123)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new d7b2a5a2ed4 [typo](docs) Fix jump link 404 in elastic expansion. md (#123) d7b2a5a2ed4 is described below commit d7b2a5a2ed4fc5c9724042cd1b9bcfc7388ca5c3 Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Tue Oct 11 14:49:34 2022 +0800 [typo](docs) Fix jump link 404 in elastic expansion. md (#123) --- docs/admin-manual/cluster-management/elastic-expansion.md | 2 +- .../current/admin-manual/cluster-management/elastic-expansion.md| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin-manual/cluster-management/elastic-expansion.md b/docs/admin-manual/cluster-management/elastic-expansion.md index be6cc0e07a8..17cb9111553 100644 --- a/docs/admin-manual/cluster-management/elastic-expansion.md +++ b/docs/admin-manual/cluster-management/elastic-expansion.md @@ -106,7 +106,7 @@ You can also view the BE node through the front-end page connection: ``http://fe All of the above methods require Doris's root user rights. -The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-meta-tool.md). +The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-repair-and-balance.md). ### Add BE nodes diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/elastic-expansion.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/elastic-expansion.md index 258f8d8a7ed..52f1fb66a2a 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/elastic-expansion.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/elastic-expansion.md @@ -102,7 +102,7 @@ FE 分为 Leader，Follower 和 Observer 三种角色。 默认一个集群， 以上方式，都需要 Doris 的 root 用户权限。 -BE 节点的扩容和缩容过程，不影响当前系统运行以及正在执行的任务，并且不会影响当前系统的性能。数据均衡会自动进行。根据集群现有数据量的大小，集群会在几个小时到1天不等的时间内，恢复到负载均衡的状态。集群负载情况，可以参见 [Tablet 负载均衡文档](../maint-monitor/tablet-repair-and-balance.html#%E5%89%AF%E6%9C%AC%E5%9D%87%E8%A1%A1)。 +BE 节点的扩容和缩容过程，不影响当前系统运行以及正在执行的任务，并且不会影响当前系统的性能。数据均衡会自动进行。根据集群现有数据量的大小，集群会在几个小时到1天不等的时间内，恢复到负载均衡的状态。集群负载情况，可以参见 [Tablet 负载均衡文档](../maint-monitor/tablet-repair-and-balance.md#%E5%89%AF%E6%9C%AC%E5%9D%87%E8%A1%A1)。 ### 增加 BE 节点 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs) Fix the jump link 404 (#126)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 72e258bab7c [typo](docs) Fix the jump link 404 (#126) 72e258bab7c is described below commit 72e258bab7c09ea45a68fb3563291967ccd9c619 Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Tue Oct 11 14:49:20 2022 +0800 [typo](docs) Fix the jump link 404 (#126) * [typo](docs)Fix the jump link 404 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs)Fix the jump link 404 in delete recover.md (#121)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new b03a0ff38fc [typo](docs)Fix the jump link 404 in delete recover.md (#121) b03a0ff38fc is described below commit b03a0ff38fc6ad404e6a0ef00e2c68847a3de8ab Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Tue Oct 11 14:49:50 2022 +0800 [typo](docs)Fix the jump link 404 in delete recover.md (#121) * [typo](docs)Fix the jump link 404 --- docs/admin-manual/data-admin/delete-recover.md | 2 +- .../current/admin-manual/data-admin/delete-recover.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/admin-manual/data-admin/delete-recover.md b/docs/admin-manual/data-admin/delete-recover.md index f02c4ada157..98b0ec3acf9 100644 --- a/docs/admin-manual/data-admin/delete-recover.md +++ b/docs/admin-manual/data-admin/delete-recover.md @@ -50,4 +50,4 @@ RECOVER PARTITION p1 FROM example_tbl; ## More Help -For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RECOVER.md) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help. +For more detailed syntax and best practices used by RECOVER, please refer to the [RECOVER](../../sql-manual/sql-reference/Database-Administration-Statements/RECOVER.md) command manual, You can also type `HELP RECOVER` on the MySql client command line for more help. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/delete-recover.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/delete-recover.md index a9ba4777b91..249037eae57 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/delete-recover.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/delete-recover.md @@ -50,4 +50,4 @@ RECOVER PARTITION p1 FROM example_tbl; ## 更多帮助 -关于 RECOVER 使用的更多详细语法及最佳实践，请参阅 [RECOVER](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RECOVER.md) 命令手册，你也可以在 MySql 客户端命令行下输入 `HELP RECOVER` 获取更多帮助信息。 +关于 RECOVER 使用的更多详细语法及最佳实践，请参阅 [RECOVER](../../sql-manual/sql-reference/Database-Administration-Statements/RECOVER.md) 命令手册，你也可以在 MySql 客户端命令行下输入 `HELP RECOVER` 获取更多帮助信息。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs)Fix FE Configuration Jump Link 404 (#120)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 96cf41ceb8f [typo](docs)Fix FE Configuration Jump Link 404 (#120) 96cf41ceb8f is described below commit 96cf41ceb8ff49353e9cd0352c51e41c8b51fe98 Author: yuanyuan8983 <99315889+yuanyuan8...@users.noreply.github.com> AuthorDate: Tue Oct 11 14:50:02 2022 +0800 [typo](docs)Fix FE Configuration Jump Link 404 (#120) --- docs/admin-manual/http-actions/fe/table-schema-action.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/admin-manual/http-actions/fe/table-schema-action.md b/docs/admin-manual/http-actions/fe/table-schema-action.md index 9737ac19aa0..c2f7cc7 100644 --- a/docs/admin-manual/http-actions/fe/table-schema-action.md +++ b/docs/admin-manual/http-actions/fe/table-schema-action.md @@ -97,7 +97,7 @@ None "count": 0 } ``` -Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](https://doris.apache.org/administrator-guide/config/fe_config.html) +Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../admin-manual/config/fe-config) ## Examples - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (0b9e9ac209 -> ed19562cb3)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 0b9e9ac209 metadata operation fix 404 error url (#13215) add ed19562cb3 And ali datax unified configuration naming, modify maxBatchSize to batchSize(#13278) No new revisions were added by this update. Summary of changes: extension/DataX/doriswriter/doc/doriswriter.md | 2 +- .../datax/plugin/writer/doriswriter/Keys.java | 2 +- .../writer/doriswriter/TestDorisWriterLoad.java| 88 -- 3 files changed, 2 insertions(+), 90 deletions(-) delete mode 100644 extension/DataX/doriswriter/src/test/java/com/alibaba/datax/plugin/writer/doriswriter/TestDorisWriterLoad.java - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4d7fad3c5f1 fix 4d7fad3c5f1 is described below commit 4d7fad3c5f132be57a4d8cd0274649576d7e4f1a Author: jiafeng.zhang AuthorDate: Tue Oct 11 14:57:39 2022 +0800 fix --- docs/advanced/alter-table/schema-change.md | 2 ++ .../Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md | 2 +- .../current/advanced/alter-table/schema-change.md | 2 ++ .../Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md | 2 +- 4 files changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/advanced/alter-table/schema-change.md b/docs/advanced/alter-table/schema-change.md index 43488665a24..3e44bbb6ba8 100644 --- a/docs/advanced/alter-table/schema-change.md +++ b/docs/advanced/alter-table/schema-change.md @@ -175,6 +175,8 @@ At the same time, columns that already exist in the Base table are not allowed t Modifying the Key column of a table is done through the `key` keyword. Let's take a look at an example below. +**This usage is only for the key column of the duplicate key table** + Source Schema : ```text diff --git a/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md b/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md index 308ede9f649..a4c106523d7 100644 --- a/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md +++ b/docs/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md @@ -215,7 +215,7 @@ ADD COLUMN v2 INT MAX DEFAULT "0" AFTER k2 TO example_rollup_index, ORDER BY (k3,k1,k2,v2,v1) FROM example_rollup_index; ``` -11. Modify the length of a field in the Key column +11. Modify the length of a field in the Key column of the Duplicate key table ```sql alter table example_tbl modify column k3 varchar(50) key null comment 'to 50' diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md index bbf6614ff67..a9c3b867262 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md @@ -185,6 +185,8 @@ ADD COLUMN k5 INT default "1" to rollup2; 修改表的 Key 列是通过 `key` 关键字完成，下面我们通过一个例子来看。 +**这个用法只针对 duplicate key 表的 key 列** + 源 Schema : ```text diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md index c796eb1feff..3f3e96db858 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md @@ -215,7 +215,7 @@ ADD COLUMN v2 INT MAX DEFAULT "0" AFTER k2 TO example_rollup_index, ORDER BY (k3,k1,k2,v2,v1) FROM example_rollup_index; ``` -11. 修改 Key 列的某个字段的长度 +11. 修改Duplicate key 表 Key 列的某个字段的长度 ```sql alter table example_tbl modify column k3 varchar(50) key null comment 'to 50' - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new c529067031b fix c529067031b is described below commit c529067031b977525fecd78add1ef0aa898eb018 Author: jiafeng.zhang AuthorDate: Tue Oct 11 15:06:04 2022 +0800 fix --- docs/advanced/alter-table/schema-change.md| 4 .../current/advanced/alter-table/schema-change.md | 4 2 files changed, 8 insertions(+) diff --git a/docs/advanced/alter-table/schema-change.md b/docs/advanced/alter-table/schema-change.md index 3e44bbb6ba8..a0c5b169d54 100644 --- a/docs/advanced/alter-table/schema-change.md +++ b/docs/advanced/alter-table/schema-change.md @@ -210,7 +210,11 @@ When done, the Schema becomes: +---+---+-+--+--+-+---+ ``` +Because the Schema Chanage job is an asynchronous operation, only one Schema chanage job can be performed on the same table at the same time. To check the operation status of the job, you can use the following command +```sql +SHOW ALTER TABLE COLUMN\G; + ## Notice diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md index a9c3b867262..f2adbbe4e4e 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/advanced/alter-table/schema-change.md @@ -220,7 +220,11 @@ alter table example_tbl modify column k3 varchar(50) key null comment 'to 50' +---+---+-+--+--+-+---+ ``` +因为Schema Chanage 作业是异步操作，同一个表同时只能进行一个Schema chanage 作业，查看作业运行情况，可以通过下面这个命令 +```sql +SHOW ALTER TABLE COLUMN\G; +``` ## 注意事项 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (c1ce48ffe4 -> 9c776c1011)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from c1ce48ffe4 [fix](new-scann) scanner may be marked close twice (#13263) add 9c776c1011 [typo](docs) Fix the jump link 404 in basic usage.md (#13169) No new revisions were added by this update. Summary of changes: docs/en/docs/data-table/basic-usage.md| 2 +- docs/zh-CN/docs/data-table/basic-usage.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (9c776c1011 -> 6dad7ee5f5)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 9c776c1011 [typo](docs) Fix the jump link 404 in basic usage.md (#13169) add 6dad7ee5f5 [typo](docs) Fix jump link 404 in elastic-expansion.md (#13168) No new revisions were added by this update. Summary of changes: docs/en/docs/admin-manual/cluster-management/elastic-expansion.md| 2 +- docs/zh-CN/docs/admin-manual/cluster-management/elastic-expansion.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (606b514329 -> b1cd87d635)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 606b514329 [fix](olap) fix core dump caused by LikeColumnPredicate with nullable column (#13250) add b1cd87d635 [typo](docs)Fix FE Configuration Jump Link 404 (#13149) No new revisions were added by this update. Summary of changes: docs/en/docs/admin-manual/http-actions/fe/table-schema-action.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](docs)fix error url (#13171)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new eb60976c25 [typo](docs)fix error url (#13171) eb60976c25 is described below commit eb60976c257db917a4f34921a2a1dc84d25f475a Author: caoliang-web <71004656+caoliang-...@users.noreply.github.com> AuthorDate: Tue Oct 11 15:41:00 2022 +0800 [typo](docs)fix error url (#13171) * fix error url --- docs/en/docs/advanced/resource.md | 2 +- .../docs/data-operate/import/import-scenes/external-storage-load.md | 2 +- docs/en/docs/data-operate/import/import-scenes/jdbc-load.md | 4 ++-- docs/en/docs/data-table/basic-usage.md | 6 +++--- .../docs/admin-manual/maint-monitor/tablet-repair-and-balance.md| 2 +- docs/zh-CN/docs/advanced/alter-table/replace-table.md | 2 +- docs/zh-CN/docs/advanced/resource.md| 2 +- .../docs/data-operate/import/import-scenes/external-storage-load.md | 2 +- docs/zh-CN/docs/data-operate/import/import-scenes/jdbc-load.md | 4 ++-- docs/zh-CN/docs/data-table/basic-usage.md | 6 +++--- docs/zh-CN/docs/data-table/hit-the-rollup.md| 2 +- 11 files changed, 17 insertions(+), 17 deletions(-) diff --git a/docs/en/docs/advanced/resource.md b/docs/en/docs/advanced/resource.md index 494ed7b200..885240d190 100644 --- a/docs/en/docs/advanced/resource.md +++ b/docs/en/docs/advanced/resource.md @@ -74,7 +74,7 @@ The following shows how the two resources are used. `spark.hadoop.fs.defaultFS`: required when master is yarn. -Other parameters are optional, refer to: http://spark.apache.org/docs/latest/configuration.html. +Other parameters are optional, refer to: http://spark.apache.org/docs/latest/configuration.html # If spark is used for ETL, also need to specify the following parameters: diff --git a/docs/en/docs/data-operate/import/import-scenes/external-storage-load.md b/docs/en/docs/data-operate/import/import-scenes/external-storage-load.md index 8b4537e097..9c9358d12f 100644 --- a/docs/en/docs/data-operate/import/import-scenes/external-storage-load.md +++ b/docs/en/docs/data-operate/import/import-scenes/external-storage-load.md @@ -82,7 +82,7 @@ Hdfs load creates an import statement. The import method is basically the same a 3. Check import status - Broker load is an asynchronous import method. The specific import results can be accessed through [SHOW LOAD](../../../sql-manual/sql-reference/Show-Statements/SHOW-LOAD.md#show-load) command to view + Broker load is an asynchronous import method. The specific import results can be accessed through [SHOW LOAD](../../../sql-manual/sql-reference/Show-Statements/SHOW-LOAD) command to view ``` mysql> show load order by createtime desc limit 1\G; diff --git a/docs/en/docs/data-operate/import/import-scenes/jdbc-load.md b/docs/en/docs/data-operate/import/import-scenes/jdbc-load.md index a74f1c9a6b..45c743dc6d 100644 --- a/docs/en/docs/data-operate/import/import-scenes/jdbc-load.md +++ b/docs/en/docs/data-operate/import/import-scenes/jdbc-load.md @@ -35,7 +35,7 @@ The INSERT statement is used in a similar way to the INSERT statement used in da * INSERT INTO table VALUES(...) -Here we only introduce the second way. For a detailed description of the INSERT command, see the [INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT.md) command documentation. +Here we only introduce the second way. For a detailed description of the INSERT command, see the [INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT) command documentation. ## Single write @@ -160,4 +160,4 @@ Please note the following: As mentioned earlier, we recommend that when using INSERT to import data, use the "batch" method to import, rather than a single insert. - At the same time, we can set a Label for each INSERT operation. Through the [Label mechanism](./load-atomicity.md#label-mechanism), the idempotency and atomicity of operations can be guaranteed, and the data will not be lost or heavy in the end. For the specific usage of Label in INSERT, you can refer to the [INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT.md) document. + At the same time, we can set a Label for each INSERT operation. Through the [Label mechanism](./load-atomicity), the idempotency and atomicity of operations can be guaranteed, and the data will not be lost or heavy in the end. For the specific usage of Label in INSERT, you can refer to the [INSERT](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/INSERT) document. diff --git a/docs
[doris] branch master updated (b1cd87d635 -> a716c74412)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from b1cd87d635 [typo](docs)Fix FE Configuration Jump Link 404 (#13149) add a716c74412 [typo](docs)Fix Docs Error Urls (#13176) No new revisions were added by this update. Summary of changes: .../Data-Definition-Statements/Alter/ALTER-TABLE-REPLACE.md | 2 +- .../sql-reference/Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: Delete creating S3 resources from the 1.1 document (#139)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 350b93dba7f Delete creating S3 resources from the 1.1 document (#139)
350b93dba7f is described below
commit 350b93dba7f9023a43596bd3744138b324dcf80a
Author: zy-kkk
AuthorDate: Tue Oct 11 17:53:29 2022 +0800
Delete creating S3 resources from the 1.1 document (#139)
---
.../Create/CREATE-RESOURCE.md | 30 --
.../Create/CREATE-RESOURCE.md | 30 --
2 files changed, 60 deletions(-)
diff --git
a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
index 2a6604a4c4f..060514fd13f 100644
---
a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
+++
b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
@@ -111,36 +111,6 @@ illustrate:
- charset: connection charset
- There is also support for implementing custom parameters per ODBC Driver,
see the description of the corresponding ODBC Driver
-3. Create S3 resource
-
- ```sql
- CREATE RESOURCE "remote_s3"
- PROPERTIES
- (
- "type" = "s3",
- "s3_endpoint" = "http://bj.s3.com";,
- "s3_region" = "bj",
- "s3_root_path" = "/path/to/root",
- "s3_access_key" = "bbb",
- "s3_secret_key" = "",
- "s3_max_connections" = "50",
- "s3_request_timeout_ms" = "3000",
- "s3_connection_timeout_ms" = "1000"
- );
-
-
- S3 related parameters are as follows:
- - Required parameters
- - s3_endpoint: s3 endpoint
- - s3_region: s3 region
- - s3_root_path: s3 root directory
- - s3_access_key: s3 access key
- - s3_secret_key: s3 secret key
- - optional parameter
- - s3_max_connections: the maximum number of s3 connections, the default
is 50
- - s3_request_timeout_ms: s3 request timeout, in milliseconds, the
default is 3000
- - s3_connection_timeout_ms: s3 connection timeout, in milliseconds, the
default is 1000
-
### Keywords
CREATE, RESOURCE
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
index 28f3146a524..2d11292ed2a 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-RESOURCE.md
@@ -111,36 +111,6 @@ PROPERTIES ("key"="value", ...);
- charset: 数据库链接的编码信息
- 另外还支持每个ODBC Driver 实现自定义的参数，参见对应ODBC Driver 的说明
-3. 创建 S3 resource
-
- ```sql
- CREATE RESOURCE "remote_s3"
- PROPERTIES
- (
- "type" = "s3",
- "s3_endpoint" = "http://bj.s3.com";,
- "s3_region" = "bj",
- "s3_root_path" = "/path/to/root",
- "s3_access_key" = "bbb",
- "s3_secret_key" = "",
- "s3_max_connections" = "50",
- "s3_request_timeout_ms" = "3000",
- "s3_connection_timeout_ms" = "1000"
- );
- ```
-
- S3 相关参数如下：
- - 必需参数
- - s3_endpoint：s3 endpoint
- - s3_region：s3 region
- - s3_root_path：s3 根目录
- - s3_access_key：s3 access key
- - s3_secret_key：s3 secret key
- - 可选参数
- - s3_max_connections：s3 最大连接数量，默认为 50
- - s3_request_timeout_ms：s3 请求超时时间，单位毫秒，默认为 3000
- - s3_connection_timeout_ms：s3 连接超时时间，单位毫秒，默认为 1000
-
### Keywords
CREATE, RESOURCE
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [typo](docs)fix some problem (#140)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new f105bfb7d96 [typo](docs)fix some problem (#140) f105bfb7d96 is described below commit f105bfb7d96b1ba3c0ad4f4af4d06b4dd396a8a7 Author: Liqf <109049295+lemonlit...@users.noreply.github.com> AuthorDate: Wed Oct 12 12:34:46 2022 +0800 [typo](docs)fix some problem (#140) --- docs/data-table/data-partition.md | 2 +- .../docusaurus-plugin-content-docs/current/data-table/data-partition.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/data-table/data-partition.md b/docs/data-table/data-partition.md index 441e43cee8e..dc733214e05 100644 --- a/docs/data-table/data-partition.md +++ b/docs/data-table/data-partition.md @@ -314,7 +314,7 @@ It is also possible to use only one layer of partitioning. When using a layer pa 2. Bucket * If a Partition is used, the `DISTRIBUTED ...` statement describes the division rules for the data in each partition. If you do not use Partition, it describes the rules for dividing the data of the entire table. -* The bucket column can be multiple columns, but it must be a Key column. The bucket column can be the same or different from the Partition column. +* The bucket column can be multiple columns,Aggregate and Unique models must be key columns, and Duplicate models can be key columns and value columns. The bucket column can be the same or different from the Partition column. * The choice of bucket column is a trade-off between **query throughput** and **query concurrency**: 1. If you select multiple bucket columns, the data is more evenly distributed. However, if the query condition does not include the equivalent condition for all bucket columns, a query will scan all buckets. The throughput of such queries will increase, and the latency of a single query will decrease. This method is suitable for large throughput and low concurrent query scenarios. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/data-partition.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/data-partition.md index 1e3be4b31b0..f166ed543f5 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/data-partition.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-table/data-partition.md @@ -321,7 +321,7 @@ Doris 支持两层的数据划分。第一层是 Partition，支持 Range 和 Li 2. **Bucket** - 如果使用了 Partition，则 `DISTRIBUTED ...` 语句描述的是数据在**各个分区内**的划分规则。如果不使用 Partition，则描述的是对整个表的数据的划分规则。 - - 分桶列可以是多列，但必须为 Key 列。分桶列可以和 Partition 列相同或不同。 + - 分桶列可以是多列，Aggregate 和 Unique 模型必须为 Key 列，Duplicate 模型可以是 key 列和 value 列。分桶列可以和 Partition 列相同或不同。 - 分桶列的选择，是在 **查询吞吐** 和 **查询并发** 之间的一种权衡： 1. 如果选择多个分桶列，则数据分布更均匀。如果一个查询条件不包含所有分桶列的等值条件，那么该查询会触发所有分桶同时扫描，这样查询的吞吐会增加，单个查询的延迟随之降低。这个方式适合大吞吐低并发的查询场景。 2. 如果仅选择一个或少数分桶列，则对应的点查询可以仅触发一个分桶扫描。此时，当多个点查询并发时，这些查询有较大的概率分别触发不同的分桶扫描，各个查询之间的IO影响较小（尤其当不同桶分布在不同磁盘上时），所以这种方式适合高并发的点查询场景。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: remove function
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 8e458c4b2d5 remove function 8e458c4b2d5 is described below commit 8e458c4b2d5bb530229c94fc068742b88a29defb Author: jiafeng.zhang AuthorDate: Thu Oct 13 18:59:14 2022 +0800 remove function --- sidebars.json | 1 - 1 file changed, 1 deletion(-) diff --git a/sidebars.json b/sidebars.json index 4b540048810..489b06be751 100644 --- a/sidebars.json +++ b/sidebars.json @@ -542,7 +542,6 @@ "sql-manual/sql-functions/window-functions/WINDOW-FUNCTION-FIRST-VALUE", "sql-manual/sql-functions/window-functions/WINDOW-FUNCTION-LAST-VALUE", "sql-manual/sql-functions/window-functions/WINDOW-FUNCTION-ROW-NUMBER", - "sql-manual/sql-functions/window-functions/WINDOW-FUNCTION-NTILE", "sql-manual/sql-functions/window-functions/WINDOW-FUNCTION-WINDOW-FUNNEL" ] }, - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: [docs](releasenote)update 1.1.3 release note (#142)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new c038daa934e [docs](releasenote)update 1.1.3 release note (#142)
c038daa934e is described below
commit c038daa934ee6ee3f3cbab239d8c40f7eb8f2d02
Author: Luzhijing <82810928+luzhij...@users.noreply.github.com>
AuthorDate: Fri Oct 14 08:57:26 2022 +0800
[docs](releasenote)update 1.1.3 release note (#142)
* Update release-1.1.3.md
---
docs/releasenotes/release-1.1.3.md | 79 ++
.../current/releasenotes/release-1.1.3.md | 77 +
2 files changed, 156 insertions(+)
diff --git a/docs/releasenotes/release-1.1.3.md
b/docs/releasenotes/release-1.1.3.md
new file mode 100644
index 000..69790eff170
--- /dev/null
+++ b/docs/releasenotes/release-1.1.3.md
@@ -0,0 +1,79 @@
+---
+{
+"title": "Release 1.1.3",
+"language": "en"
+}
+---
+
+
+
+
+In this release, Doris Team has fixed more than 80 issues or performance
improvement since 1.1.2. This release is a bugfix release on 1.1 and all users
are encouraged to upgrade to this release.
+
+
+# Features
+
+- Support escape identifiers for sqlserver and postgresql in ODBC table.
+
+- Could use Parquet as output file format.
+
+# Improvements
+
+- Optimize flush policy to avoid small segments.
[#12706](https://github.com/apache/doris/pull/12706)
[#12716](https://github.com/apache/doris/pull/12716)
+
+- Refactor runtime filter to reduce the prepare time.
[#13127](https://github.com/apache/doris/pull/13127)
+
+- Lots of memory control related issues during query or load process.
[#12682](https://github.com/apache/doris/pull/12682)
[#12688](https://github.com/apache/doris/pull/12688)
[#12708](https://github.com/apache/doris/pull/12708)
[#12776](https://github.com/apache/doris/pull/12776)
[#12782](https://github.com/apache/doris/pull/12782)
[#12791](https://github.com/apache/doris/pull/12791)
[#12794](https://github.com/apache/doris/pull/12794)
[#12820](https://github.com/apache/doris/pull/12820 [...]
+
+# BugFix
+
+- Core dump on compaction with largeint.
[#10094](https://github.com/apache/doris/pull/10094)
+
+- Grouping sets cause be core or return wrong results.
[#12313](https://github.com/apache/doris/pull/12313)
+
+- PREAGGREGATION flag in orthogonal_bitmap_union_count operator is wrong.
[#12581](https://github.com/apache/doris/pull/12581)
+
+- Level1Iterator should release iterators in heap and it may cause memory
leak. [#12592](https://github.com/apache/doris/pull/12592)
+
+- Fix decommission failure with 2 BEs and existing colocation table.
[#12644](https://github.com/apache/doris/pull/12644)
+
+- BE may core dump because of stack-buffer-overflow when
TBrokerOpenReaderResponse too large.
[#12658](https://github.com/apache/doris/pull/12658)
+
+- BE may OOM during load when error code -238 occurs.
[#12666](https://github.com/apache/doris/pull/12666)
+
+- Fix wrong child expression of lead function.
[#12587](https://github.com/apache/doris/pull/12587)
+
+- Fix intersect query failed in row storage code.
[#12712](https://github.com/apache/doris/pull/12712)
+
+- Fix wrong result produced by curdate()/current_date() function.
[#12720](https://github.com/apache/doris/pull/12720)
+
+- Fix lateral view explode_split with temp table bug.
[#13643](https://github.com/apache/doris/pull/13643)
+
+- Bucket shuffle join plan is wrong in two same table.
[#12930](https://github.com/apache/doris/pull/12930)
+
+- Fix bug that tablet version may be wrong when doing alter and load.
[#13070](https://github.com/apache/doris/pull/13070)
+
+- BE core when load data using broker with md5sum()/sm3sum().
[#13009](https://github.com/apache/doris/pull/13009)
+
+# Behavior Changes
+
+Disable PageCache and ChunkAllocator by default to reduce memory usage. User
could enable this by `changing disable_storage_page_cache` and
chunk_reserved_bytes_limit.
+
+
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/releasenotes/release-1.1.3.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/releasenotes/release-1.1.3.md
new file mode 100644
index 000..ecd26e09577
--- /dev/null
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/releasenotes/release-1.1.3.md
@@ -0,0 +1,77 @@
+---
+{
+"title": "Release 1.1.3",
+"language": "zh-CN"
+}
+---
+
+
+
+
+
+作为 1.1.2 LTS（Long-term Support，长周期支持）版本基础之上的 Bugfix 版本，在 Apache Doris 1.1.3
版本中，有超过 80 个 Issue 或性能优化项被合入，优化了在导入或查询过程中的内存控制，修复了许多导致 BE Core
以及产生错误查询结果的问题，系统稳定性和性能得以进一步加强，推荐所有用户下载和使用。
+
+# 新增功能
+
+- 在 ODBC 表中支持 SQLServer 和 PostgreSQL 的转义标识符。
+
+- 支持使用 Parquet 作为导出文件格式。
+
+# 优化改进
+
+- 优化了 Flush 策略以及避免过多 Segment 小文件。
[#12706](https://github.com/apache/doris/pull/12706)
[doris-website] branch master updated: Update sidebars.json (#143)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 757c7f0485f Update sidebars.json (#143) 757c7f0485f is described below commit 757c7f0485faceb8ab9a309725507b3ce738866a Author: Luzhijing <82810928+luzhij...@users.noreply.github.com> AuthorDate: Fri Oct 14 08:57:42 2022 +0800 Update sidebars.json (#143) --- sidebars.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sidebars.json b/sidebars.json index 489b06be751..08f7f0777ea 100644 --- a/sidebars.json +++ b/sidebars.json @@ -978,7 +978,8 @@ "items": [ "releasenotes/release-1.1.0", "releasenotes/release-1.1.1", -"releasenotes/release-1.1.2" +"releasenotes/release-1.1.2", +"releasenotes/release-1.1.3" ] } ] - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix-doc27 (#144)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 106b4fd2b22 fix-doc27 (#144)
106b4fd2b22 is described below
commit 106b4fd2b22032a71ca282746e98cfa3e9fa1858
Author: Liqf <109049295+lemonlit...@users.noreply.github.com>
AuthorDate: Fri Oct 14 10:40:05 2022 +0800
fix-doc27 (#144)
---
docs/ecosystem/udf/java-user-defined-function.md | 183 -
.../ecosystem/udf/java-user-defined-function.md| 182
2 files changed, 365 deletions(-)
diff --git a/docs/ecosystem/udf/java-user-defined-function.md
b/docs/ecosystem/udf/java-user-defined-function.md
deleted file mode 100644
index aff3c7874fc..000
--- a/docs/ecosystem/udf/java-user-defined-function.md
+++ /dev/null
@@ -1,183 +0,0 @@
-{
-"title": "Java UDF",
-"language": "en"
-}
-
-
-
-# Java UDF
-
-Java UDF provides users with a Java interface written in UDF to facilitate the
execution of user-defined functions in Java language. Compared with native UDF
implementation, Java UDF has the following advantages and limitations:
-1. The advantages
-* Compatibility: Using Java UDF can be compatible with different Doris
versions, so when upgrading Doris version, Java UDF does not need additional
migration. At the same time, Java UDF also follows the same programming
specifications as hive / spark and other engines, so that users can directly
move Hive / Spark UDF jar to Doris.
-* Security: The failure or crash of Java UDF execution will only cause the JVM
to report an error, not the Doris process to crash.
-* Flexibility: In Java UDF, users can package the third-party dependencies
together in the user jar.
-
-2. Restrictions on use
-* Performance: Compared with native UDF, Java UDF will bring additional JNI
overhead, but through batch execution, we have minimized the JNI overhead as
much as possible.
-* Vectorized engine: Java UDF is only supported on vectorized engine now.
-
-### Type correspondence
-
-|Type|UDF Argument Type|
-||-|
-|Bool|Boolean|
-|TinyInt|Byte|
-|SmallInt|Short|
-|Int|Integer|
-|BigInt|Long|
-|LargeInt|BigInteger|
-|Float|Float|
-|Double|Double|
-|Date|LocalDate|
-|Datetime|LocalDateTime|
-|Char|String|
-|Varchar|String|
-|Decimal|BigDecimal|
-
-## Write UDF functions
-
-This section mainly introduces how to develop a Java UDF. Samples for the Java
version are provided under `samples/doris-demo/java-udf-demo/` for your
reference, Check it out
[here](https://github.com/apache/incubator-doris/tree/master/samples/doris-demo/java-udf-demo)
-
-To use Java UDF, the main entry of UDF must be the `evaluate` function. This
is consistent with other engines such as Hive. In the example of `AddOne`, we
have completed the operation of adding an integer as the UDF.
-
-It is worth mentioning that this example is not only the Java UDF supported by
Doris, but also the UDF supported by Hive, that's to say, for users, Hive UDF
can be directly migrated to Doris.
-
-## Create UDF
-
-```sql
-CREATE FUNCTION
-name ([,...])
-[RETURNS] rettype
-PROPERTIES (["key"="value"][,...])
-```
-Instructions:
-
-1. `symbol` in properties represents the class name containing UDF classes.
This parameter must be set.
-2. The jar package containing UDF represented by `file` in properties must be
set.
-3. The UDF call type represented by `type` in properties is native by default.
When using java UDF, it is transferred to `Java_UDF`.
-4. `name`: A function belongs to a DB and name is of the
form`dbName`.`funcName`. When `dbName` is not explicitly specified, the db of
the current session is used`dbName`.
-
-Sample：
-```sql
-CREATE FUNCTION java_udf_add_one(int) RETURNS int PROPERTIES (
-"file"="file:///path/to/java-udf-demo-jar-with-dependencies.jar",
-"symbol"="org.apache.doris.udf.AddOne",
-"type"="JAVA_UDF"
-);
-```
-
-## Create UDAF
-
-When using Java code to write UDAF, there are some functions that must be
implemented (mark required) and an inner class State, which will be explained
with a specific example below.
-The following SimpleDemo will implement a simple function similar to sum, the
input parameter is INT, and the output parameter is INT
-
-```JAVA
-package org.apache.doris.udf;
-
-public class SimpleDemo {
-//Need an inner class to store data
-/*required*/
-public static class State {
-/*some variables if you need */
-public int sum = 0;
-}
-
-/*required*/
-public State create() {
-/* here could do some init work if needed */
-return new State();
-}
-
-/*required*/
-public void destroy(State state) {
- /* here could do some destroy work if
[doris-website] branch master updated: 1.1.3 upgrade
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new c3913cae3c3 1.1.3 upgrade c3913cae3c3 is described below commit c3913cae3c3cb480ec4fe289367e010b36114310 Author: jiafeng.zhang AuthorDate: Sat Oct 15 12:10:22 2022 +0800 1.1.3 upgrade --- docs/admin-manual/cluster-management/upgrade.md| 18 ++ .../current/admin-manual/cluster-management/upgrade.md | 18 ++ 2 files changed, 36 insertions(+) diff --git a/docs/admin-manual/cluster-management/upgrade.md b/docs/admin-manual/cluster-management/upgrade.md index a04a69a5394..10292366472 100644 --- a/docs/admin-manual/cluster-management/upgrade.md +++ b/docs/admin-manual/cluster-management/upgrade.md @@ -74,6 +74,24 @@ Doris can upgrade smoothly by rolling upgrades. The following steps are recommen ``` The metadata directory to be backed up is `doris-meta/` +3. **important! ! Upgrade 1.1.3 Caution** + +Storage Page Cache and Chunk Allocator need to be disabled by default. + +Storage Page Cache and Chunk Allocator cache user data blocks and memory preallocation, respectively. + +These two functions take up a certain percentage of memory and will not be released. This part of memory cannot be flexibly allocated, which leads to insufficient memory for other tasks due to this part of memory occupation in some scenarios, affecting system stability and availability. Therefore, we disabled these two features by default in version 1.1.3. + +However, in some latency-sensitive reporting scenarios, turning off this feature may lead to increased query latency. If you are worried about the impact of this feature on your business after upgrade, you can add the following parameters to be.conf to keep the same behavior as the previous version. + +``` +disable_storage_page_cache=false +chunk_reserved_bytes_limit=10% +``` + +* ``disable_storage_page_cache``: Whether to disable Storage Page Cache. version 1.1.2 (inclusive), the default is false, i.e., on. version 1.1.3 defaults to true, i.e., off. +* `chunk_reserved_bytes_limit`: Chunk allocator reserved memory size. 1.1.2 (and earlier), the default is 10% of the overall memory. 1.1.3 version default is 209715200 (200MB). + ## Test the correctness of BE upgrade 1. Arbitrarily select a BE node and deploy the latest doris_be binary file. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/upgrade.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/upgrade.md index befa55d6cfb..385026e4aea 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/upgrade.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/upgrade.md @@ -73,6 +73,24 @@ Doris 可以通过滚动升级的方式，平滑进行升级。建议按照以 ``` 需要备份的元数据目录为 `doris-meta/` +3. **升级 1.1.3 注意事项** + +需要默认关闭 Storage Page Cache 和 Chunk Allocator + +Storage Page Cache 和 Chunk Allocator 分别缓存用户数据块和内存预分配。 + +这两个功能会占用一定比例的内存，并且不会释放。 这部分内存占用无法灵活调配，导致在某些场景下，因这部分内存占用而导致其他任务内存不足，影响系统稳定性和可用性。因此我们在 1.1.3 版本中默认关闭了这两个功能。 + +但在某些延迟敏感的报表场景下，关闭该功能可能会导致查询延迟增加。如用户担心升级后该功能对业务造成影响，可以通过在 be.conf 中增加以下参数以保持和之前版本行为一致。 + +``` +disable_storage_page_cache=false +chunk_reserved_bytes_limit=10% +``` + +* `disable_storage_page_cache`：是否关闭 Storage Page Cache。 1.1.2（含）之前的版本，默认是false，即打开。1.1.3 版本默认为 true，即关闭。 +* `chunk_reserved_bytes_limit`：Chunk allocator 预留内存大小。1.1.2（含）之前的版本，默认是整体内存的 10%。1.1.3 版本默认为 209715200（200MB）。 + ## 测试 BE 升级正确性 1. 任意选择一个 BE 节点，部署最新的 doris_be 二进制文件。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: 1.1.3 release note
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 9fcf1e899c6 1.1.3 release note 9fcf1e899c6 is described below commit 9fcf1e899c6ac936e7f8b9e8b360ea81c50c1d85 Author: jiafeng.zhang AuthorDate: Sat Oct 15 12:21:07 2022 +0800 1.1.3 release note --- docs/admin-manual/cluster-management/upgrade.md| 18 -- docs/releasenotes/release-1.1.3.md | 17 +++-- .../current/admin-manual/cluster-management/upgrade.md | 18 -- .../current/releasenotes/release-1.1.3.md | 16 +++- 4 files changed, 30 insertions(+), 39 deletions(-) diff --git a/docs/admin-manual/cluster-management/upgrade.md b/docs/admin-manual/cluster-management/upgrade.md index 10292366472..a04a69a5394 100644 --- a/docs/admin-manual/cluster-management/upgrade.md +++ b/docs/admin-manual/cluster-management/upgrade.md @@ -74,24 +74,6 @@ Doris can upgrade smoothly by rolling upgrades. The following steps are recommen ``` The metadata directory to be backed up is `doris-meta/` -3. **important! ! Upgrade 1.1.3 Caution** - -Storage Page Cache and Chunk Allocator need to be disabled by default. - -Storage Page Cache and Chunk Allocator cache user data blocks and memory preallocation, respectively. - -These two functions take up a certain percentage of memory and will not be released. This part of memory cannot be flexibly allocated, which leads to insufficient memory for other tasks due to this part of memory occupation in some scenarios, affecting system stability and availability. Therefore, we disabled these two features by default in version 1.1.3. - -However, in some latency-sensitive reporting scenarios, turning off this feature may lead to increased query latency. If you are worried about the impact of this feature on your business after upgrade, you can add the following parameters to be.conf to keep the same behavior as the previous version. - -``` -disable_storage_page_cache=false -chunk_reserved_bytes_limit=10% -``` - -* ``disable_storage_page_cache``: Whether to disable Storage Page Cache. version 1.1.2 (inclusive), the default is false, i.e., on. version 1.1.3 defaults to true, i.e., off. -* `chunk_reserved_bytes_limit`: Chunk allocator reserved memory size. 1.1.2 (and earlier), the default is 10% of the overall memory. 1.1.3 version default is 209715200 (200MB). - ## Test the correctness of BE upgrade 1. Arbitrarily select a BE node and deploy the latest doris_be binary file. diff --git a/docs/releasenotes/release-1.1.3.md b/docs/releasenotes/release-1.1.3.md index 69790eff170..cfa7151097d 100644 --- a/docs/releasenotes/release-1.1.3.md +++ b/docs/releasenotes/release-1.1.3.md @@ -72,8 +72,21 @@ In this release, Doris Team has fixed more than 80 issues or performance improve - BE core when load data using broker with md5sum()/sm3sum(). [#13009](https://github.com/apache/doris/pull/13009) -# Behavior Changes +# Upgrade Notes -Disable PageCache and ChunkAllocator by default to reduce memory usage. User could enable this by `changing disable_storage_page_cache` and chunk_reserved_bytes_limit. +PageCache and ChunkAllocator are disabled by default to reduce memory usage and can be re-enabled by modifying the configuration items `disable_storage_page_cache` and `chunk_reserved_bytes_limit`. +Storage Page Cache and Chunk Allocator cache user data chunks and memory preallocation, respectively. + +These two functions take up a certain percentage of memory and are not freed. This part of memory cannot be flexibly allocated, which may lead to insufficient memory for other tasks in some scenarios, affecting system stability and availability. Therefore, we disabled these two features by default in version 1.1.3. + +However, in some latency-sensitive reporting scenarios, turning off this feature may lead to increased query latency. If you are worried about the impact of this feature on your business after upgrade, you can add the following parameters to be.conf to keep the same behavior as the previous version. + +``` +disable_storage_page_cache=false +chunk_reserved_bytes_limit=10% +``` + +* ``disable_storage_page_cache``: Whether to disable Storage Page Cache. version 1.1.2 (inclusive), the default is false, i.e., on. version 1.1.3 defaults to true, i.e., off. +* `chunk_reserved_bytes_limit`: Chunk allocator reserved memory size. 1.1.2 (and earlier), the default is 10% of the overall memory. 1.1.3 version default is 209715200 (200MB). diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/upgrade.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/cluster-management/upgrade.md index
[doris-website] branch master updated: add 1.1.3 download links (#145)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 7169b12a98e add 1.1.3 download links (#145)
7169b12a98e is described below
commit 7169b12a98eae6db6a97ba16291d9cc036f7959d
Author: song7788q
AuthorDate: Mon Oct 17 10:40:13 2022 +0800
add 1.1.3 download links (#145)
---
src/constant/download.data.ts | 169 ++
src/pages/download/index.tsx | 11 +--
2 files changed, 77 insertions(+), 103 deletions(-)
diff --git a/src/constant/download.data.ts b/src/constant/download.data.ts
index 6db94efb3ce..d4712707214 100644
--- a/src/constant/download.data.ts
+++ b/src/constant/download.data.ts
@@ -26,6 +26,70 @@ const CHINA_ALL_RELEASE_MIRROR_LINK =
'https://mirrors.tuna.tsinghua.edu.cn/apac
export function getAllDownloadLinks(locale: string) {
const sources = locale.toLocaleUpperCase() === 'EN' ? APACHE_LINK :
CHINA_MIRROR_LINK;
return [
+{
+id: '1.1.3-intel-avx2-jdk8',
+items: [
+{
+label: 'apache-doris-fe-1.1.3-bin.tar.gz',
+links: {
+source:
`${sources}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz`,
+signature:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz.asc`,
+sha512:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz.sha512`,
+},
+},
+{
+label: 'apache-doris-be-1.1.3-bin-x86_64.tar.gz',
+links: {
+source:
`${sources}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-x86_64.tar.gz`,
+signature:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-x86_64.tar.gz.asc`,
+sha512:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-x86_64.tar.gz.sha512`,
+},
+},
+],
+},
+{
+id: '1.1.3-intel-noavx2-jdk8',
+items: [
+{
+label: 'apache-doris-fe-1.1.3-bin.tar.gz',
+links: {
+source:
`${sources}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz`,
+signature:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz.asc`,
+sha512:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz.sha512`,
+},
+},
+{
+label: 'apache-doris-be-1.1.3-bin-x86_64-noavx2.tar.gz',
+links: {
+source:
`${sources}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-x86_64-noavx2.tar.gz`,
+signature:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-x86_64-noavx2.tar.gz.asc`,
+sha512:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-x86_64-noavx2.tar.gz.sha512`,
+},
+},
+],
+},
+{
+id: '1.1.3-arm-jdk8',
+items: [
+{
+label: 'apache-doris-fe-1.1.3-bin.tar.gz',
+links: {
+source:
`${sources}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz`,
+signature:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz.asc`,
+sha512:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-fe-1.1.3-bin.tar.gz.sha512`,
+},
+},
+{
+label: 'apache-doris-be-1.1.3-bin-arm.tar.gz',
+links: {
+source:
`${sources}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-arm.tar.gz`,
+signature:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-arm.tar.gz.asc`,
+sha512:
`${APACHE_LINK}doris/1.1/1.1.3-rc02/apache-doris-be-1.1.3-bin-arm.tar.gz.sha512`,
+},
+},
+],
+},
+
{
id: '1.1.2-intel-avx2-jdk8',
items: [
@@ -89,99 +153,6 @@ export function getAllDownloadLinks(locale: string) {
},
],
},
-{
-id: '1.1.1-intel-avx2-jdk8',
-items: [
-{
-label: 'apache-doris-1.1.1-bin-x64.tar.gz',
-links: {
-source:
`${sources}doris/1.1/1.1.1-r
[doris-website] branch master updated: update homepage news (#146)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 77d9e7e7752 update homepage news (#146)
77d9e7e7752 is described below
commit 77d9e7e77526e10aa42927a3a74a612dad762e1b
Author: wangyongfeng <943155...@qq.com>
AuthorDate: Mon Oct 17 10:41:34 2022 +0800
update homepage news (#146)
* update homepage news
* update news
Co-authored-by: wangyf0555
---
i18n/zh-CN/code.json | 4 ++--
i18n/zh-CN/docusaurus-plugin-content-docs/current.json | 2 +-
i18n/zh-CN/docusaurus-theme-classic/footer.json| 2 +-
src/pages/index.tsx| 4 ++--
4 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/i18n/zh-CN/code.json b/i18n/zh-CN/code.json
index 0c2a2b3424a..8f9154245db 100644
--- a/i18n/zh-CN/code.json
+++ b/i18n/zh-CN/code.json
@@ -315,7 +315,7 @@
"message": "Doris 团队"
},
"homepage.news": {
-"message": "2022.9.13，Apache Doris 1.1.2 版本正式发布",
+"message": "2022.10.17，Apache Doris 1.1.3 版本正式发布",
"description": "The label for the link to homepage news"
},
"homepage.what": {
@@ -762,4 +762,4 @@
"download.quick.download.version.tips": {
"message": "apache-doris-1.1.2-bin-x86-noavx2 版本暂不提供下载，我们会在进行更新。"
}
-}
\ No newline at end of file
+}
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
b/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
index c3aa673266f..34e9d140db6 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current.json
@@ -69,7 +69,7 @@
},
"sidebar.docs.category.Data Cache": {
"message": "数据缓存",
-"description": "The label for category Date Cache in sidebar docs"
+"description": "The label for category Data Cache in sidebar docs"
},
"sidebar.docs.category.Best Practice": {
"message": "最佳实践",
diff --git a/i18n/zh-CN/docusaurus-theme-classic/footer.json
b/i18n/zh-CN/docusaurus-theme-classic/footer.json
index 64ceb328f96..5304ade6c58 100644
--- a/i18n/zh-CN/docusaurus-theme-classic/footer.json
+++ b/i18n/zh-CN/docusaurus-theme-classic/footer.json
@@ -13,7 +13,7 @@
},
"link.item.label.Docs": {
"message": "文档",
-"description": "The label of footer link with label=Docs linking to
/docs/get-starting"
+"description": "The label of footer link with label=Docs linking to
/learning"
},
"link.item.label.Foundation": {
"message": "基金会",
diff --git a/src/pages/index.tsx b/src/pages/index.tsx
index 81419ff24b3..c9dbb4faf81 100644
--- a/src/pages/index.tsx
+++ b/src/pages/index.tsx
@@ -151,9 +151,9 @@ export default function Home(): JSX.Element {
-
+
-2022.9.13, Apache Doris 1.1.2 is
officially released
+2022.10.17, Apache Doris 1.1.3 is
officially released
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: release 1.1.3
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 6e42c09a864 release 1.1.3
6e42c09a864 is described below
commit 6e42c09a86455d7c507fe2c9dec02d137ef331dc
Author: jiafeng.zhang
AuthorDate: Mon Oct 17 12:04:40 2022 +0800
release 1.1.3
---
blog/release-1.1.3.md | 95 ++
.../release-1.1.3.md | 94 +
2 files changed, 189 insertions(+)
diff --git a/blog/release-1.1.3.md b/blog/release-1.1.3.md
new file mode 100644
index 000..e8d55c4a642
--- /dev/null
+++ b/blog/release-1.1.3.md
@@ -0,0 +1,95 @@
+---
+{
+'title': 'Apache Doris announced the official release of version 1.1.3',
+'summary': 'Dear community, we are pleased to announce that we have
officially released Apache Doris 1.1.3 on October 17, 2022! This release is a
hotfix version of 1.1.2',
+'date': '2022-09-13',
+'author': 'Luzhijing',
+'tags': ['Release Notes'],
+}
+---
+
+
+
+
+In this release, Doris Team has fixed more than 80 issues or performance
improvement since 1.1.2. This release is a bugfix release on 1.1 and all users
are encouraged to upgrade to this release.
+
+
+# Features
+
+- Support escape identifiers for sqlserver and postgresql in ODBC table.
+
+- Could use Parquet as output file format.
+
+# Improvements
+
+- Optimize flush policy to avoid small segments.
[#12706](https://github.com/apache/doris/pull/12706)
[#12716](https://github.com/apache/doris/pull/12716)
+
+- Refactor runtime filter to reduce the prepare time.
[#13127](https://github.com/apache/doris/pull/13127)
+
+- Lots of memory control related issues during query or load process.
[#12682](https://github.com/apache/doris/pull/12682)
[#12688](https://github.com/apache/doris/pull/12688)
[#12708](https://github.com/apache/doris/pull/12708)
[#12776](https://github.com/apache/doris/pull/12776)
[#12782](https://github.com/apache/doris/pull/12782)
[#12791](https://github.com/apache/doris/pull/12791)
[#12794](https://github.com/apache/doris/pull/12794)
[#12820](https://github.com/apache/doris/pull/12820 [...]
+
+# BugFix
+
+- Core dump on compaction with largeint.
[#10094](https://github.com/apache/doris/pull/10094)
+
+- Grouping sets cause be core or return wrong results.
[#12313](https://github.com/apache/doris/pull/12313)
+
+- PREAGGREGATION flag in orthogonal_bitmap_union_count operator is wrong.
[#12581](https://github.com/apache/doris/pull/12581)
+
+- Level1Iterator should release iterators in heap and it may cause memory
leak. [#12592](https://github.com/apache/doris/pull/12592)
+
+- Fix decommission failure with 2 BEs and existing colocation table.
[#12644](https://github.com/apache/doris/pull/12644)
+
+- BE may core dump because of stack-buffer-overflow when
TBrokerOpenReaderResponse too large.
[#12658](https://github.com/apache/doris/pull/12658)
+
+- BE may OOM during load when error code -238 occurs.
[#12666](https://github.com/apache/doris/pull/12666)
+
+- Fix wrong child expression of lead function.
[#12587](https://github.com/apache/doris/pull/12587)
+
+- Fix intersect query failed in row storage code.
[#12712](https://github.com/apache/doris/pull/12712)
+
+- Fix wrong result produced by curdate()/current_date() function.
[#12720](https://github.com/apache/doris/pull/12720)
+
+- Fix lateral view explode_split with temp table bug.
[#13643](https://github.com/apache/doris/pull/13643)
+
+- Bucket shuffle join plan is wrong in two same table.
[#12930](https://github.com/apache/doris/pull/12930)
+
+- Fix bug that tablet version may be wrong when doing alter and load.
[#13070](https://github.com/apache/doris/pull/13070)
+
+- BE core when load data using broker with md5sum()/sm3sum().
[#13009](https://github.com/apache/doris/pull/13009)
+
+# Upgrade Notes
+
+PageCache and ChunkAllocator are disabled by default to reduce memory usage
and can be re-enabled by modifying the configuration items
`disable_storage_page_cache` and `chunk_reserved_bytes_limit`.
+
+Storage Page Cache and Chunk Allocator cache user data chunks and memory
preallocation, respectively.
+
+These two functions take up a certain percentage of memory and are not freed.
This part of memory cannot be flexibly allocated, which may lead to
insufficient memory for other tasks in some scenarios, affecting system
stability and availability. Therefore, we disabled these two features by
default in version 1.1.3.
+
+However, in some latency-sensitive reporting scenarios, turning off this
feature may lead to increased query latency. If you are worried about the
impact of this feature on your business after upgrade,
[doris-website] branch master updated: fix-doc27 (#147)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 5e90943087a fix-doc27 (#147) 5e90943087a is described below commit 5e90943087aa6a7d695d4f620d968571a9673283 Author: Liqf <109049295+lemonlit...@users.noreply.github.com> AuthorDate: Wed Oct 19 11:49:51 2022 +0800 fix-doc27 (#147) --- docs/benchmark/tpch.md | 2 +- docs/data-operate/import/import-way/broker-load-manual.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-docs/current/benchmark/tpch.md | 2 +- .../current/data-operate/import/import-way/broker-load-manual.md| 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/benchmark/tpch.md b/docs/benchmark/tpch.md index c4319022d34..f6cebadb6b2 100644 --- a/docs/benchmark/tpch.md +++ b/docs/benchmark/tpch.md @@ -196,7 +196,7 @@ sh ./load-tpch-data.sh ### 7.5 Check Imported Data -Execute the following SQL statement to check the imported data volume is consistent with the above data volume. +Execute the following SQL statement to check that the imported data is consistent with the above data. ```sql select count(*) from lineitem; diff --git a/docs/data-operate/import/import-way/broker-load-manual.md b/docs/data-operate/import/import-way/broker-load-manual.md index 64607a71f16..c4d8502745e 100644 --- a/docs/data-operate/import/import-way/broker-load-manual.md +++ b/docs/data-operate/import/import-way/broker-load-manual.md @@ -28,7 +28,7 @@ under the License. Broker load is an asynchronous import method, and the supported data sources depend on the data sources supported by the [Broker](../../../advanced/broker.md) process. -Because the data in the Doris table is ordered, Broker load uses the doris cluster resources to sort the data when importing data. To complete the migration of massive historical data for Spark load, the Doris cluster resource usage is relatively large. , this method is used when the user does not have Spark computing resources. If there are Spark computing resources, it is recommended to use [Spark load](../../../data-operate/import/import-way/spark-load-manual.md). +Because the data in the Doris table is ordered, Broker load uses the doris cluster resources to sort the data when importing data. Complete massive historical data migration relative to Spark load, the Doris cluster resource usage is relatively large. , this method is used when the user does not have Spark computing resources. If there are Spark computing resources, it is recommended to use [Spark load](../../../data-operate/import/import-way/spark-load-manual.md). Users need to create [Broker load](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Load/BROKER-LOAD.md) import through MySQL protocol and import by viewing command to check the import result. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/benchmark/tpch.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/benchmark/tpch.md index 64f8d80ffb4..e63c0625bae 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/benchmark/tpch.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/benchmark/tpch.md @@ -199,7 +199,7 @@ sh ./load-tpch-data.sh ### 7.5 检查导入数据 -执行下面的 SQL 语句检查导入的数据量上 上面的数据量是一致。 +执行下面的 SQL 语句检查导入的数据与上面的数据量是一致。 ```sql select count(*) from lineitem; diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md index ba77d591635..73b1c297c8e 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md @@ -28,7 +28,7 @@ under the License. Broker load 是一个异步的导入方式，支持的数据源取决于 [Broker](../../../advanced/broker.md) 进程支持的数据源。 -因为 Doris 表里的数据是有序的，所以 Broker load 在导入数据的时是要利用doris 集群资源对数据进行排序，想对于 Spark load 来完成海量历史数据迁移，对 Doris 的集群资源占用要比较大，这种方式是在用户没有 Spark 这种计算资源的情况下使用，如果有 Spark 计算资源建议使用 [Spark load](../../../data-operate/import/import-way/spark-load-manual.md)。 +因为 Doris 表里的数据是有序的，所以 Broker load 在导入数据的时是要利用doris 集群资源对数据进行排序，相对于 Spark load 来完成海量历史数据迁移，对 Doris 的集群资源占用要比较大，这种方式是在用户没有 Spark 这种计算资源的情况下使用，如果有 Spark 计算资源建议使用 [Spark load](../../../data-operate/import/import-way/spark-load-manual.md)。 用户需要通过 MySQL 协议 创建 [Broker load](../../../sql-manual/sql-reference/Data-Manipulation-Statements/Load/BROKER-LOAD.md) 导入，并通过查看导入命令检查导入结果。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 5f22c23977b fix
5f22c23977b is described below
commit 5f22c23977b089927a98a75170df302248c1ff16
Author: jiafeng.zhang
AuthorDate: Wed Oct 19 18:01:23 2022 +0800
fix
---
blog/release-1.1.3.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/blog/release-1.1.3.md b/blog/release-1.1.3.md
index e8d55c4a642..675b1e66c78 100644
--- a/blog/release-1.1.3.md
+++ b/blog/release-1.1.3.md
@@ -2,7 +2,7 @@
{
'title': 'Apache Doris announced the official release of version 1.1.3',
'summary': 'Dear community, we are pleased to announce that we have
officially released Apache Doris 1.1.3 on October 17, 2022! This release is a
hotfix version of 1.1.2',
-'date': '2022-09-13',
+'date': '2022-10-17',
'author': 'Luzhijing',
'tags': ['Release Notes'],
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new c8325895d70 fix c8325895d70 is described below commit c8325895d702b687fff219d1f2833dce6ef78fb3 Author: jiafeng.zhang AuthorDate: Wed Oct 19 18:07:56 2022 +0800 fix --- blog/1.1 Release.md | 2 +- blog/release-1.1.1.md| 2 +- blog/release-1.1.2.md| 2 +- blog/release-1.1.3.md| 2 +- blog/release-note-0.15.0.md | 4 ++-- blog/release-note-1.0.0.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-blog/1.1 Release.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-blog/release-1.1.1.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-blog/release-1.1.2.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-blog/release-1.1.3.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-blog/release-note-0.15.0.md | 6 +++--- i18n/zh-CN/docusaurus-plugin-content-blog/release-note-1.0.0.md | 2 +- 12 files changed, 15 insertions(+), 15 deletions(-) diff --git a/blog/1.1 Release.md b/blog/1.1 Release.md index 44388303aef..87fb37153ff 100644 --- a/blog/1.1 Release.md +++ b/blog/1.1 Release.md @@ -3,7 +3,7 @@ 'title': 'Apache Doris announced the official release of version 1.1', 'summary': 'Dear community, we are pleased to announce that we have officially released Apache Doris 1.1 on July 14, 2022! This is the first release version after Apache Doris graduated from the Apache incubator and became an Apache Top-Level Project.', 'date': '2022-07-14', -'author': 'Luzhijing', +'author': 'Apache Doris', 'tags': ['Release Notes'], } --- diff --git a/blog/release-1.1.1.md b/blog/release-1.1.1.md index c6e5dabf76a..38921c0a658 100644 --- a/blog/release-1.1.1.md +++ b/blog/release-1.1.1.md @@ -3,7 +3,7 @@ 'title': 'Apache Doris announced the official release of version 1.1.1', 'summary': 'Dear community, we are pleased to announce that we have officially released Apache Doris 1.1.1 on July 29, 2022! This release is a hotfix version of 1.1.0', 'date': '2022-07-29', -'author': 'Luzhijing', +'author': 'Apache Doris', 'tags': ['Release Notes'], } --- diff --git a/blog/release-1.1.2.md b/blog/release-1.1.2.md index cbd5f423f13..569746da480 100644 --- a/blog/release-1.1.2.md +++ b/blog/release-1.1.2.md @@ -3,7 +3,7 @@ 'title': 'Apache Doris announced the official release of version 1.1.2', 'summary': 'Dear community, we are pleased to announce that we have officially released Apache Doris 1.1.2 on September 13, 2022! This release is a hotfix version of 1.1.1', 'date': '2022-09-13', -'author': 'Luzhijing', +'author': 'Apache Doris', 'tags': ['Release Notes'], } --- diff --git a/blog/release-1.1.3.md b/blog/release-1.1.3.md index 675b1e66c78..864b6f37acb 100644 --- a/blog/release-1.1.3.md +++ b/blog/release-1.1.3.md @@ -3,7 +3,7 @@ 'title': 'Apache Doris announced the official release of version 1.1.3', 'summary': 'Dear community, we are pleased to announce that we have officially released Apache Doris 1.1.3 on October 17, 2022! This release is a hotfix version of 1.1.2', 'date': '2022-10-17', -'author': 'Luzhijing', +'author': 'Apache Doris', 'tags': ['Release Notes'], } --- diff --git a/blog/release-note-0.15.0.md b/blog/release-note-0.15.0.md index 9ec4c9f3392..103f4dc06d1 100644 --- a/blog/release-note-0.15.0.md +++ b/blog/release-note-0.15.0.md @@ -3,7 +3,7 @@ 'title': '[Release Note] Apache Doris(Incubating) 0.15.0 Release', 'summary': '[Release Note] Apache Doris(Incubating) 0.15.0 Release', 'date': '2021-11-29', -'author': 'Luzhijing', +'author': 'Apache Doris', 'tags': ['Release Notes'], } --- @@ -63,7 +63,7 @@ It can be enabled via the session variable `set enable_cost_based_join_reorder=t * Support Flink-Doris-Connector. * Support for DataX doriswriter plugin. * Spark-Doris-Connector support for data writing to Doris. - + ## Feature Opti
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 851157aa774 fix 851157aa774 is described below commit 851157aa774571c10a7eb29a7f68e25437971243 Author: jiafeng.zhang AuthorDate: Sat Oct 22 19:33:13 2022 +0800 fix --- .../sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md | 2 ++ .../sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md index f6a33014f5a..32780c16efb 100644 --- a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md +++ b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md @@ -232,6 +232,8 @@ distribution_desc `DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` +Suggestion: It is recommended to use random bucketing when there is no suitable key for hash bucketing to make the data of the table evenly distributed. + * `rollup_list` Multiple materialized views (ROLLUP) can be created at the same time as the table is built. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md index c9cbcf78be8..988243a51c8 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md @@ -233,6 +233,8 @@ distribution_desc `DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` + 建议: 当没有合适的key做哈希分桶使得表的数据均匀分布的时候，建议使用RANDOM分桶方式。 + * `rollup_list` 建表的同时可以创建多个物化视图（ROLLUP）。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new a0cd6239cb2 fix a0cd6239cb2 is described below commit a0cd6239cb26bd39bbb3572587559e9380dfbfb2 Author: jiafeng.zhang AuthorDate: Sat Oct 22 19:55:29 2022 +0800 fix --- .../Create/CREATE-TABLE.md | 3 +- .../Load/BROKER-LOAD.md| 9 ++ .../Load/CREATE-ROUTINE-LOAD.md| 8 ++ .../Load/STREAM-LOAD.md| 126 ++-- .../Load/BROKER-LOAD.md| 6 + .../Load/CREATE-ROUTINE-LOAD.md| 9 ++ .../Load/STREAM-LOAD.md| 128 +++-- 7 files changed, 163 insertions(+), 126 deletions(-) diff --git a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md index 32780c16efb..d99832c852e 100644 --- a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md +++ b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md @@ -230,7 +230,8 @@ distribution_desc Define the data bucketing method. -`DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` +1. Hash Syntax: `DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` Explain: Hash bucketing using the specified key column. +2. Random Syntax: `DISTRIBUTED BY RANDOM [BUCKETS num]` Explain: Use random numbers for bucketing. Suggestion: It is recommended to use random bucketing when there is no suitable key for hash bucketing to make the data of the table evenly distributed. diff --git a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/BROKER-LOAD.md b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/BROKER-LOAD.md index 881ff65a56e..d033478033e 100644 --- a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/BROKER-LOAD.md +++ b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/BROKER-LOAD.md @@ -167,6 +167,15 @@ WITH BROKER broker_name - `timezone` Specify the time zone for some functions that are affected by time zones, such as `strftime/alignment_timestamp/from_unixtime`, etc. Please refer to the [timezone](../../../../advanced/time-zone) documentation for details. If not specified, the "Asia/Shanghai" timezone is used + + - `send_batch_parallelism` : + +Used to set the default parallelism for sending batch, if the value for parallelism exceed `max_send_batch_parallelism_per_job` in BE config, then the coordinator BE will use the value of `max_send_batch_parallelism_per_job`. + + - `load_to_single_tablet` + + Boolean type, True means that one task can only load data to one tablet in the corresponding partition at a time. The default value is false. The number of tasks for the job depends on the overall concurrency. This parameter can only be set when loading data into the OLAP table with random partition. + ### Example diff --git a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-ROUTINE-LOAD.md b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-ROUTINE-LOAD.md index c6ec707cf9b..4988946c158 100644 --- a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-ROUTINE-LOAD.md +++ b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/CREATE-ROUTINE-LOAD.md @@ -219,6 +219,14 @@ FROM data_source [data_source_properties] `-H "json_root: $.RECORDS"` + 10. `send_batch_parallelism` : + + Integer, Used to set the default parallelism for sending batch, if the value for parallelism exceed `max_send_batch_parallelism_per_job` in BE config, then the coordinator BE will use the value of `max_send_batch_parallelism_per_job`. + + 11. `load_to_single_tablet` + + Boolean type, True means that one task can only load data to one tablet in the corresponding partition at a time. The default value is false. This parameter can only be set when loading data into the OLAP table with random partition. + - `FROM data_source [data_source_properties]` The type of data source. Currently supports: diff --git a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md index e38fa2b684a..3e7ded9a1f7 100644 --- a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md +++ b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md @@ -53,19 +53,19 @@ In addition, it is best for users to set the content of the Expect Header field Parameter introduction: Users can pass
[doris-website] branch master updated: improve the outfile doc (#148)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new ba38b47eba2 improve the outfile doc (#148) ba38b47eba2 is described below commit ba38b47eba26080a0365e2bdfe8a9fcd7e7de75a Author: zy-kkk AuthorDate: Sat Oct 22 20:01:28 2022 +0800 improve the outfile doc (#148) --- docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md | 2 +- .../sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md index 676949581f3..65410eeba68 100644 --- a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md +++ b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md @@ -206,7 +206,7 @@ illustrate: Verify on cos 1. A path that does not exist will be automatically created - 2. Access.key/secret.key/endpoint needs to be confirmed with cos students. Especially the value of endpoint does not need to fill in bucket_name. + 2. Access.key/secret.key/endpoint needs to be confirmed with students of cos. Especially the value of endpoint does not need to fill in bucket_name. 6. Use the s3 protocol to export to bos, and enable concurrent export. diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md index 026d83321c5..7afc83093e2 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md @@ -204,7 +204,7 @@ INTO OUTFILE "file_path" 在cos上验证 1. 不存在的path会自动创建 -2. access.key/secret.key/endpoint需要和cos的同学确认。尤其是endpoint的值，不需要填写bucket_name。 +2. access.key/secret.key/endpoint需要和cos的同学确认。尤其是endpoint的值，不需要填写bucket_name。 6. 使用 s3 协议导出到 bos，并且并发导出开启。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 53ca9c12978 fix
53ca9c12978 is described below
commit 53ca9c12978f474ecaab1618d16fe675208fe5a9
Author: jiafeng.zhang
AuthorDate: Sun Oct 23 10:49:36 2022 +0800
fix
---
.../Create/CREATE-POLICY.md| 24 --
.../Create/CREATE-POLICY.md| 22
2 files changed, 46 deletions(-)
diff --git
a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
index 73e7b240c7d..616333f4194 100644
---
a/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
+++
b/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
@@ -59,12 +59,6 @@ PROPERTIES ("key"="value", ...);
illustrate：
- PROPERTIES has such keys:
1. storage_resource：storage resource name for policy
-2. cooldown_datetime：cool down time for tablet, can't be set with
cooldown_ttl.
-3. cooldown_ttl：hot data stay time. The time cost between the time of
tablet created and
-the time of migrated to cold data, formatted as：
-1d：1 day
-1h：1 hour
-5: 5 second
### Example
@@ -93,24 +87,6 @@ illustrate：
select * from (select * from table1 where c1 = 'a' and c2 = 'b' or c3 = 'c'
or c4 = 'd')
```
-2. Create policy for storage
-1. Create policy on cooldown_datetime
-```sql
-CREATE STORAGE POLICY testPolicy
-PROPERTIES(
- "storage_resource" = "s3",
- "cooldown_datetime" = "2022-06-08 00:00:00"
-);
-```
-2. Create policy on cooldown_ttl
-```sql
-CREATE STORAGE POLICY testPolicy
-PROPERTIES(
- "storage_resource" = "s3",
- "cooldown_ttl" = "1d"
-);
-```
-
### Keywords
CREATE, POLICY
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
index b3b9c7f041b..e90bf42398f 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-POLICY.md
@@ -59,11 +59,6 @@ PROPERTIES ("key"="value", ...);
参数说明：
- PROPERTIES中需要指定资源的类型:
1. storage_resource：指定策略使用的storage resource名称。
-2. cooldown_datetime：热数据转为冷数据时间，不能与cooldown_ttl同时存在。
-3. cooldown_ttl：热数据持续时间。从数据分片生成时开始计算，经过指定时间后转为冷数据。支持的格式：
-1d：1天
-1h：1小时
-5: 5秒
### Example
@@ -91,23 +86,6 @@ PROPERTIES ("key"="value", ...);
```sql
select * from (select * from table1 where c1 = 'a' and c2 = 'b' or c3 = 'c'
or c4 = 'd')
```
-2. 创建数据迁移策略
-1. 指定数据冷却时间创建数据迁移策略
-```sql
-CREATE STORAGE POLICY testPolicy
-PROPERTIES(
- "storage_resource" = "s3",
- "cooldown_datetime" = "2022-06-08 00:00:00"
-);
-```
-2. 指定热数据持续时间创建数据迁移策略
-```sql
-CREATE STORAGE POLICY testPolicy
-PROPERTIES(
- "storage_resource" = "s3",
- "cooldown_ttl" = "1d"
-);
-```
### Keywords
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: update create table random bucket
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4abeb82388c update create table random bucket 4abeb82388c is described below commit 4abeb82388c54f776f1a3257c7bdc0dd1afe70ed Author: jiafeng.zhang AuthorDate: Mon Oct 24 19:53:18 2022 +0800 update create table random bucket --- .../sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md index 988243a51c8..a8660daff9d 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE.md @@ -231,9 +231,9 @@ distribution_desc 定义数据分桶方式。 -`DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` +1) Hash 分桶 语法： `DISTRIBUTED BY HASH (k1[,k2 ...]) [BUCKETS num]` 说明： 使用指定的 key 列进行哈希分桶。 - 建议: 当没有合适的key做哈希分桶使得表的数据均匀分布的时候，建议使用RANDOM分桶方式。 + 2. Random 分桶 语法： `DISTRIBUTED BY RANDOM [BUCKETS num]` 说明： 使用随机数进行分桶。 * `rollup_list` - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: link 404
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4b205bca302 link 404 4b205bca302 is described below commit 4b205bca302e524e8d412cc101997e048a1f5de4 Author: jiafeng.zhang AuthorDate: Tue Oct 25 15:52:17 2022 +0800 link 404 --- docs/admin-manual/config/be-config.md | 2 +- .../maint-monitor/metadata-operation.md| 2 +- docs/admin-manual/maint-monitor/multi-tenant.md| 238 - .../maint-monitor/tablet-repair-and-balance.md | 2 +- docs/advanced/alter-table/replace-table.md | 2 +- docs/advanced/alter-table/schema-change.md | 2 +- docs/data-operate/export/outfile.md| 2 +- .../import/import-scenes/external-storage-load.md | 2 +- .../data-operate/import/import-scenes/jdbc-load.md | 2 +- .../import/import-scenes/kafka-load.md | 2 +- docs/data-table/basic-usage.md | 2 +- docs/data-table/hit-the-rollup.md | 2 +- docs/ecosystem/doris-manager/space-list.md | 2 +- docs/ecosystem/logstash.md | 2 +- docs/ecosystem/udf/contribute-udf.md | 2 +- docs/faq/install-faq.md| 8 +- .../Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Alter/ALTER-TABLE-REPLACE.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Drop/DROP-DATABASE.md | 2 +- .../Data-Definition-Statements/Drop/DROP-TABLE.md | 2 +- .../Load/BROKER-LOAD.md| 18 +- .../Load/CREATE-SYNC-JOB.md| 2 +- .../Load/STREAM-LOAD.md| 10 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../cluster-management/elastic-expansion.md| 2 +- .../current/admin-manual/config/be-config.md | 2 +- .../maint-monitor/metadata-operation.md| 2 +- .../maint-monitor/tablet-repair-and-balance.md | 2 +- .../current/admin-manual/multi-tenant.md | 232 .../current/advanced/alter-table/replace-table.md | 2 +- .../current/advanced/alter-table/schema-change.md | 2 +- .../import/import-scenes/external-storage-load.md | 2 +- .../data-operate/import/import-scenes/jdbc-load.md | 2 +- .../current/data-table/basic-usage.md | 2 +- .../current/data-table/hit-the-rollup.md | 2 +- .../current/ecosystem/doris-manager/space-list.md | 2 +- .../current/ecosystem/logstash.md | 2 +- .../current/ecosystem/udf/contribute-udf.md| 2 +- .../Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Alter/ALTER-TABLE-REPLACE.md | 4 +- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Drop/DROP-DATABASE.md | 2 +- .../Data-Definition-Statements/Drop/DROP-TABLE.md | 2 +- .../Load/BROKER-LOAD.md| 22 +- .../Load/CREATE-SYNC-JOB.md| 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../alter-table/alter-table-bitmap-index.md| 20 +- .../alter-table/alter-table-replace-table.md | 4 +- .../alter-table/alter-table-temp-partition.md | 2 +- .../administrator-guide/config/fe_config.md| 2 +- .../load-data/broker-load-manual.md| 2 +- .../load-data/routine-load-manual.md | 2 +- .../administrator-guide/operation/disk-capacity.md | 4 +- .../version-0.15/administrator-guide/outfile.md| 2 +- .../administrator-guide/resource-management.md | 2 +- .../best-practices/star-schema-benchmark.md| 8 +- .../version-0.15/extending-doris/logstash.md | 4 +- .../alter-table/alter-table-bitmap-index.md| 18 +- .../alter-table/alter-table-replace-table.md | 4 +- .../alter-table/alter-table-temp-partition.md | 2 +- .../administrator-guide/config/fe_config.md| 2 +- .../load-data/broker-load-manual.md| 2 +- .../load-data/routine-load-manual.md | 2 +- .../administrator-guide/operation/disk-capacity.md | 4 +- .../version-0.15/administrator-guide/outfile.md| 2 +- .../administrator-guide/resource-management.md | 2 +- .../best-practices/star-schema-benchmark.md| 4 +- 68 files changed, 118 insertions(+), 588 deletions(-) diff --git a/docs/admin-manual/config/be-config.md b/docs/admin-manual/config/be-config.md index 66282b1acfb..e27118a83ed 100644 --- a/docs/admin-manual/config/be-config.md +++ b/docs/admin-manual/config/be-config.md @@ -450,7 +450,7 @@ Cgroups assigned to doris ### `doris_max_scan_key_num` * Type: int
[doris-website] branch master updated: SHOW-ALTER-TABLE-MATERIALIZED-VIEW
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 888cdfe6d4a SHOW-ALTER-TABLE-MATERIALIZED-VIEW 888cdfe6d4a is described below commit 888cdfe6d4aaf489669100a4857b4f46ed6e83c9 Author: jiafeng.zhang AuthorDate: Tue Oct 25 16:32:43 2022 +0800 SHOW-ALTER-TABLE-MATERIALIZED-VIEW --- sidebars.json | 1 + 1 file changed, 1 insertion(+) diff --git a/sidebars.json b/sidebars.json index 08f7f0777ea..0c58b003d39 100644 --- a/sidebars.json +++ b/sidebars.json @@ -720,6 +720,7 @@ "type": "category", "label": "Show", "items": [ + "sql-manual/sql-reference/Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW", "sql-manual/sql-reference/Show-Statements/SHOW-ALTER", "sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS", "sql-manual/sql-reference/Show-Statements/SHOW-BACKUP", - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 8e47a9294cf fix 8e47a9294cf is described below commit 8e47a9294cf95871020eb025dc5b5efe7eb7c6d0 Author: jiafeng.zhang AuthorDate: Tue Oct 25 18:05:56 2022 +0800 fix --- blog/Annoucing.md | 2 +- .../cluster-management/elastic-expansion.md| 4 ++-- docs/advanced/alter-table/replace-table.md | 4 ++-- .../import/import-scenes/external-storage-load.md | 2 +- docs/data-operate/import/import-scenes/jdbc-load.md| 4 ++-- docs/data-table/basic-usage.md | 14 +++--- docs/data-table/hit-the-rollup.md | 4 ++-- .../Alter/ALTER-TABLE-REPLACE.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- .../Data-Definition-Statements/Drop/DROP-DATABASE.md | 2 +- .../Data-Manipulation-Statements/Load/BROKER-LOAD.md | 4 ++-- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-blog/Annoucing.md | 2 +- .../cluster-management/elastic-expansion.md| 4 ++-- .../current/advanced/alter-table/replace-table.md | 2 +- .../import/import-scenes/external-storage-load.md | 8 .../data-operate/import/import-scenes/jdbc-load.md | 4 ++-- .../current/data-table/basic-usage.md | 18 +- .../current/data-table/hit-the-rollup.md | 2 +- .../Alter/ALTER-TABLE-REPLACE.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- .../Data-Definition-Statements/Drop/DROP-DATABASE.md | 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../best-practices/star-schema-benchmark.md| 4 ++-- .../extending-doris/udf/user-defined-function.md | 2 +- .../best-practices/star-schema-benchmark.md| 4 ++-- .../extending-doris/udf/user-defined-function.md | 2 +- 27 files changed, 55 insertions(+), 55 deletions(-) diff --git a/blog/Annoucing.md b/blog/Annoucing.md index 77bef3a0853..7ce634eb78d 100644 --- a/blog/Annoucing.md +++ b/blog/Annoucing.md @@ -52,4 +52,4 @@ Once again, we sincerely thank all contributors who participated in the construc **See How to subscribe:** -[https://doris.apache.org/zh-CN/community/subscribe-mail-list](https://doris.apache.org/zh-CN/community/subscribe-mail-list) +[https://doris.apache.org/community/subscribe-mail-list](https://doris.apache.org/community/subscribe-mail-list/) diff --git a/docs/admin-manual/cluster-management/elastic-expansion.md b/docs/admin-manual/cluster-management/elastic-expansion.md index 17cb9111553..ea821f69bbb 100644 --- a/docs/admin-manual/cluster-management/elastic-expansion.md +++ b/docs/admin-manual/cluster-management/elastic-expansion.md @@ -106,7 +106,7 @@ You can also view the BE node through the front-end page connection: ``http://fe All of the above methods require Doris's root user rights. -The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-repair-and-balance.md). +The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-repair-and-balance). ### Add BE nodes @@ -140,7 +140,7 @@ DECOMMISSION clause: > ```CANCEL ALTER SYSTEM DECOMMISSION BACKEND > "be_host:be_heartbeat_service_port";``` > The order was cancelled. When cancelled, the data on the BE will > maintain the current amount of data remaining. Follow-up Doris re-load > balancing -**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../multi-tenant.md).** +**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../multi-tenant).** ## Broker Expansion and Shrinkage diff --git a/docs/advanced/alter-table/replace-table.md b/docs/advanced/alter-table/replace-table.md index 9
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4f58e9d9f3e fix 4f58e9d9f3e is described below commit 4f58e9d9f3ec4009a57d7d6fae2a91113b535968 Author: jiafeng.zhang AuthorDate: Tue Oct 25 18:33:47 2022 +0800 fix --- docs/admin-manual/cluster-management/elastic-expansion.md | 2 +- docs/admin-manual/config/be-config.md | 2 +- docs/advanced/alter-table/schema-change.md| 2 +- docs/data-operate/import/import-scenes/external-storage-load.md | 2 +- docs/data-operate/import/import-scenes/jdbc-load.md | 4 ++-- docs/data-table/basic-usage.md| 4 ++-- .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- .../current/admin-manual/cluster-management/elastic-expansion.md | 2 +- .../current/admin-manual/config/be-config.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- 12 files changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/admin-manual/cluster-management/elastic-expansion.md b/docs/admin-manual/cluster-management/elastic-expansion.md index ea821f69bbb..e8fd4525298 100644 --- a/docs/admin-manual/cluster-management/elastic-expansion.md +++ b/docs/admin-manual/cluster-management/elastic-expansion.md @@ -106,7 +106,7 @@ You can also view the BE node through the front-end page connection: ``http://fe All of the above methods require Doris's root user rights. -The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-repair-and-balance). +The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../../maint-monitor/tablet-repair-and-balance). ### Add BE nodes diff --git a/docs/admin-manual/config/be-config.md b/docs/admin-manual/config/be-config.md index e27118a83ed..90e9d044208 100644 --- a/docs/admin-manual/config/be-config.md +++ b/docs/admin-manual/config/be-config.md @@ -450,7 +450,7 @@ Cgroups assigned to doris ### `doris_max_scan_key_num` * Type: int -* Description: Used to limit the maximum number of scan keys that a scan node can split in a query request. When a conditional query request reaches the scan node, the scan node will try to split the conditions related to the key column in the query condition into multiple scan key ranges. After that, these scan key ranges will be assigned to multiple scanner threads for data scanning. A larger value usually means that more scanner threads can be used to increase the parallelism of the s [...] +* Description: Used to limit the maximum number of scan keys that a scan node can split in a query request. When a conditional query request reaches the scan node, the scan node will try to split the conditions related to the key column in the query condition into multiple scan key ranges. After that, these scan key ranges will be assigned to multiple scanner threads for data scanning. A larger value usually means that more scanner threads can be used to increase the parallelism of the s [...] * Default value: 1024 When the concurrency cannot be improved in high concurrency scenarios, try to reduce this value and observe the impact. diff --git a/docs/advanced/alter-table/schema-change.md b/docs/advanced/alter-table/schema-change.md index 3cf43598ae7..23f9fa77084 100644 --- a/docs/advanced/alter-table/schema-change.md +++ b/docs/advanced/alter-table/schema-change.md @@ -282,5 +282,5 @@ SHOW ALTER TABLE COLUMN\G; ## More Help -For more detailed syntax and best practices used by Schema Change, see [ALTER TABLE COLUMN](../../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md ) command manual, you can also enter `HELP ALTER TABLE COLUMN` in the MySql client command line for more help information. +For more detailed syntax and best practices used by Schema Change
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new ae4125aa154 fix ae4125aa154 is described below commit ae4125aa1545ea7e94b9a104fa1e29f8d452fb77 Author: jiafeng.zhang AuthorDate: Tue Oct 25 19:27:57 2022 +0800 fix --- docs/admin-manual/cluster-management/elastic-expansion.md | 2 +- docs/admin-manual/cluster-management/upgrade.md | 2 +- docs/admin-manual/http-actions/fe/table-schema-action.md | 2 +- docs/admin-manual/maint-monitor/disk-capacity.md | 2 +- docs/advanced/alter-table/schema-change.md| 4 ++-- docs/data-operate/import/import-scenes/external-storage-load.md | 8 docs/data-operate/import/import-scenes/jdbc-load.md | 4 ++-- docs/data-table/basic-usage.md| 6 +++--- docs/ecosystem/external-table/hive-bitmap-udf.md | 4 ++-- .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Data-Manipulation-Statements/Load/STREAM-LOAD.md | 8 .../current/admin-manual/cluster-management/elastic-expansion.md | 2 +- .../current/admin-manual/cluster-management/upgrade.md| 2 +- .../current/admin-manual/http-actions/fe/table-schema-action.md | 2 +- .../data-operate/import/import-scenes/external-storage-load.md| 8 .../current/ecosystem/external-table/hive-bitmap-udf.md | 4 ++-- .../administrator-guide/operation/metadata-operation.md | 4 ++-- .../administrator-guide/alter-table/alter-table-rollup.md | 4 ++-- .../administrator-guide/operation/metadata-operation.md | 4 ++-- versioned_docs/version-0.15/extending-doris/logstash.md | 2 +- .../version-0.15/extending-doris/udf/user-defined-function.md | 2 +- 21 files changed, 39 insertions(+), 39 deletions(-) diff --git a/docs/admin-manual/cluster-management/elastic-expansion.md b/docs/admin-manual/cluster-management/elastic-expansion.md index e8fd4525298..b10b7f57c68 100644 --- a/docs/admin-manual/cluster-management/elastic-expansion.md +++ b/docs/admin-manual/cluster-management/elastic-expansion.md @@ -140,7 +140,7 @@ DECOMMISSION clause: > ```CANCEL ALTER SYSTEM DECOMMISSION BACKEND > "be_host:be_heartbeat_service_port";``` > The order was cancelled. When cancelled, the data on the BE will > maintain the current amount of data remaining. Follow-up Doris re-load > balancing -**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../multi-tenant).** +**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../../multi-tenant).** ## Broker Expansion and Shrinkage diff --git a/docs/admin-manual/cluster-management/upgrade.md b/docs/admin-manual/cluster-management/upgrade.md index a04a69a5394..99b7cbea5dd 100644 --- a/docs/admin-manual/cluster-management/upgrade.md +++ b/docs/admin-manual/cluster-management/upgrade.md @@ -113,7 +113,7 @@ Therefore, it is recommended to upgrade some nodes and observe the business oper **Illegal rollback operation may cause data loss and damage.** ## Documentation -1. [Doris metadata design document](../../../community/design/metadata-design) +1. [Doris metadata design document](/community/design/metadata-design) 2. [Metadata Operations and Maintenance](../../admin-manual/maint-monitor/metadata-operation.md) 3. [Data replica management](../../admin-manual/maint-monitor/tablet-repair-and-balance.md) 4. [Installation Deployment Document](../../install/install-deploy.md) diff --git a/docs/admin-manual/http-actions/fe/table-schema-action.md b/docs/admin-manual/http-actions/fe/table-schema-action.md index c2f7cc7..122b7ed06d9 100644 --- a/docs/admin-manual/http-actions/fe/table-schema-action.md +++ b/docs/admin-manual/http-actions/fe/table-schema-action.md @@ -97,7 +97,7 @@ None "count": 0 } ``` -Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../admin-manual/config/fe-config) +Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../../../admin-manual/config/fe-config) ## Examples diff --git a/docs/admin-manual/maint-monitor/disk-capacity.md b/docs/admin-manual/maint-monitor/disk-capacity.md index 59412f846b2..
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 401d4f87f6d fix 401d4f87f6d is described below commit 401d4f87f6d06f051bc89b57c0c26538f693108d Author: jiafeng.zhang AuthorDate: Tue Oct 25 20:11:52 2022 +0800 fix --- docs/admin-manual/http-actions/fe/table-schema-action.md| 2 +- docs/admin-manual/maint-monitor/disk-capacity.md| 2 +- docs/advanced/alter-table/schema-change.md | 2 +- .../sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md | 2 +- .../current/admin-manual/http-actions/fe/table-schema-action.md | 2 +- .../sql-reference/Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../version-0.15/best-practices/star-schema-benchmark.md| 2 +- versioned_docs/version-0.15/best-practices/star-schema-benchmark.md | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/admin-manual/http-actions/fe/table-schema-action.md b/docs/admin-manual/http-actions/fe/table-schema-action.md index 122b7ed06d9..3af0088b31f 100644 --- a/docs/admin-manual/http-actions/fe/table-schema-action.md +++ b/docs/admin-manual/http-actions/fe/table-schema-action.md @@ -97,7 +97,7 @@ None "count": 0 } ``` -Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../../../admin-manual/config/fe-config) +Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../../../../admin-manual/config/fe-config) ## Examples diff --git a/docs/admin-manual/maint-monitor/disk-capacity.md b/docs/admin-manual/maint-monitor/disk-capacity.md index 1f377faa897..e84ca8c1f9a 100644 --- a/docs/admin-manual/maint-monitor/disk-capacity.md +++ b/docs/admin-manual/maint-monitor/disk-capacity.md @@ -162,6 +162,6 @@ When the disk capacity is higher than High Watermark or even Flood Stage, many o ```rm -rf data/0/12345/``` -* Delete tablet metadata (refer to [Tablet metadata management tool](../../tablet-meta-tool)） +* Delete tablet metadata (refer to [Tablet metadata management tool](../tablet-meta-tool)） ```./lib/meta_tool --operation=delete_header --root_path=/path/to/root_path --tablet_id=12345 --schema_hash= 35278``` diff --git a/docs/advanced/alter-table/schema-change.md b/docs/advanced/alter-table/schema-change.md index bf0fa481c95..809015c4dd0 100644 --- a/docs/advanced/alter-table/schema-change.md +++ b/docs/advanced/alter-table/schema-change.md @@ -68,7 +68,7 @@ The basic process of executing a Schema Change is to generate a copy of the inde Before starting the conversion of historical data, Doris will obtain a latest transaction ID. And wait for all import transactions before this Transaction ID to complete. This Transaction ID becomes a watershed. This means that Doris guarantees that all import tasks after the watershed will generate data for both the original Index and the new Index. In this way, when the historical data conversion is completed, the data in the new Index can be guaranteed to be complete. ## Create Job -The specific syntax for creating a Schema Change can be found in the help [ALTER TABLE COLUMN](../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN) for the description of the Schema Change section . +The specific syntax for creating a Schema Change can be found in the help [ALTER TABLE COLUMN](../../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN) for the description of the Schema Change section . The creation of Schema Change is an asynchronous process. After the job is submitted successfully, the user needs to view the job progress through the `SHOW ALTER TABLE COLUMN` command. ## View Job diff --git a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md index 210c4706c6f..897be9f24da 100644 --- a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md +++ b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md @@ -432,7 +432,7 @@ curl --location-trusted -u root -H "columns: k1,k2,source_sequence,v1,v2" -H "fu 7. Strict Mode - The `strict_mode` attribute is used to set whether the import task runs in strict mode. The format affects the results of column mapping, transformati
[doris] branch master updated (9e3a61989b -> 67a6ad648e)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 9e3a61989b [refactor](es) remove BE generated dsl for es query #15751 add 67a6ad648e [typo](doc) command of manually trigger compaction incorrect (#15709) No new revisions were added by this update. Summary of changes: docs/zh-CN/docs/admin-manual/http-actions/compaction-action.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
