Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
github-actions[bot] commented on PR #26017: URL: https://github.com/apache/doris/pull/26017#issuecomment-1782400640 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Nereids): Convert topn(x, 1) to max(x) [doris]
doris-robot commented on PR #26004: URL: https://github.com/apache/doris/pull/26004#issuecomment-1782402990 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.71 seconds stream load tsv: 575 seconds loaded 74807831229 Bytes, about 124 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162266362 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
airborne12 commented on PR #26017: URL: https://github.com/apache/doris/pull/26017#issuecomment-1782403054 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](complex type) add docs for complex and nested types [doris]
AshinGau merged PR #25865: URL: https://github.com/apache/doris/pull/25865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [doc](complex type) add docs for complex and nested types (#25865)
This is an automated email from the ASF dual-hosted git repository. ashingau pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new cad3d8fb993 [doc](complex type) add docs for complex and nested types (#25865) cad3d8fb993 is described below commit cad3d8fb9935b92735ac19ebcf1bde7133005917 Author: Ashin Gau AuthorDate: Fri Oct 27 15:04:53 2023 +0800 [doc](complex type) add docs for complex and nested types (#25865) Add docs for complex and nested types. --- docs/en/docs/ecosystem/udf/java-user-defined-function.md| 2 +- docs/en/docs/lakehouse/multi-catalog/hive.md| 6 +++--- docs/zh-CN/docs/ecosystem/udf/java-user-defined-function.md | 2 +- docs/zh-CN/docs/lakehouse/multi-catalog/hive.md | 6 +++--- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/en/docs/ecosystem/udf/java-user-defined-function.md b/docs/en/docs/ecosystem/udf/java-user-defined-function.md index d1e3ee29984..ead333c0080 100644 --- a/docs/en/docs/ecosystem/udf/java-user-defined-function.md +++ b/docs/en/docs/ecosystem/udf/java-user-defined-function.md @@ -61,7 +61,7 @@ Java UDF provides users with a Java interface written in UDF to facilitate the e |```array```|```ArrayList```| |```map```|```HashMap```| -* Array/Map types can nested basic types, Eg: In Doris: ```array``` corresponds to JAVA UDF Argument Type: ```ArrayList```, Others is also. +* Array/Map types can nested other types, Eg: In Doris: ```array>``` corresponds to JAVA UDF Argument Type: ```ArrayList>```, and so on. ## Write UDF functions This section mainly introduces how to develop a Java UDF. Samples for the Java version are provided under `samples/doris-demo/java-udf-demo/` for your reference, Check it out [here](https://github.com/apache/incubator-doris/tree/master/samples/doris-demo/java-udf-demo) diff --git a/docs/en/docs/lakehouse/multi-catalog/hive.md b/docs/en/docs/lakehouse/multi-catalog/hive.md index 398468a611a..4724d0e903b 100644 --- a/docs/en/docs/lakehouse/multi-catalog/hive.md +++ b/docs/en/docs/lakehouse/multi-catalog/hive.md @@ -390,9 +390,9 @@ For Hive/Iceberge/Hudi | char| char | | | varchar| varchar| | | decimal| decimal | | -| `array` | `array`| 支持array嵌套，如 `array>` | -| `map` | `map` | 暂不支持嵌套，KeyType 和 ValueType 需要为基础类型 | -| `struct` | `struct` | 暂不支持嵌套，Type1, Type2, ... 需要为基础类型 | +| `array` | `array`| support nested type, for example `array>` | +| `map` | `map` | support nested type, for example `map>` | +| `struct` | `struct` | support nested type, for example `struct, col2: map>` | | other | unsupported | | ## Whether to truncate char or varchar columns according to the schema of the hive table diff --git a/docs/zh-CN/docs/ecosystem/udf/java-user-defined-function.md b/docs/zh-CN/docs/ecosystem/udf/java-user-defined-function.md index e7e37b56e5a..0888cdd5d09 100644 --- a/docs/zh-CN/docs/ecosystem/udf/java-user-defined-function.md +++ b/docs/zh-CN/docs/ecosystem/udf/java-user-defined-function.md @@ -59,7 +59,7 @@ Java UDF 为用户提供UDF编写的Java接口，以方便用户使用Java语言 |```array```|```ArrayList```| |```map```|```HashMap```| -* array/map类型可以嵌套基本类型，例如Doris: ```array```对应JAVA UDF Argument Type: ```ArrayList```, 其他依此类推 +* array/map类型可以嵌套其它类型，例如Doris: ```array>```对应JAVA UDF Argument Type: ```ArrayList>```, 其他依此类推 ## 编写 UDF 函数 本小节主要介绍如何开发一个 Java UDF。在 `samples/doris-demo/java-udf-demo/` 下提供了示例，可供参考，查看点击[这里](https://github.com/apache/doris/tree/master/samples/doris-demo/java-udf-demo) diff --git a/docs/zh-CN/docs/lakehouse/multi-catalog/hive.md b/docs/zh-CN/docs/lakehouse/multi-catalog/hive.md index 4ed341b7a1e..09fbe4b2178 100644 --- a/docs/zh-CN/docs/lakehouse/multi-catalog/hive.md +++ b/docs/zh-CN/docs/lakehouse/multi-catalog/hive.md @@ -369,9 +369,9 @@ CREATE CATALOG hive PROPERTIES ( | char| char | | | varchar| varchar| | | decimal| decimal | | -| `array` | `array`| 支持array嵌套，如 `array>` | -| `map` | `map` | 暂不支持嵌套，KeyType 和 ValueType 需要为基础类型 | -| `struct` | `struct` | 暂不支持嵌套，Type1, Type2, ... 需要为基础类型 | +| `array` | `array`| 支持嵌套，如 `array>` | +| `map` | `map` | 支持嵌套，如 `map>` | +| `struct` | `struct` | 支持嵌套，如 `struct, col2: map>` | | other | unsupported | | ## 是否按照 hive 表的 schema 来截断 char 或者 varchar 列 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Nereids) Support hint in `insert into` statement [doris]
doris-robot commented on PR #26008: URL: https://github.com/apache/doris/pull/26008#issuecomment-1782411475 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.24 seconds stream load tsv: 581 seconds loaded 74807831229 Bytes, about 122 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17162386119 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](meta) update killed query state [doris]
doris-robot commented on PR #25917: URL: https://github.com/apache/doris/pull/25917#issuecomment-1782412095 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.05 seconds stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.4 seconds inserted 1000 Rows, about 340K ops/s storage size: 17162504340 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
airborne12 commented on PR #26017: URL: https://github.com/apache/doris/pull/26017#issuecomment-1782414835 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] Data loss for some indicators in the audit log [doris]
Larborator closed issue #25840: [Bug] Data loss for some indicators in the audit log URL: https://github.com/apache/doris/issues/25840 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] Data loss for some indicators in the audit log [doris]
Larborator commented on issue #25840: URL: https://github.com/apache/doris/issues/25840#issuecomment-1782417079 > the problem can be solved by `set enable_pipeline_engine = false` and I am trying to fix it. I found it has been fixed in master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](window_function) window function first_value/last_value should be always nullable [doris]
doris-robot commented on PR #26014: URL: https://github.com/apache/doris/pull/26014#issuecomment-1782418250 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.9 seconds stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162190123 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [date](parser) Support DateV1 keyword [doris]
doris-robot commented on PR #25414: URL: https://github.com/apache/doris/pull/25414#issuecomment-1782418778 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.18 seconds stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17162386450 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function)return NULL rather than 'null' if path not found [doris]
github-actions[bot] commented on PR #25880: URL: https://github.com/apache/doris/pull/25880#issuecomment-1782424295 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Nereids): Convert topn(x, 1) to max(x) [doris]
jackwener commented on PR #26004: URL: https://github.com/apache/doris/pull/26004#issuecomment-1782433741 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
doris-robot commented on PR #26017: URL: https://github.com/apache/doris/pull/26017#issuecomment-1782434621 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.66 seconds stream load tsv: 574 seconds loaded 74807831229 Bytes, about 124 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17162094404 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) bound error when in diff value of table name case config [doris]
github-actions[bot] commented on PR #25957: URL: https://github.com/apache/doris/pull/25957#issuecomment-1782439713 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
doris-robot commented on PR #26017: URL: https://github.com/apache/doris/pull/26017#issuecomment-1782442378 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.34 seconds stream load tsv: 576 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162496455 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [bug] Doris2.0.2 执行固定SQL后，BE报错，甚至导致所有BE宕机。 [doris]
DA1OOO commented on issue #25644: URL: https://github.com/apache/doris/issues/25644#issuecomment-1782447741 @zhiqiang- I checkout to branch-2.0 and compile frontend by myself and meet new bad case, Tableau can’t connect to doris by mysql odbc driver 5.3. Error like "wrong data type string. expected integer" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](pipeline) simplify runtime state ctor [doris]
github-actions[bot] commented on PR #25995: URL: https://github.com/apache/doris/pull/25995#issuecomment-1782447817 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](pipeline) simplify runtime state ctor [doris]
github-actions[bot] commented on PR #25995: URL: https://github.com/apache/doris/pull/25995#issuecomment-1782447858 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function)return NULL rather than 'null' if path not found [doris]
Yulei-Yang commented on PR #25880: URL: https://github.com/apache/doris/pull/25880#issuecomment-1782451116 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Feature] doris 分词字典如何配置 [doris]
xyls82 opened a new issue, #26018: URL: https://github.com/apache/doris/issues/26018 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description 对于倒排索引自定义分词字典如何配置 ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](update) Support `update on current_timestamp` [doris]
github-actions[bot] commented on PR #25884: URL: https://github.com/apache/doris/pull/25884#issuecomment-1782454419 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](update) Support `update on current_timestamp` [doris]
github-actions[bot] commented on PR #25884: URL: https://github.com/apache/doris/pull/25884#issuecomment-1782454381 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](pipeline) simplify runtime state ctor [doris]
Gabriel39 merged PR #25995: URL: https://github.com/apache/doris/pull/25995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (cad3d8fb993 -> a01922cdc55)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from cad3d8fb993 [doc](complex type) add docs for complex and nested types (#25865) add a01922cdc55 [refactor](pipeline) simplify runtime state ctor (#25995) No new revisions were added by this update. Summary of changes: be/src/pipeline/pipeline_fragment_context.cpp | 9 ++--- .../pipeline_x/pipeline_x_fragment_context.cpp | 9 ++--- be/src/runtime/runtime_state.cpp| 21 ++--- be/src/runtime/runtime_state.h | 8 +--- 4 files changed, 27 insertions(+), 20 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
doris-robot commented on PR #26017: URL: https://github.com/apache/doris/pull/26017#issuecomment-1782455327 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.26 seconds stream load tsv: 578 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162147999 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](pipeline) use selectdb-robot to comment ckb result [doris]
hello-stephen opened a new pull request, #26019: URL: https://github.com/apache/doris/pull/26019 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipeline) use selectdb-robot to comment ckb result [doris]
hello-stephen commented on PR #26019: URL: https://github.com/apache/doris/pull/26019#issuecomment-1782455868 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipeline) use selectdb-robot to comment ckb result [doris]
github-actions[bot] commented on PR #26019: URL: https://github.com/apache/doris/pull/26019#issuecomment-1782458690 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [bug] Doris2.0.2 执行固定SQL后，BE报错，甚至导致所有BE宕机。 [doris]
DA1OOO commented on issue #25644: URL: https://github.com/apache/doris/issues/25644#issuecomment-1782459634 Are there any adjustments in the branch 2.0 in this problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Feature](Nereids) support short circuit query in Nereids [doris]
eldenmoon opened a new pull request, #26021: URL: https://github.com/apache/doris/pull/26021 Add new rule `LogicalResultSinkToShortCircuitPointQuery` similar to original `SelectStmt.checkAndSetPointQuery` ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Nereids) support short circuit query in Nereids [doris]
eldenmoon commented on PR #26021: URL: https://github.com/apache/doris/pull/26021#issuecomment-1782463570 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipeline](refactor) simplify runtime state constructor [doris]
Gabriel39 closed pull request #25506: [pipeline](refactor) simplify runtime state constructor URL: https://github.com/apache/doris/pull/25506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](s3load) Add regression testing and modify description text for s3load [doris]
TangSiyang2001 commented on PR #25947: URL: https://github.com/apache/doris/pull/25947#issuecomment-1782467587 related case seems failed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [minor](reader) remove redundant code [doris]
Gabriel39 closed pull request #25206: [minor](reader) remove redundant code URL: https://github.com/apache/doris/pull/25206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Nereids) support short circuit query in Nereids [doris]
github-actions[bot] commented on PR #26021: URL: https://github.com/apache/doris/pull/26021#issuecomment-1782470737 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Nereids) support short circuit query in Nereids [doris]
github-actions[bot] commented on PR #26021: URL: https://github.com/apache/doris/pull/26021#issuecomment-1782473722 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Nereids): Convert topn(x, 1) to max(x) [doris]
doris-robot commented on PR #26004: URL: https://github.com/apache/doris/pull/26004#issuecomment-1782474891 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.14 seconds stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162194430 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Nereids) support short circuit query in Nereids [doris]
github-actions[bot] commented on PR #26021: URL: https://github.com/apache/doris/pull/26021#issuecomment-1782473937 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-flink-connector] branch master updated: [Improve](schemaChange)schema change support rename column (#206)
This is an automated email from the ASF dual-hosted git repository.
diwu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git
The following commit(s) were added to refs/heads/master by this push:
new 78ed055 [Improve](schemaChange)schema change support rename column
(#206)
78ed055 is described below
commit 78ed055fd2e7f7657dadfb1d3f4f4dd9e9caadf7
Author: DongLiang-0 <46414265+donglian...@users.noreply.github.com>
AuthorDate: Fri Oct 27 16:02:32 2023 +0800
[Improve](schemaChange)schema change support rename column (#206)
---
.../sink/writer/JsonDebeziumSchemaSerializer.java | 20 ++-
.../flink/sink/writer/SchemaChangeHelper.java | 23 +++-
.../flink/sink/writer/SchemaChangeHelperTest.java | 65 ++
.../writer/TestJsonDebeziumSchemaSerializer.java | 25 +
4 files changed, 130 insertions(+), 3 deletions(-)
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/JsonDebeziumSchemaSerializer.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/JsonDebeziumSchemaSerializer.java
index fd3c92a..bf7b81f 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/JsonDebeziumSchemaSerializer.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/JsonDebeziumSchemaSerializer.java
@@ -82,6 +82,8 @@ public class JsonDebeziumSchemaSerializer implements
DorisRecordSerializer updateFiledSchema = new LinkedHashMap<>();
for (JsonNode column : columns) {
buildFieldSchema(updateFiledSchema, column);
}
SchemaChangeHelper.compareSchema(updateFiledSchema,
originFieldSchemaMap);
-// In order to avoid operations such as rename or change, which may
lead to the accidental deletion of the doris column.
+// In order to avoid other source table column change operations other
than add/drop/rename,
+// which may lead to the accidental deletion of the doris column.
Matcher matcher = addDropDDLPattern.matcher(ddl);
if (!matcher.find()) {
return null;
@@ -262,6 +276,10 @@ public class JsonDebeziumSchemaSerializer implements
DorisRecordSerializer
originFieldSchemaMap) {
+this.originFieldSchemaMap = originFieldSchemaMap;
+}
@VisibleForTesting
public boolean schemaChange(JsonNode recordRoot) {
boolean status = false;
diff --git
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/SchemaChangeHelper.java
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/SchemaChangeHelper.java
index dc8d83b..8e6307b 100644
---
a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/SchemaChangeHelper.java
+++
b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/SchemaChangeHelper.java
@@ -32,8 +32,9 @@ public class SchemaChangeHelper {
private static final List addFieldSchemas =
Lists.newArrayList();
// Used to determine whether the doris table supports ddl
private static final List ddlSchemas = Lists.newArrayList();
-public static final String ADD_DDL = "ALTER TABLE %s ADD COLUMN %s %s";
-public static final String DROP_DDL = "ALTER TABLE %s DROP COLUMN %s";
+private static final String ADD_DDL = "ALTER TABLE %s ADD COLUMN %s %s";
+private static final String DROP_DDL = "ALTER TABLE %s DROP COLUMN %s";
+private static final String RENAME_DDL = "ALTER TABLE %s RENAME COLUMN %s
%s";
public static void compareSchema(Map
updateFiledSchemaMap,
Map originFieldSchemaMap) {
@@ -57,6 +58,24 @@ public class SchemaChangeHelper {
}
}
+public static List generateRenameDDLSql(String table, String
oldColumnName, String newColumnName,
+Map originFieldSchemaMap) {
+ddlSchemas.clear();
+List ddlList = Lists.newArrayList();
+FieldSchema fieldSchema = null;
+for (Entry originFieldSchema :
originFieldSchemaMap.entrySet()) {
+if (originFieldSchema.getKey().equals(oldColumnName)) {
+fieldSchema = originFieldSchema.getValue();
+String renameSQL = String.format(RENAME_DDL, table,
oldColumnName, newColumnName);
+ddlList.add(renameSQL);
+ddlSchemas.add(new DDLSchema(oldColumnName, false));
+}
+}
+originFieldSchemaMap.remove(oldColumnName);
+originFieldSchemaMap.put(newColumnName, fieldSchema);
+return ddlList;
+}
+
public static List generateDDLSql(String table) {
ddlSchemas.clear();
List ddlList = Lists.newArrayList();
diff --git
a/flink-doris-connector/src/test/java/org/apache/doris/flink/sink/writer/SchemaChangeHelperTest.java
b/flink-doris-connector/src/test/java/org/apache/doris/flink/sink/writer/SchemaChangeHelperTest.java
new file mode 100644
index 000..62906df
--- /dev/
Re: [PR] [feature](backup) add property to remove snapshot before creating repo [doris]
w41ter commented on PR #25847: URL: https://github.com/apache/doris/pull/25847#issuecomment-1782475537 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improve](schemaChange)schema change support rename column [doris-flink-connector]
JNSimba merged PR #206: URL: https://github.com/apache/doris-flink-connector/pull/206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0-var updated: fix chinese sub column name
This is an automated email from the ASF dual-hosted git repository.
eldenmoon pushed a commit to branch branch-2.0-var
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0-var by this push:
new b35e1366699 fix chinese sub column name
b35e1366699 is described below
commit b35e1366699542f11d5d836e6f4bacbd7e47d1bd
Author: eldenmoon <15605149...@163.com>
AuthorDate: Sun Oct 22 21:23:00 2023 +0800
fix chinese sub column name
---
be/src/vec/columns/column_object.cpp | 11 +--
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/be/src/vec/columns/column_object.cpp
b/be/src/vec/columns/column_object.cpp
index 13dfe477303..38a93b37086 100644
--- a/be/src/vec/columns/column_object.cpp
+++ b/be/src/vec/columns/column_object.cpp
@@ -904,10 +904,10 @@ bool ColumnObject::is_finalized() const {
[](const auto& entry) { return
entry->data.is_finalized(); });
}
-static bool check_if_valid_column_name(const PathInData& path) {
-static const std::regex
COLUMN_NAME_REGEX("^[_a-zA-Z@0-9][.a-zA-Z0-9_+-/>data.check_if_sparse_column(num_rows) ||
- !check_if_valid_column_name(entry->path))) {
+if (!ignore_sparse && (entry->data.check_if_sparse_column(num_rows))) {
// TODO seperate ambiguous path
sparse_columns.add(entry->path, entry->data);
continue;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)add visitMarkJoinReference method in ExpressionDeepCopier [doris]
starocean999 commented on PR #25874: URL: https://github.com/apache/doris/pull/25874#issuecomment-1782477379 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) Fix not check if the tablet is nullptr [doris]
github-actions[bot] commented on PR #26015: URL: https://github.com/apache/doris/pull/26015#issuecomment-1782481360 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](disk balance) Prevent duplicate disk balance tasks afte… [doris]
deardeng commented on code in PR #25990:
URL: https://github.com/apache/doris/pull/25990#discussion_r1374219882
##
be/src/agent/task_worker_pool.cpp:
##
@@ -2011,8 +2011,9 @@ Status
StorageMediumMigrateTaskPool::_check_migrate_request(const TStorageMedium
*dest_store = stores[0];
}
if (tablet->data_dir()->path() == (*dest_store)->path()) {
-return Status::InternalError("tablet is already on specified path {}",
- tablet->data_dir()->path());
+LOG_WARNING("tablet is already on specified path")
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
github-actions[bot] commented on PR #26017: URL: https://github.com/apache/doris/pull/26017#issuecomment-1782493626 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](load) Fix not check if the tablet is nullptr [doris]
zhannngchen commented on PR #26015: URL: https://github.com/apache/doris/pull/26015#issuecomment-1782494320 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0-var updated (b35e1366699 -> 1fdb66b00a7)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a change to branch branch-2.0-var in repository https://gitbox.apache.org/repos/asf/doris.git discard b35e1366699 fix chinese sub column name new 1fdb66b00a7 fix chinese sub column name This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (b35e1366699) \ N -- N -- N refs/heads/branch-2.0-var (1fdb66b00a7) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: regression-test/suites/variant_p0/desc.groovy | 4 1 file changed, 4 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 01/01: fix chinese sub column name
This is an automated email from the ASF dual-hosted git repository.
eldenmoon pushed a commit to branch branch-2.0-var
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 1fdb66b00a7d30555ba540e318cc89d4d9f92b53
Author: eldenmoon <15605149...@163.com>
AuthorDate: Sun Oct 22 21:23:00 2023 +0800
fix chinese sub column name
---
be/src/vec/columns/column_object.cpp | 11 +--
regression-test/suites/variant_p0/desc.groovy | 4
2 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/be/src/vec/columns/column_object.cpp
b/be/src/vec/columns/column_object.cpp
index 13dfe477303..38a93b37086 100644
--- a/be/src/vec/columns/column_object.cpp
+++ b/be/src/vec/columns/column_object.cpp
@@ -904,10 +904,10 @@ bool ColumnObject::is_finalized() const {
[](const auto& entry) { return
entry->data.is_finalized(); });
}
-static bool check_if_valid_column_name(const PathInData& path) {
-static const std::regex
COLUMN_NAME_REGEX("^[_a-zA-Z@0-9][.a-zA-Z0-9_+-/>data.check_if_sparse_column(num_rows) ||
- !check_if_valid_column_name(entry->path))) {
+if (!ignore_sparse && (entry->data.check_if_sparse_column(num_rows))) {
// TODO seperate ambiguous path
sparse_columns.add(entry->path, entry->data);
continue;
diff --git a/regression-test/suites/variant_p0/desc.groovy
b/regression-test/suites/variant_p0/desc.groovy
index c5918c56629..bef263aabda 100644
--- a/regression-test/suites/variant_p0/desc.groovy
+++ b/regression-test/suites/variant_p0/desc.groovy
@@ -187,6 +187,10 @@ suite("regression_test_variant_desc", "variant_type_desc"){
sql """set describe_extend_variant_column = true"""
qt_sql_9_1 """desc ${table_name}"""
sql "truncate table ${table_name}"
+sql """insert into ${table_name} values (1, '{"中文": "这是中文"}')"""
+sql """insert into ${table_name} values (1, '{"英文": "This is
english"}')"""
+qt_sql_9_2 """desc ${table_name}"""
+qt_sql_9_2 """select v:中文 from ${table_name}"""
} finally {
// reset flags
set_be_config.call("ratio_of_defaults_as_sparse_column", "0.95")
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](nereids) use rowcount to compute runtime filter size if ndv is not available [doris]
englefly opened a new pull request, #26022: URL: https://github.com/apache/doris/pull/26022 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](nereids) use rowcount to compute runtime filter size if ndv is not available [doris]
englefly commented on PR #26022: URL: https://github.com/apache/doris/pull/26022#issuecomment-1782500800 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](variant) fix groupby multi variant columns leading to error [doris]
eldenmoon opened a new pull request, #26023: URL: https://github.com/apache/doris/pull/26023 ``` ERROR 1105 (HY000): errCode = 2, detailMessage = select list expression not produced by aggregation output (missing from GROUP BY clause?): CAST(`var` AS TEXT) ``` The hash code of slot ref should consider sub column names ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [cleanup](load) remove unused dynamic schema code (#25932)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 340a37116f2 [cleanup](load) remove unused dynamic schema code (#25932)
340a37116f2 is described below
commit 340a37116f268a7dd444a87b077f6ad205bfcac2
Author: Kaijie Chen
AuthorDate: Fri Oct 27 16:21:40 2023 +0800
[cleanup](load) remove unused dynamic schema code (#25932)
---
be/src/olap/rowset/beta_rowset_writer.cpp| 73
be/src/olap/rowset/beta_rowset_writer.h | 10
be/src/olap/rowset/beta_rowset_writer_v2.cpp | 6 ---
be/src/olap/rowset/beta_rowset_writer_v2.h | 4 --
4 files changed, 93 deletions(-)
diff --git a/be/src/olap/rowset/beta_rowset_writer.cpp
b/be/src/olap/rowset/beta_rowset_writer.cpp
index 7a3eb1d9bd8..c4a8dbb568a 100644
--- a/be/src/olap/rowset/beta_rowset_writer.cpp
+++ b/be/src/olap/rowset/beta_rowset_writer.cpp
@@ -748,77 +748,4 @@ Status
BetaRowsetWriter::flush_segment_writer_for_segcompaction(
return Status::OK();
}
-Status BetaRowsetWriter::_unfold_variant_column(vectorized::Block& block,
-TabletSchemaSPtr&
flush_schema) {
-if (block.rows() == 0) {
-return Status::OK();
-}
-
-// Sanitize block to match exactly from the same type of frontend meta
-vectorized::schema_util::FullBaseSchemaView schema_view;
-schema_view.table_id = _context.tablet_schema->table_id();
-vectorized::ColumnWithTypeAndName* variant_column =
-block.try_get_by_name(BeConsts::DYNAMIC_COLUMN_NAME);
-if (!variant_column) {
-return Status::OK();
-}
-auto base_column = variant_column->column;
-vectorized::ColumnObject& object_column =
-
assert_cast(base_column->assume_mutable_ref());
-if (object_column.empty()) {
-block.erase(BeConsts::DYNAMIC_COLUMN_NAME);
-return Status::OK();
-}
-object_column.finalize();
-// Has extended columns
-
RETURN_IF_ERROR(vectorized::schema_util::send_fetch_full_base_schema_view_rpc(&schema_view));
-// Dynamic Block consists of two parts, dynamic part of columns and static
part of columns
-// static dynamic
-// | - | --- |
-// The static ones are original _tablet_schame columns
-flush_schema = std::make_shared(*_context.tablet_schema);
-vectorized::Block flush_block(std::move(block));
-// The dynamic ones are auto generated and extended, append them the the
orig_block
-for (auto& entry : object_column.get_subcolumns()) {
-const std::string& column_name = entry->path.get_path();
-auto column_iter = schema_view.column_name_to_column.find(column_name);
-if (UNLIKELY(column_iter == schema_view.column_name_to_column.end())) {
-// Column maybe dropped by light weight schema change DDL
-continue;
-}
-TabletColumn column(column_iter->second);
-auto data_type =
vectorized::DataTypeFactory::instance().create_data_type(
-column, column.is_nullable());
-// Dynamic generated columns does not appear in original tablet schema
-if (_context.tablet_schema->field_index(column.name()) < 0) {
-flush_schema->append_column(column);
-flush_block.insert({data_type->create_column(), data_type,
column.name()});
-}
-}
-
-// Ensure column are all present at this schema version.Otherwise there
will be some senario:
-// Load1 -> version(10) with schema [a, b, c, d, e], d & e is new added
columns and schema version became 10
-// Load2 -> version(10) with schema [a, b, c] and has no extended columns
and fetched the schema at version 10
-// Load2 will persist meta with [a, b, c] but Load1 will persist meta
with [a, b, c, d, e]
-// So we should make sure that rowset at the same schema version alawys
contain the same size of columns.
-// so that all columns at schema_version is in either
_context.tablet_schema or schema_change_recorder
-for (const auto& [name, column] : schema_view.column_name_to_column) {
-if (_context.tablet_schema->field_index(name) == -1) {
-const auto& tcolumn = schema_view.column_name_to_column[name];
-TabletColumn new_column(tcolumn);
-_context.schema_change_recorder->add_extended_columns(column,
-
schema_view.schema_version);
-}
-}
-
-// Last schema alignment before flush to disk, due to the schema maybe
variant before this procedure
-// Eg. add columnA(INT) -> drop ColumnA -> add ColumnA(Double), then
columnA could be type of `Double`,
-// unfold will cast to Double type
-RETURN_IF_ERROR(vectorized::schema_util::unfold_object(
-flush_block.get_position_b
Re: [PR] [cleanup](load) remove unused dynamic schema code [doris]
dataroaring merged PR #25932: URL: https://github.com/apache/doris/pull/25932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Test pipelinex [doris]
Gabriel39 closed pull request #24871: Test pipelinex URL: https://github.com/apache/doris/pull/24871 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](submodule) print current submodule commit id when building [doris]
qidaye merged PR #26017: URL: https://github.com/apache/doris/pull/26017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Enhancement](submodule) print current submodule commit id when building (#26017)
This is an automated email from the ASF dual-hosted git repository.
jianliangqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 11e04f76fb8 [Enhancement](submodule) print current submodule commit id
when building (#26017)
11e04f76fb8 is described below
commit 11e04f76fb887c4bb9699f3c210821098c755e80
Author: airborne12
AuthorDate: Fri Oct 27 16:28:27 2023 +0800
[Enhancement](submodule) print current submodule commit id when building
(#26017)
---
build.sh | 7 +++
1 file changed, 7 insertions(+)
diff --git a/build.sh b/build.sh
index 9c8f415d773..225bad107c6 100755
--- a/build.sh
+++ b/build.sh
@@ -281,6 +281,13 @@ update_submodule() {
echo "Update ${submodule_name} submodule ..."
git submodule update --init --recursive "${submodule_path}"
exit_code=$?
+if [[ "${exit_code}" -eq 0 ]]; then
+cd "${submodule_path}"
+submodule_commit_id=$(git rev-parse HEAD)
+cd -
+expect_submodule_commit_id=$(git ls-tree HEAD "${submodule_path}" |
awk '{print $3}')
+echo "Current commit ID of ${submodule_name} submodule:
${submodule_commit_id}, expected is ${expect_submodule_commit_id}"
+fi
set -e
if [[ "${exit_code}" -ne 0 ]]; then
set +e
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[I] when using spark loading datas,it returns out of update load job etl status failed [doris]
mxli441 opened a new issue, #26024:
URL: https://github.com/apache/doris/issues/26024
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
doris apache-doris-2.0-beta-bin-x64-noavx2
spark spark-3.5.0-bin-hadoop3
hadoop hadoop-3.3.5
### What's Wrong?
CREATE EXTERNAL RESOURCE "spark_yarn"
PROPERTIES
(
"type" = "spark",
"spark.master" = "yarn",
"spark.submit.deployMode" = "cluster",
"spark.executor.memory" = "3g",
"spark.hadoop.yarn.resourcemanager.address" = "hdfs://hostname:8032",
"spark.hadoop.fs.defaultFS" = "hdfs://hostname:9000",
"working_dir" = "hdfs://hostname:9000/spark_load",
"broker" = "broker_name"
);
LOAD LABEL god.label2023102719
(
DATA INFILE("hdfs://hostname:9000/spark_load/customer.csv")
INTO TABLE customer
COLUMNS TERMINATED BY "|"
(c_custkey,c_name,c_address,c_city,c_nation,c_region,c_phone,c_mktsegment)
)
WITH RESOURCE 'spark_yarn'
(
"spark.executor.memory" = "2g",
"spark.shuffle.compress" = "true"
)
PROPERTIES
(
"timeout" = "3600"
);
mysql> show load order by createtime desc limit 1\G
*** 1. row ***
JobId: 179014
Label: label23
State: CANCELLED
Progress: Unknown id: 179014
Type: SPARK
EtlInfo: NULL
TaskInfo: cluster:spark_yarn; timeout(s):3600; max_filter_ratio:0.0
ErrorMsg: type:ETL_RUN_FAIL; msg:errCode = 2, detailMessage = spark
etl job failed. msg: spark app state: FAILED
CreateTime: 2023-10-27 16:26:01
EtlStartTime: 2023-10-27 16:26:17
EtlFinishTime: NULL
LoadStartTime: NULL
LoadFinishTime: 2023-10-27 16:27:56
URL: http://hostname:8088/proxy/application_1698388341294_0013/
JobDetails: {"Unfinished
backends":{},"ScannedRows":0,"TaskNumber":0,"LoadBytes":0,"All
backends":{},"FileNumber":0,"FileSize":0}
TransactionId: 159014
ErrorTablets: {}
User: root
Comment:
1 row in set (0.01 sec)
fe.warn.log
2023-10-27 16:09:52,115 WARN (Load etl checker|45)
[LoadJob.unprotectedExecuteCancel():659] LOAD_JOB=179012,
transaction_id={159013}, error_msg={Failed to execute load with error: errCode
= 2, detailMessage = spark etl job failed. msg: spark app state: FAILED}
2023-10-27 16:27:56,841 WARN (Load etl checker|45)
[LoadManager.lambda$processEtlStateJobs$10():442] update load job etl status
failed. job id: 179014
org.apache.doris.common.LoadException: errCode = 2, detailMessage = spark
etl job failed. msg: spark app state: FAILED
at
org.apache.doris.load.loadv2.SparkLoadJob.updateEtlStatus(SparkLoadJob.java:309)
~[doris-fe.jar:1.2-SNAPSHOT]
at
org.apache.doris.load.loadv2.LoadManager.lambda$processEtlStateJobs$10(LoadManager.java:436)
~[doris-fe.jar:1.2-SNAPSHOT]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
~[?:1.8.0_131]
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
~[?:1.8.0_131]
at
java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3566)
~[?:1.8.0_131]
at
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
~[?:1.8.0_131]
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
~[?:1.8.0_131]
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
~[?:1.8.0_131]
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
~[?:1.8.0_131]
at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
~[?:1.8.0_131]
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
~[?:1.8.0_131]
at
org.apache.doris.load.loadv2.LoadManager.processEtlStateJobs(LoadManager.java:434)
~[doris-fe.jar:1.2-SNAPSHOT]
at
org.apache.doris.load.loadv2.LoadEtlChecker.runAfterCatalogReady(LoadEtlChecker.java:43)
~[doris-fe.jar:1.2-SNAPSHOT]
at
org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58)
~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116)
~[doris-fe.jar:1.2-SNAPSHOT]
2023-10-27 16:27:56,842 WARN (Load etl checker|45)
[LoadJob.unprotectedExecuteCancel():659] LOAD_JOB=179014,
transaction_id={159014}, error_msg={Failed to execute load with error: errCode
= 2, detailMessage = spark etl job failed. msg: spark app state: FAILED}
### What You Expected?
this task can run smoothly,and the data can load in my table
### How to Reproduce?
_No response_
### Anything Else?
_No response_
[PR] [Fix](statistics)Fix external couldn't analyze database bug. [doris]
Jibing-Li opened a new pull request, #26025: URL: https://github.com/apache/doris/pull/26025 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](Nereids) could not run multi group_concat distinct (#25851)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 165957658a8 [fix](Nereids) could not run multi group_concat distinct
(#25851)
165957658a8 is described below
commit 165957658a8f6908944d1e431048ca0ca018e233
Author: morrySnow <101034200+morrys...@users.noreply.github.com>
AuthorDate: Fri Oct 27 16:42:00 2023 +0800
[fix](Nereids) could not run multi group_concat distinct (#25851)
could not run multi group_concat distinct with more than one parameters.
This bug is not just for group_concat, but we usually use literal as
parameters in group_concat. So group_concat brought the problem to light.
In the original logic, we think only distinct aggregate function with
zero or one parameter could run in multi distinct mode. But it is wrong.
We could process all distinct aggregate function with not more than one
input slots.
Think about sql:
```sql
SELECT
group_concat(distinct c1, ','), group_concat(distinct c2, ',')
FROM t
GROUP BY c3
```
---
.../nereids/properties/PhysicalProperties.java | 4 +++
.../nereids/rules/analysis/CheckAnalysis.java | 21 +++--
.../nereids/rules/analysis/NormalizeAggregate.java | 3 +-
.../rules/implementation/AggregateStrategies.java | 36 +-
4 files changed, 53 insertions(+), 11 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/PhysicalProperties.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/PhysicalProperties.java
index 3c89f87340b..beed2617a34 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/PhysicalProperties.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/properties/PhysicalProperties.java
@@ -72,9 +72,13 @@ public class PhysicalProperties {
this.orderSpec = orderSpec;
}
+/**
+ * create hash info from orderedShuffledColumns, ignore non slot reference
expression.
+ */
public static PhysicalProperties createHash(
Collection orderedShuffledColumns,
ShuffleType shuffleType) {
List partitionedSlots = orderedShuffledColumns.stream()
+.filter(SlotReference.class::isInstance)
.map(SlotReference.class::cast)
.map(SlotReference::getExprId)
.collect(Collectors.toList());
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/CheckAnalysis.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/CheckAnalysis.java
index fc6ddfcf79b..49de801b783 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/CheckAnalysis.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/CheckAnalysis.java
@@ -140,8 +140,25 @@ public class CheckAnalysis implements AnalysisRuleFactory {
private void checkAggregate(LogicalAggregate aggregate) {
Set aggregateFunctions =
aggregate.getAggregateFunctions();
-boolean distinctMultiColumns = aggregateFunctions.stream()
-.anyMatch(fun -> fun.isDistinct() && fun.arity() > 1);
+boolean distinctMultiColumns = false;
+for (AggregateFunction func : aggregateFunctions) {
+if (!func.isDistinct()) {
+continue;
+}
+if (func.arity() <= 1) {
+continue;
+}
+for (int i = 1; i < func.arity(); i++) {
+if (!func.child(i).getInputSlots().isEmpty()) {
+// think about group_concat(distinct col_1, ',')
+distinctMultiColumns = true;
+break;
+}
+}
+if (distinctMultiColumns) {
+break;
+}
+}
long distinctFunctionNum = aggregateFunctions.stream()
.filter(AggregateFunction::isDistinct)
.count();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeAggregate.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeAggregate.java
index c287f2dffe9..0acd366f1c2 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeAggregate.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeAggregate.java
@@ -30,6 +30,7 @@ import
org.apache.doris.nereids.trees.expressions.SlotReference;
import org.apache.doris.nereids.trees.expressions.SubqueryExpr;
import org.apache.doris.nereids.trees.expressions.WindowExpression;
import
org.apache.doris.nereids.trees.expressions.functions.agg.AggregateFunction;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
import
org.apache.doris.nereids.trees.expressions.visitor.Defa
Re: [PR] [fix](Nereids) could not run multi group_concat distinct [doris]
morrySnow merged PR #25851: URL: https://github.com/apache/doris/pull/25851 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch2.0] disable some external hive case for complex type [doris]
morningman merged PR #25994: URL: https://github.com/apache/doris/pull/25994 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [branch2.0] disable some external hive case for complex type (#25994)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 10e991c8bd0 [branch2.0] disable some external hive case for complex
type (#25994)
10e991c8bd0 is described below
commit 10e991c8bd0674d2d1cb467935b618b7904a8f2e
Author: Mingyu Chen
AuthorDate: Fri Oct 27 16:42:49 2023 +0800
[branch2.0] disable some external hive case for complex type (#25994)
branch 2.0 does not support complex type in Nereids, disable these cases
---
.../orc/test_hive_read_orc_complex_type.groovy | 292 ++---
.../test_hive_read_parquet_comlex_type.groovy | 256 +-
2 files changed, 274 insertions(+), 274 deletions(-)
diff --git
a/regression-test/suites/external_table_p0/export/hive_read/orc/test_hive_read_orc_complex_type.groovy
b/regression-test/suites/external_table_p0/export/hive_read/orc/test_hive_read_orc_complex_type.groovy
index 9edac8229a1..9dc12b968f3 100644
---
a/regression-test/suites/external_table_p0/export/hive_read/orc/test_hive_read_orc_complex_type.groovy
+++
b/regression-test/suites/external_table_p0/export/hive_read/orc/test_hive_read_orc_complex_type.groovy
@@ -108,173 +108,173 @@ suite("test_hive_read_orc_complex_type",
"external,hive,external_docker") {
}
// 1. struct NULL type
-try {
+// try {
+
+// def doris_field_define = "`s_info` STRUCT NULL"
+//
+// def hive_field_define = "`s_info` STRUCT"
+//
-def doris_field_define = "`s_info` STRUCT NULL"
-
-def hive_field_define = "`s_info` STRUCT"
-
+// // create table to export data
+// create_table(export_table_name, doris_field_define)
+
+// // insert data
+// sql """ insert into ${export_table_name} values (1, 'doris1', {1,
'sn1', 'sa1'}); """
+// sql """ insert into ${export_table_name} values (2, 'doris2',
struct(2, 'sn2', 'sa2')); """
+// sql """ insert into ${export_table_name} values (3, 'doris3',
named_struct('s_id', 3, 's_name', 'sn3', 's_address', 'sa3')); """
+// sql """ insert into ${export_table_name} values (4, 'doris4',
null); """
+// sql """ insert into ${export_table_name} values (5, 'doris5',
struct(5, null, 'sa5')); """
+// sql """ insert into ${export_table_name} values (6, 'doris6',
struct(null, null, null)); """
+// sql """ insert into ${export_table_name} values (7, null,
struct(null, null, null)); """
+// sql """ insert into ${export_table_name} values (8, null, null); """
+
+// // test base data
+// qt_select_base1 """ SELECT * FROM ${export_table_name} t ORDER BY
user_id; """
+
+// // test outfile to hdfs
+// def outfile_url = outfile_to_HDFS()
-// create table to export data
-create_table(export_table_name, doris_field_define)
+// // create hive table
+// create_hive_table(hive_table, hive_field_define)
-// insert data
-sql """ insert into ${export_table_name} values (1, 'doris1', {1,
'sn1', 'sa1'}); """
-sql """ insert into ${export_table_name} values (2, 'doris2',
struct(2, 'sn2', 'sa2')); """
-sql """ insert into ${export_table_name} values (3, 'doris3',
named_struct('s_id', 3, 's_name', 'sn3', 's_address', 'sa3')); """
-sql """ insert into ${export_table_name} values (4, 'doris4', null);
"""
-sql """ insert into ${export_table_name} values (5, 'doris5',
struct(5, null, 'sa5')); """
-sql """ insert into ${export_table_name} values (6, 'doris6',
struct(null, null, null)); """
-sql """ insert into ${export_table_name} values (7, null, struct(null,
null, null)); """
-sql """ insert into ${export_table_name} values (8, null, null); """
+// qt_select_tvf1 """ select * from HDFS(
+// "uri" = "${outfile_url}0.orc",
+// "hadoop.username" = "${hdfsUserName}",
+// "format" = "${format}");
+// """
-// test base data
-qt_select_base1 """ SELECT * FROM ${export_table_name} t ORDER BY
user_id; """
+// qt_hive_docker_01 """ SELECT * FROM ${hive_database}.${hive_table}
ORDER BY user_id;"""
+
+// } finally {
+// }
+
+// // 2. test Map
+// try {
+// def doris_field_define = "`m_info` Map NULL"
+//
+// def hive_field_define = "`m_info` Map"
+//
-// test outfile to hdfs
-def outfile_url = outfile_to_HDFS()
+// // create table to export data
+// create_table(export_table_name, doris_field_define)
-// create hive table
-create_hive_table(hive_table, hive_field_define)
+// // insert data
+// sql """ insert into ${export_ta
Re: [PR] [improvement](insert) refactor group commit insert into [doris]
mymeiyi commented on PR #25795: URL: https://github.com/apache/doris/pull/25795#issuecomment-1782532062 run feut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Nereids): Convert topn(x, 1) to max(x) [doris]
github-actions[bot] commented on PR #26004: URL: https://github.com/apache/doris/pull/26004#issuecomment-1782535027 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [DNM](Nereids) test disable fallback [doris]
morrySnow commented on PR #25344: URL: https://github.com/apache/doris/pull/25344#issuecomment-1782535303 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function)return NULL rather than 'null' if path not found [doris]
Yulei-Yang commented on PR #25880: URL: https://github.com/apache/doris/pull/25880#issuecomment-1782536742 run clickbench -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: fix upgrade for predict column delete collection type will make core (#26007)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new e81faeeca7f fix upgrade for predict column delete collection type will
make core (#26007)
e81faeeca7f is described below
commit e81faeeca7fd70235d05ce173d3147259f6af8b0
Author: amory
AuthorDate: Fri Oct 27 16:47:27 2023 +0800
fix upgrade for predict column delete collection type will make core
(#26007)
---
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 5 +++--
be/src/olap/schema.cpp | 4 +++-
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
index ce04b05d4b4..28274752b05 100644
--- a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
+++ b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
@@ -1816,8 +1816,9 @@ Status
SegmentIterator::_next_batch_internal(vectorized::Block* block) {
auto cid = _schema->column_id(i);
auto column_desc = _schema->column(cid);
if (_is_pred_column[cid]) {
-_current_return_columns[cid] =
-Schema::get_predicate_column_ptr(*column_desc,
_opts.io_ctx.reader_type);
+RETURN_IF_CATCH_EXCEPTION(_current_return_columns[cid] =
+
Schema::get_predicate_column_ptr(
+ *column_desc,
_opts.io_ctx.reader_type));
_current_return_columns[cid]->set_rowset_segment_id(
{_segment->rowset_id(), _segment->id()});
_current_return_columns[cid]->reserve(_opts.block_row_max);
diff --git a/be/src/olap/schema.cpp b/be/src/olap/schema.cpp
index a3297b4c2fa..d6c737de03c 100644
--- a/be/src/olap/schema.cpp
+++ b/be/src/olap/schema.cpp
@@ -200,7 +200,9 @@ vectorized::IColumn::MutablePtr
Schema::get_predicate_column_ptr(const Field& fi
ptr =
doris::vectorized::PredicateColumnType::create();
break;
default:
-LOG(FATAL) << "Unexpected type when choosing predicate column, type="
<< int(field.type());
+throw Exception(ErrorCode::SCHEMA_SCHEMA_FIELD_INVALID,
+fmt::format("Unexpected type when choosing predicate
column, type={}",
+int(field.type(;
}
if (field.is_nullable()) {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](upgrade)fix upgrade for predict column delete collection type will make core [doris]
yiguolei merged PR #26007: URL: https://github.com/apache/doris/pull/26007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Outfile) Export `DateTimev2` type of doris to ORC's `TimeStamp` type [doris]
BePPPower commented on PR #25470: URL: https://github.com/apache/doris/pull/25470#issuecomment-1782549260 run pipelinex_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipeline](branch-2.0) pr to branch-2.0 also require to run regressio… [doris]
github-actions[bot] commented on PR #25942: URL: https://github.com/apache/doris/pull/25942#issuecomment-1782552185 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](meta) fix follower sync image timeout after checkpoint. [doris]
yiguolei merged PR #26003: URL: https://github.com/apache/doris/pull/26003 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](meta) fix follower sync image timeout after checkpoint. (#26003)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 2464a22ac53 [fix](meta) fix follower sync image timeout after
checkpoint. (#26003)
2464a22ac53 is described below
commit 2464a22ac53b31303837dcf9e8cdd1497d2ecc63
Author: luozenglin
AuthorDate: Fri Oct 27 16:58:42 2023 +0800
[fix](meta) fix follower sync image timeout after checkpoint. (#26003)
The image file of our cluster reaches 2.3G. After the checkpoint, Followers
synchronize the image timeout, resulting in the continuous increase of the bdb
directory.
related pr: #25768
---
fe/fe-common/src/main/java/org/apache/doris/common/Config.java | 10 +-
.../main/java/org/apache/doris/httpv2/meta/MetaService.java| 4 +---
2 files changed, 6 insertions(+), 8 deletions(-)
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index c9b75cb84f5..c4e7c1b3d4a 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -2260,11 +2260,11 @@ public class Config extends ConfigBase {
public static boolean ignore_unknown_metadata_module = false;
@ConfField(mutable = true, masterOnly = true, description = {
-
"FE扩容时，从主节点同步元数据的timeout时间，根据${meta_dir}/image文件夹下面的image文件大小和节点间的网络环境调整，"
-+ "单位为秒，默认值300",
-"The timeout for new FE Follower/Observer synchronizing metadata from
the FE Master, "
-+ "adjust by the size of image file in the ${meta_dir}/image and
the network environment between nodes. "
-+ "The default values is 300s."
+
"从主节点同步image文件的超时时间，用户可根据${meta_dir}/image文件夹下面的image文件大小和节点间的网络环境调整，"
++ "单位为秒，默认值300",
+"The timeout for FE Follower/Observer synchronizing an image file
from the FE Master, can be adjusted by "
++ "the user on the size of image file in the
${meta_dir}/image and the network environment between "
++ "nodes. The default values is 300."
})
public static int sync_image_timeout_second = 300;
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
b/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
index ce71581f5c2..65a3ac8d2e5 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
@@ -49,8 +49,6 @@ import javax.servlet.http.HttpServletResponse;
public class MetaService extends RestBaseController {
private static final Logger LOG = LogManager.getLogger(MetaService.class);
-private static final int TIMEOUT_SECOND = 10;
-
private static final String VERSION = "version";
private static final String HOST = "host";
private static final String PORT = "port";
@@ -162,7 +160,7 @@ public class MetaService extends RestBaseController {
String filename = Storage.IMAGE + "." + versionStr;
File dir = new File(Env.getCurrentEnv().getImageDir());
try {
-MetaHelper.getRemoteFile(url, TIMEOUT_SECOND * 1000,
MetaHelper.getFile(filename, dir));
+MetaHelper.getRemoteFile(url, Config.sync_image_timeout_second *
1000, MetaHelper.getFile(filename, dir));
MetaHelper.complete(filename, dir);
} catch (FileNotFoundException e) {
return ResponseEntityBuilder.notFound("file not found.");
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Nereids): Convert topn(x, 1) to max(x) [doris]
github-actions[bot] commented on PR #26004: URL: https://github.com/apache/doris/pull/26004#issuecomment-1782562901 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipeline) use selectdb-robot to comment ckb result [doris]
github-actions[bot] commented on PR #26019: URL: https://github.com/apache/doris/pull/26019#issuecomment-1782565428 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [regression-test](variant) add github events p0 case and fix and modify case [doris]
eldenmoon opened a new pull request, #26026: URL: https://github.com/apache/doris/pull/26026 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [date](parser) Support DateV1 keyword [doris]
github-actions[bot] commented on PR #25414: URL: https://github.com/apache/doris/pull/25414#issuecomment-1782585840 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression-test](variant) add github events p0 case and fix and modify case [doris]
github-actions[bot] commented on PR #26026: URL: https://github.com/apache/doris/pull/26026#issuecomment-1782587304 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](disk balance) Prevent duplicate disk balance tasks afte… [doris]
deardeng commented on code in PR #25990: URL: https://github.com/apache/doris/pull/25990#discussion_r1374307517 ## fe/fe-core/src/main/java/org/apache/doris/clone/TabletScheduler.java: ## @@ -1642,6 +1656,7 @@ public boolean finishStorageMediaMigrationTask(StorageMediaMigrationTask migrati // if we have a success task, then stat must be refreshed before schedule a new task updateDiskBalanceLastSuccTime(tabletCtx.getSrcBackendId(), tabletCtx.getSrcPathHash()); updateDiskBalanceLastSuccTime(tabletCtx.getDestBackendId(), tabletCtx.getDestPathHash()); +updateDestPathHash(tabletCtx.getDestBackendId(), tabletCtx.getDestPathHash(), tabletCtx.getStorageMedium()); Review Comment: fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Nereids) Support hint in `insert into` statement [doris]
bobhan1 commented on PR #26008: URL: https://github.com/apache/doris/pull/26008#issuecomment-1782594491 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Nereids) Support hint in `insert into` statement [doris]
bobhan1 commented on PR #26008: URL: https://github.com/apache/doris/pull/26008#issuecomment-1782594461 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function)return NULL rather than 'null' if path not found [doris]
Yulei-Yang commented on PR #25880: URL: https://github.com/apache/doris/pull/25880#issuecomment-1782599262 run clickbench -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](outfile) Fix parquet writer [doris]
BePPPower opened a new pull request, #26027: URL: https://github.com/apache/doris/pull/26027 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](disk balance) Prevent duplicate disk balance tasks afte… [doris]
github-actions[bot] commented on PR #25990: URL: https://github.com/apache/doris/pull/25990#issuecomment-1782603621 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](bitmap index) fix is_like_predicate for bitmap index evaluate [doris]
airborne12 commented on PR #25961: URL: https://github.com/apache/doris/pull/25961#issuecomment-1782611073 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](update) Support `update on current_timestamp` [doris]
zhannngchen commented on code in PR #25884:
URL: https://github.com/apache/doris/pull/25884#discussion_r1374328664
##
regression-test/suites/nereids_p0/insert_into_table/update_on_current_timestamp.groovy:
##
@@ -0,0 +1,174 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("nereids_update_on_current_timestamp") {
+sql 'set experimental_enable_nereids_planner=true'
+sql 'set enable_fallback_to_original_planner=false'
+sql 'set enable_nereids_dml=true'
+sql "sync;"
+
+
+def t1 = "nereids_update_on_current_timestamp1"
+sql """ DROP TABLE IF EXISTS ${t1};"""
+sql """ CREATE TABLE ${t1} (
+`id` int(11) NOT NULL COMMENT "用户 ID",
+`name` varchar(65533) DEFAULT "unknown" COMMENT "用户姓名",
+`score` int(11) NOT NULL COMMENT "用户得分",
+`test` int(11) NULL COMMENT "null test",
+`dft` int(11) DEFAULT "4321",
+`update_time` datetime default current_timestamp on update
current_timestamp,
+`update_time2` datetime(6) default current_timestamp(5) on
update current_timestamp(5))
+UNIQUE KEY(`id`) DISTRIBUTED BY HASH(`id`) BUCKETS 1
+PROPERTIES(
+"replication_num" = "1",
+"enable_unique_key_merge_on_write" = "true"
+);"""
+
+def res = sql "show create table ${t1};"
+assertTrue(res.toString().containsIgnoreCase("`update_time` datetime NULL
DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP"))
+assertTrue(res.toString().containsIgnoreCase("`update_time2` datetime(6)
NULL DEFAULT CURRENT_TIMESTAMP(5) ON UPDATE CURRENT_TIMESTAMP(5)"))
+
+// set enable_unique_key_partial_update=false, it's a row update
+sql "set enable_unique_key_partial_update=false;"
+sql "sync;"
+sql """ insert into ${t1}(id,name,score,test,dft) values
+(2, "doris2", 2000, 223, 1),
+(1, "doris", 1000, 123, 1);"""
+qt_sql "select id,name,score,test,dft from ${t1} order by id;"
+qt_sql "select count(distinct update_time) from ${t1} where update_time >
'2023-10-01 00:00:00';"
+qt_sql "select count(distinct update_time2) from ${t1} where update_time2
> '2023-10-01 00:00:00';"
+
+
+// set enable_unique_key_partial_update=true, it's a partial update
+// don't specify the `update_time` column
+// it will be automatically updated to current_timestamp()
+sql "set enable_unique_key_partial_update=true;"
+sql "set enable_insert_strict=false;"
+sql "sync;"
+sql """ insert into ${t1}(id, score) values
Review Comment:
also add a case for `update` statement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](outfile) Fix parquet writer [doris]
github-actions[bot] commented on PR #26027: URL: https://github.com/apache/doris/pull/26027#issuecomment-1782616267 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[refactor](pipeline) simplify runtime state ctor (#25995)" [doris]
Gabriel39 commented on PR #26029: URL: https://github.com/apache/doris/pull/26029#issuecomment-1782618215 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Revert "[refactor](pipeline) simplify runtime state ctor (#25995)" [doris]
Gabriel39 opened a new pull request, #26029: URL: https://github.com/apache/doris/pull/26029 This reverts commit a01922cdc55e2b3a63d9a9aafb38ac5ed64c6dd3. ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](bitmap index) fix is_like_predicate for bitmap index evaluate [doris]
github-actions[bot] commented on PR #25961: URL: https://github.com/apache/doris/pull/25961#issuecomment-1782622259 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[refactor](pipeline) simplify runtime state ctor" [doris]
zclllyybb commented on PR #26030: URL: https://github.com/apache/doris/pull/26030#issuecomment-1782623267 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch dev_join updated (b0bbea05d6c -> 9b11421bd12)
This is an automated email from the ASF dual-hosted git repository. panxiaolei pushed a change to branch dev_join in repository https://gitbox.apache.org/repos/asf/doris.git from b0bbea05d6c update add 9b11421bd12 fix No new revisions were added by this update. Summary of changes: be/src/vec/common/hash_table/hash_map.h| 13 ++- be/src/vec/exec/join/process_hash_table_probe.h| 2 +- .../vec/exec/join/process_hash_table_probe_impl.h | 99 ++ 3 files changed, 31 insertions(+), 83 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[refactor](pipeline) simplify runtime state ctor (#25995)" [doris]
github-actions[bot] commented on PR #26029: URL: https://github.com/apache/doris/pull/26029#issuecomment-1782631071 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](statistics)Fix external couldn't analyze database bug. [doris]
Jibing-Li commented on PR #26025: URL: https://github.com/apache/doris/pull/26025#issuecomment-178265 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[refactor](pipeline) simplify runtime state ctor" [doris]
github-actions[bot] commented on PR #26030: URL: https://github.com/apache/doris/pull/26030#issuecomment-1782634508 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[refactor](pipeline) simplify runtime state ctor" [doris]
zclllyybb closed pull request #26030: Revert "[refactor](pipeline) simplify runtime state ctor" URL: https://github.com/apache/doris/pull/26030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Improve](thread)Add a unified thread name setting method [doris]
CalvinKirs opened a new pull request, #26031: URL: https://github.com/apache/doris/pull/26031 ## Proposed changes We have some asynchronous threads that don't have names set, which makes troubleshooting very difficult Issue Number: close #xxx ## Further comments  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[refactor](pipeline) simplify runtime state ctor (#25995)" [doris]
github-actions[bot] commented on PR #26029: URL: https://github.com/apache/doris/pull/26029#issuecomment-1782645026 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
