Re: [PR] [fix](Nereids) fix can't choosing best plan for tpcds [doris]
github-actions[bot] commented on PR #25511: URL: https://github.com/apache/doris/pull/25511#issuecomment-1765794107 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch dev_join updated (2b2afa8d78c -> cf626619449)
This is an automated email from the ASF dual-hosted git repository. panxiaolei pushed a change to branch dev_join in repository https://gitbox.apache.org/repos/asf/doris.git from 2b2afa8d78c merge block to single block on join/set node add cf626619449 update No new revisions were added by this update. Summary of changes: be/src/pipeline/exec/hashjoin_build_sink.cpp | 12 +-- be/src/pipeline/pipeline_x/dependency.h| 2 +- be/src/vec/common/hash_table/hash_map.h| 21 ++-- be/src/vec/common/hash_table/hash_map_context.h| 6 +- .../vec/common/hash_table/hash_table_set_build.h | 3 +- be/src/vec/exec/join/process_hash_table_probe.h| 2 +- .../vec/exec/join/process_hash_table_probe_impl.h | 114 +++-- be/src/vec/exec/join/vhash_join_node.cpp | 12 +-- be/src/vec/exec/join/vhash_join_node.h | 63 ++-- be/src/vec/exec/vset_operation_node.cpp| 7 +- 10 files changed, 43 insertions(+), 199 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Correct the alignment process when the table has sequence column and add cases [doris]
github-actions[bot] commented on PR #25346: URL: https://github.com/apache/doris/pull/25346#issuecomment-1765795200 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip function ipv4stringtonum(ordefault, ornull), inet_aton [doris]
github-actions[bot] commented on PR #25510: URL: https://github.com/apache/doris/pull/25510#issuecomment-1765795549 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](hive-udf)(doc) minimize hive-udf and add some docs. [doris]
wm1581066 commented on PR #24786: URL: https://github.com/apache/doris/pull/24786#issuecomment-1765795703 ZJCF -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [enhancement](backup) Add build version in backup snapshot info [doris]
xy720 opened a new pull request, #25513: URL: https://github.com/apache/doris/pull/25513 ## Proposed changes The snapshot info uploaded to S3 obj system by backup job is a json file, which is useful for us to get information about this snapshot. If we add version info to the JSON file, we can know what version of the cluster used to manufacture this snapshot was, in order to prevent restoring the newer version of the snapshot to the older version of the cluster. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Correct the alignment process when the table has sequence column and add cases [doris]
github-actions[bot] commented on PR #25346: URL: https://github.com/apache/doris/pull/25346#issuecomment-1765795994 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](case) Fix test_analyze case [doris]
zhangstar333 merged PR #25476: URL: https://github.com/apache/doris/pull/25476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](case) Fix test_analyze case (#25476)
This is an automated email from the ASF dual-hosted git repository.
zhangstar333 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 8c5af5a088a [fix](case) Fix test_analyze case (#25476)
8c5af5a088a is described below
commit 8c5af5a088aeb45762611e4406210516b34f017f
Author: AKIRA <33112463+kikyou1...@users.noreply.github.com>
AuthorDate: Tue Oct 17 15:06:01 2023 +0800
[fix](case) Fix test_analyze case (#25476)
It has following problems before this PR
use count(*) to check if all column analyzed
return directly when fe count > 1
Co-authored-by: AKIHA
---
.../suites/statistics/analyze_stats.groovy | 38 +++---
1 file changed, 19 insertions(+), 19 deletions(-)
diff --git a/regression-test/suites/statistics/analyze_stats.groovy
b/regression-test/suites/statistics/analyze_stats.groovy
index 7bf0bef89fd..5f6735c7fb2 100644
--- a/regression-test/suites/statistics/analyze_stats.groovy
+++ b/regression-test/suites/statistics/analyze_stats.groovy
@@ -18,7 +18,9 @@ import java.util.stream.Collectors
// under the License.
suite("test_analyze") {
-String db = "regression_test_statistics"
+
+String db = "test_analyze"
+
String tbl = "analyzetestlimited_duplicate_all"
sql """
@@ -39,6 +41,7 @@ suite("test_analyze") {
DROP TABLE IF EXISTS `${tbl}`
"""
+
sql """
CREATE TABLE IF NOT EXISTS `${tbl}` (
`analyzetestlimitedk3` int(11) null comment "",
@@ -86,12 +89,6 @@ suite("test_analyze") {
'ps7qwcZjBjkGfcXYMw5HQMwnElzoHqinwk8vhQCbVoGBgfotc4oSkpD3tP34h4h0tTogDMwFu60iJm1bofUzyUQofTeRwZk8','4692206687866847780')
"""
-def frontends = sql """
-SHOW FRONTENDS;
-"""
-if (frontends.size > 1) {
-return;
-}
sql """
ANALYZE DATABASE ${db}
"""
@@ -102,7 +99,7 @@ suite("test_analyze") {
sql """
SET enable_nereids_planner=true;
-
+
"""
sql """
SET enable_fallback_to_original_planner=false;
@@ -114,7 +111,7 @@ suite("test_analyze") {
Thread.sleep(1000 * 60)
sql """
-SELECT COUNT(*) FROM ${tbl};
+SELECT * FROM ${tbl};
"""
sql """
@@ -125,7 +122,7 @@ suite("test_analyze") {
try {
sql """
-SELECT * FROM ${tbl};
+SELECT * FROM ${tbl};
"""
} catch (Exception e) {
exception = e
@@ -140,7 +137,7 @@ suite("test_analyze") {
"""
sql """
-SELECT COUNT(*) FROM ${tbl};
+SELECT * FROM ${tbl};
"""
sql """
@@ -149,7 +146,7 @@ suite("test_analyze") {
try {
sql """
-SELECT COUNT(*) FROM ${tbl};
+SELECT * FROM ${tbl};
"""
} catch (Exception e) {
exception = e
@@ -240,8 +237,8 @@ suite("test_analyze") {
SHOW COLUMN CACHED STATS analyze_partitioned_tbl_test(col1)
"""
-def expected_result = { r ->
-for (int i = 0; i < r.size; i++) {
+def expected_result = { r->
+for(int i = 0; i < r.size; i++) {
if ((int) Double.parseDouble(r[i][1]) == 6) {
return true
} else {
@@ -867,7 +864,7 @@ PARTITION `p598` VALUES IN (598),
PARTITION `p599` VALUES IN (599)
)
-DISTRIBUTED BY HASH(`id`) BUCKETS 16
+DISTRIBUTED BY HASH(`id`) BUCKETS 1
PROPERTIES
(
"replication_num" = "1"
@@ -876,7 +873,10 @@ PARTITION `p599` VALUES IN (599)
"""
sql """
-INSERT INTO test_600_partition_table_analyze VALUES
(0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13),(14),(15),(16),(17),(18),(19),(20),(21),(22),(23),(24),(25),(26),(27),(28),(29),(30),(31),(32),(33),(34),(35),(36),(37),(38),(39),(40),(41),(42),(43),(44),(45),(46),(47),(48),(49),(50),(51),(52),(53),(54),(55),(56),(57),(58),(59),(60),(61),(62),(63),(64),(65),(66),(67),(68),(69),(70),(71),(72),(73),(74),(75),(76),(77),(78),(79),(80),(81),(82),(83),(84),(85),(86),(87),(8
[...]
+INSERT INTO test_600_partition_table_analyze VALUES
(0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10),(11),(12),(13),(14),(15),(16),(17),(18),(19),(20),(21),(22),(23),(24),(25),(26),(27),(28),(29),(30),(31),(32),(33),(34),(35),(36),(37),(38),(39),(40),(41),(42),(43),(44),(45),(46),(47),(48),(49),(50),(51),(52),(53),(54),(55),(56),(57),(58),(59),(60),(61),(62),(63),(64),(65),(66),(67),(68),(69),(70),(71),(72),(73),(74),(75),(76),(77),(78),(79),(80),(81),(82),(83),(84),(85),(86),(87),(8
[...]
+"""
+sql """
+INSERT INTO test_600_partition_table_analyze VALUES
(301),(302),(303),(304),(305),(306),(307),(308),(309),(310),(311),(312),(313),(314),(315),(316),(317),(318),(319),(320),(321),(322),(323),(324),(325),(326),(327),(328),(329),(330),(331),(332),(333),(334),(335),(336),(337),(338),(339),(340),(341),(342),(343),(344),(345),(346),(347),(
Re: [PR] [fix](profile) add projection time in total time [doris]
github-actions[bot] commented on code in PR #25482:
URL: https://github.com/apache/doris/pull/25482#discussion_r1361602965
##
be/src/util/runtime_profile.cpp:
##
@@ -628,6 +628,15 @@ void RuntimeProfile::to_thrift(TRuntimeProfileTree* tree) {
to_thrift(&tree->nodes);
}
+void RuntimeProfile::sub_projection() {
Review Comment:
warning: method 'sub_projection' can be made static
[readability-convert-member-functions-to-static]
be/src/util/runtime_profile.h:346:
```diff
- void sub_projection();
+ static void sub_projection();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](backup) Add build version in backup snapshot info [doris]
xy720 commented on PR #25513: URL: https://github.com/apache/doris/pull/25513#issuecomment-1765797177 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](fe) add function 'BitmapAgg' in nereids [doris]
github-actions[bot] commented on PR #25508: URL: https://github.com/apache/doris/pull/25508#issuecomment-1765801257 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](fe) add function 'BitmapAgg' in nereids [doris]
github-actions[bot] commented on PR #25508: URL: https://github.com/apache/doris/pull/25508#issuecomment-1765801331 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) fix handle oracle date format [doris]
github-actions[bot] commented on PR #25487: URL: https://github.com/apache/doris/pull/25487#issuecomment-1765801422 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) fix handle oracle date format [doris]
github-actions[bot] commented on PR #25487: URL: https://github.com/apache/doris/pull/25487#issuecomment-1765801500 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) fix handle oracle date format [doris]
github-actions[bot] commented on PR #25487: URL: https://github.com/apache/doris/pull/25487#issuecomment-1765801502 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jdbc catalog) fix handle oracle date format [doris]
zy-kkk merged PR #25487: URL: https://github.com/apache/doris/pull/25487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](jdbc catalog) fix handle oracle date format (#25487)
This is an automated email from the ASF dual-hosted git repository.
zykkk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 652d6c57c03 [fix](jdbc catalog) fix handle oracle date format (#25487)
652d6c57c03 is described below
commit 652d6c57c03e25a0f957a5c08534ff6949c93fe7
Author: zy-kkk
AuthorDate: Tue Oct 17 02:10:28 2023 -0500
[fix](jdbc catalog) fix handle oracle date format (#25487)
---
.../main/java/org/apache/doris/planner/external/jdbc/JdbcScanNode.java | 3 +--
.../data/external_table_p0/jdbc/test_oracle_jdbc_catalog.out | 3 +++
.../suites/external_table_p0/jdbc/test_oracle_jdbc_catalog.groovy | 1 +
3 files changed, 5 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/jdbc/JdbcScanNode.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/jdbc/JdbcScanNode.java
index 614b71484ce..3295284da92 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/jdbc/JdbcScanNode.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/jdbc/JdbcScanNode.java
@@ -32,7 +32,6 @@ import org.apache.doris.analysis.TupleDescriptor;
import org.apache.doris.catalog.Column;
import org.apache.doris.catalog.Env;
import org.apache.doris.catalog.JdbcTable;
-import org.apache.doris.catalog.Type;
import org.apache.doris.catalog.external.JdbcExternalTable;
import org.apache.doris.common.AnalysisException;
import org.apache.doris.common.Config;
@@ -364,7 +363,7 @@ public class JdbcScanNode extends ExternalScanNode {
private static String handleOracleDateFormat(Expr expr) {
if (expr.isConstant()
-&& (expr.getType().equals(Type.DATETIME) ||
expr.getType().equals(Type.DATETIMEV2))) {
+&& (expr.getType().isDatetime() ||
expr.getType().isDatetimeV2())) {
return "to_date('" + expr.getStringValue() + "', '-mm-dd
hh24:mi:ss')";
}
return expr.toMySql();
diff --git
a/regression-test/data/external_table_p0/jdbc/test_oracle_jdbc_catalog.out
b/regression-test/data/external_table_p0/jdbc/test_oracle_jdbc_catalog.out
index ad2572ddb35..1be0feb1a90 100644
--- a/regression-test/data/external_table_p0/jdbc/test_oracle_jdbc_catalog.out
+++ b/regression-test/data/external_table_p0/jdbc/test_oracle_jdbc_catalog.out
@@ -99,6 +99,9 @@
1 2022-01-21T05:23:01 \N \N
2 2022-11-12T20:32:56 \N \N
+-- !date7 --
+3 \N 2019-11-12T20:33:57.999 \N \N \N \N \N
+
-- !test_insert1 --
doris1 18
diff --git
a/regression-test/suites/external_table_p0/jdbc/test_oracle_jdbc_catalog.groovy
b/regression-test/suites/external_table_p0/jdbc/test_oracle_jdbc_catalog.groovy
index 79e4367793e..77d3015239c 100644
---
a/regression-test/suites/external_table_p0/jdbc/test_oracle_jdbc_catalog.groovy
+++
b/regression-test/suites/external_table_p0/jdbc/test_oracle_jdbc_catalog.groovy
@@ -80,6 +80,7 @@ suite("test_oracle_jdbc_catalog",
"p0,external,oracle,external_docker,external_d
order_qt_date4 """ select * from TEST_DATE where (T1 > '2022-01-21
00:00:00' and T1 < '2022-01-22 00:00:00') or (T1 > '2022-01-20 00:00:00' and T1
< '2022-01-23 00:00:00'); """
order_qt_date5 """ select * from TEST_DATE where T1 < '2022-01-22
00:00:00' or T1 = '2022-01-21 05:23:01'; """
order_qt_date6 """ select * from TEST_DATE where (T1 < '2022-01-22
00:00:00' or T1 > '2022-01-20 00:00:00') and (T1 < '2022-01-23 00:00:00' or T1
> '2022-01-19 00:00:00'); """
+order_qt_date7 """select * from TEST_TIMESTAMP where T2 <
str_to_date('2020-12-21 12:34:56', '%Y-%m-%d %H:%i:%s');"""
// The result of TEST_RAW will change
// So instead of qt, we're using sql here.
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](pipelineX) Simplify set operation [doris]
Gabriel39 merged PR #25502: URL: https://github.com/apache/doris/pull/25502 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [refactor](pipelineX) Simplify set operation (#25502)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 31a5e072e76 [refactor](pipelineX) Simplify set operation (#25502)
31a5e072e76 is described below
commit 31a5e072e76eb6f5956c1fedc135cc6e95d982d8
Author: Gabriel
AuthorDate: Tue Oct 17 15:11:46 2023 +0800
[refactor](pipelineX) Simplify set operation (#25502)
---
be/src/pipeline/exec/set_probe_sink_operator.cpp | 6 +--
be/src/pipeline/exec/set_probe_sink_operator.h | 5 +-
be/src/pipeline/exec/set_sink_operator.cpp | 6 +--
be/src/pipeline/exec/set_sink_operator.h | 6 ++-
.../vec/common/hash_table/hash_table_set_build.h | 55 +++---
.../vec/common/hash_table/hash_table_set_probe.h | 52 +++-
be/src/vec/exec/vset_operation_node.cpp| 2 +-
be/src/vec/exec/vset_operation_node.h | 3 ++
8 files changed, 30 insertions(+), 105 deletions(-)
diff --git a/be/src/pipeline/exec/set_probe_sink_operator.cpp
b/be/src/pipeline/exec/set_probe_sink_operator.cpp
index 81c30d45d1e..aea9aff0e75 100644
--- a/be/src/pipeline/exec/set_probe_sink_operator.cpp
+++ b/be/src/pipeline/exec/set_probe_sink_operator.cpp
@@ -131,9 +131,9 @@ Status
SetProbeSinkOperatorX::sink(RuntimeState* state, vectorized
[&](auto&& arg) -> Status {
using HashTableCtxType = std::decay_t;
if constexpr (!std::is_same_v) {
-vectorized::HashTableProbeX
-process_hashtable_ctx(local_state, probe_rows);
-return
process_hashtable_ctx.mark_data_in_hashtable(local_state, arg);
+vectorized::HashTableProbe
+process_hashtable_ctx(&local_state,
probe_rows);
+return
process_hashtable_ctx.mark_data_in_hashtable(arg);
} else {
LOG(FATAL) << "FATAL: uninited hash table";
}
diff --git a/be/src/pipeline/exec/set_probe_sink_operator.h
b/be/src/pipeline/exec/set_probe_sink_operator.h
index 2c2f1ce1c60..45176fd0099 100644
--- a/be/src/pipeline/exec/set_probe_sink_operator.h
+++ b/be/src/pipeline/exec/set_probe_sink_operator.h
@@ -31,7 +31,7 @@ class RuntimeState;
namespace vectorized {
class Block;
template
-struct HashTableProbeX;
+struct HashTableProbe;
} // namespace vectorized
namespace pipeline {
@@ -81,11 +81,12 @@ public:
: Base(parent, state) {}
Status init(RuntimeState* state, LocalSinkStateInfo& info) override;
+int64_t* valid_element_in_hash_tbl() { return
&_shared_state->valid_element_in_hash_tbl; }
private:
friend class SetProbeSinkOperatorX;
template
-friend struct vectorized::HashTableProbeX;
+friend struct vectorized::HashTableProbe;
//record insert column id during probe
std::vector _probe_column_inserted_id;
diff --git a/be/src/pipeline/exec/set_sink_operator.cpp
b/be/src/pipeline/exec/set_sink_operator.cpp
index 604729a4700..6725deffa14 100644
--- a/be/src/pipeline/exec/set_sink_operator.cpp
+++ b/be/src/pipeline/exec/set_sink_operator.cpp
@@ -116,9 +116,9 @@ Status SetSinkOperatorX::_process_build_block(
[&](auto&& arg) {
using HashTableCtxType = std::decay_t;
if constexpr (!std::is_same_v) {
-vectorized::HashTableBuildX
-hash_table_build_process(rows, raw_ptrs, offset,
state);
-static_cast(hash_table_build_process(local_state,
arg));
+vectorized::HashTableBuild
+hash_table_build_process(&local_state, rows,
raw_ptrs, offset, state);
+static_cast(hash_table_build_process(arg,
local_state._arena));
} else {
LOG(FATAL) << "FATAL: uninited hash table";
}
diff --git a/be/src/pipeline/exec/set_sink_operator.h
b/be/src/pipeline/exec/set_sink_operator.h
index 945ec06891c..5383b1b3a55 100644
--- a/be/src/pipeline/exec/set_sink_operator.h
+++ b/be/src/pipeline/exec/set_sink_operator.h
@@ -29,7 +29,7 @@ class ExecNode;
namespace vectorized {
template
-struct HashTableBuildX;
+struct HashTableBuild;
}
namespace pipeline {
@@ -74,10 +74,12 @@ public:
Status init(RuntimeState* state, LocalSinkStateInfo& info) override;
+int64_t* mem_used() { return &_shared_state->mem_used; };
+
private:
friend class SetSinkOperatorX;
template
-friend struct vectorized::HashTableBuildX;
+friend struct vectorized::HashTableBuild;
RuntimeProfile::Counter* _build_timer; // time to build hash table
vectorized::MutableBlock _mutable_block;
diff --git a/be/src/vec/common/hash_table/hash_table_set_buil
Re: [PR] [refactor](delete) refactor FE DeleteHandler related logic [doris]
TangSiyang2001 commented on PR #25497: URL: https://github.com/apache/doris/pull/25497#issuecomment-1765803806 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](delete) refactor FE DeleteHandler related logic [doris]
TangSiyang2001 commented on PR #25497: URL: https://github.com/apache/doris/pull/25497#issuecomment-1765805325 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](catalog)compatible with paimon 0.5 [doris]
zddr commented on PR #24985: URL: https://github.com/apache/doris/pull/24985#issuecomment-1765806649 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](outfile)support underscore prefix when select outfile [doris]
BePPPower commented on code in PR #25395:
URL: https://github.com/apache/doris/pull/25395#discussion_r1361610056
##
fe/fe-core/src/main/java/org/apache/doris/common/FeNameFormat.java:
##
@@ -138,6 +140,12 @@ public static void checkCommonName(String type, String
name) throws AnalysisExce
}
}
+public static void checkSuccessFileName(String type, String name) throws
AnalysisException {
Review Comment:
```suggestion
public static void checkOutfileSuccessFileName(String type, String name)
throws AnalysisException {
```
May it is better to call this method as `checkOutfileSuccessFileName`
##
fe/fe-core/src/test/java/org/apache/doris/common/FeNameFormatTest.java:
##
@@ -55,6 +55,9 @@ public void testCheckColumnName() {
ExceptionChecker.expectThrows(AnalysisException.class, () ->
FeNameFormat.checkCommonName("fakeType", "_commonName"));
ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkCommonName("fakeType", "common-Name"));
ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkCommonName("fakeType", "commonName-"));
+
+// check success file name prefix
+ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkColumnName("_success"));
Review Comment:
```suggestion
// check success file name prefix
ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkSuccessFileName("_success"));
```
Is this wrong here?It should be `checkSuccessFileName`？
##
fe/fe-core/src/test/java/org/apache/doris/common/FeNameFormatTest.java:
##
@@ -55,6 +55,9 @@ public void testCheckColumnName() {
ExceptionChecker.expectThrows(AnalysisException.class, () ->
FeNameFormat.checkCommonName("fakeType", "_commonName"));
ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkCommonName("fakeType", "common-Name"));
ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkCommonName("fakeType", "commonName-"));
+
+// check success file name prefix
+ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkColumnName("_success"));
Review Comment:
```suggestion
// check success file name prefix
ExceptionChecker.expectThrowsNoException(() ->
FeNameFormat.checkSuccessFileName("_success"));
```
Is this wrong here?It should be `checkSuccessFileName`？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](docker)add docker-iceberg init tables [doris]
wuwenchi commented on PR #25424: URL: https://github.com/apache/doris/pull/25424#issuecomment-1765805830 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip function ipv4stringtonum(ordefault, ornull), inet_aton [doris]
github-actions[bot] commented on PR #25510: URL: https://github.com/apache/doris/pull/25510#issuecomment-1765807896 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) fix can't choosing best plan for tpcds [doris]
XieJiann commented on PR #25511: URL: https://github.com/apache/doris/pull/25511#issuecomment-1765810074 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](outfile)support underscore prefix when select outfile [doris]
BePPPower commented on code in PR #25395:
URL: https://github.com/apache/doris/pull/25395#discussion_r1361610056
##
fe/fe-core/src/main/java/org/apache/doris/common/FeNameFormat.java:
##
@@ -138,6 +140,12 @@ public static void checkCommonName(String type, String
name) throws AnalysisExce
}
}
+public static void checkSuccessFileName(String type, String name) throws
AnalysisException {
Review Comment:
```suggestion
public static void checkOutfileSuccessFileName(String type, String name)
throws AnalysisException {
```
May it is better to call this method as `checkOutfileSuccessFileName`, and
add some comment for this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] test [doris]
github-actions[bot] commented on code in PR #25493:
URL: https://github.com/apache/doris/pull/25493#discussion_r1361616411
##
be/src/vec/common/hash_table/hash_map.h:
##
@@ -193,10 +195,82 @@ class HashMapTable : public HashTable {
bool has_null_key_data() const { return false; }
};
+template ,
+ typename Grower = HashTableGrower<>, typename Allocator =
HashTableAllocator>
+class JoinHashMapTable : public HashMapTable {
+public:
+using Self = JoinHashMapTable;
+using Base = HashMapTable;
+
+using key_type = Key;
+using value_type = typename Cell::value_type;
+using mapped_type = typename Cell::Mapped;
+
+using LookupResult = typename Base::LookupResult;
+
+using HashMapTable::HashMapTable;
+
+void expanse_for_add_elem(size_t num_elem) {
+bucket_size = calc_bucket_size(num_elem + 1);
+first.resize(bucket_size, 0);
+}
+
+static uint32_t calc_bucket_size(size_t num_elem) {
+size_t expect_bucket_size = static_cast(num_elem) + (num_elem
- 1) / 7;
+return phmap::priv::NormalizeCapacity(expect_bucket_size) + 1;
+}
+
+void build(const Key* __restrict keys, const size_t* __restrict
hash_values, int num_elem) {
+build_keys = keys;
+next.resize(num_elem);
+for (size_t i = 1; i < num_elem; i++) {
+uint32_t bucket_num = hash_values[i] & (bucket_size - 1);
+next[i] = first[bucket_num];
+first[bucket_num] = i;
+}
+}
+
+uint32_t bucket_num(const Key& key) const { return Base::hash(key) &
(bucket_size - 1); }
+
+LookupResult ALWAYS_INLINE find(Key key) { return Base::find(key); }
+LookupResult ALWAYS_INLINE find(Key x, size_t hash_value) { return
Base::find(x, hash_value); }
+
+auto find_batch(const Key* __restrict keys, const size_t* __restrict
hash_values, int probe_idx,
+int probe_rows, std::vector& probe_idxs,
+std::vector& build_idxs) {
+auto matched_cnt = 0;
+while (probe_idx < probe_rows && matched_cnt < 4096) {
Review Comment:
warning: 4096 is a magic number; consider replacing it with a named constant
[readability-magic-numbers]
```cpp
while (probe_idx < probe_rows && matched_cnt < 4096) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](profile) profilev2 distinguish Sink and Operator in pipelineX [doris]
github-actions[bot] commented on PR #25491: URL: https://github.com/apache/doris/pull/25491#issuecomment-1765813392 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](profile) profilev2 distinguish Sink and Operator in pipelineX [doris]
github-actions[bot] commented on PR #25491: URL: https://github.com/apache/doris/pull/25491#issuecomment-1765813312 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Feature] Doris support Arrow Flight SQL [doris]
xinyiZzz opened a new issue, #25514:
URL: https://github.com/apache/doris/issues/25514
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Description
# 1. Motivation
I am trying to implement Arrow Flight SQL Server in Apache Doris to support
ADBC. The goal is to speed up the return of large query results from Doris to
Python. Now Python uses mysql-client to load large batches of data from Doris
very slowly. The scenario is data science/ Machine learning.
# 2. Implementation method
Doris is a column-stored database. It is very expensive to convert the
in Doris into into
and then convert it back to . The time consuming is mainly in the process of transposing the data
rows and columns twice and serialization.
Borrowing a Dremio diagram, it is similar to accelerating python reading
Doris based on Arrow Flight SQL. Change the above process to convert the
in Doris into and
then to to speed up data conversion and
transmission.
# 3. Introduction to Arrow Flight SQL
1. Arrow, a column storage data format
2. Arrow Flight, an RPC framework for transmitting Arrow data format
3. Arrow Flight SQL, a client-server protocol developed by the Apache Arrow
community, is used for clients to use the Arrow format to interact with
databases that implement the Flight RPC framework. Compared with JDBC/ODBC, it
avoids serialization and twice data rows and columns. Transpose. And supports
parallel reading and writing.
4. ADBC, a client API that supports different languages to access the
database. The database needs to implement Arrow Flight Sql server, similar to
JDBC/ODBC.
# 4. Purpose
1. Speed up Python Pandas to load all data field types of Doris.
2. Supports SQL query, Set Session Veriable, and DDL, but does not support
data import yet.
3. In the future, it may be possible to replace the interface for reading
Doris by other systems such as Spark.
# 5. Design
## 5.1 Outline design
[picture]
1) ADBC Client sends a query request to Doris FE and completes the
authentication on the first request.
2) FE parses the query plan and sends the Fragment to be executed to BE.
3) After BE completes the prepare and open of the Fragment, it returns the
Schema of the query result in Arrow format to FE, starts executing the query,
and puts the query results into a queue.
4) FE sends the QueryID, the Schema of the query result, and the BE address
(Endpoints) where the query result is located back to the ADBC Client.
5) ADBC Client requests BE to pull the query results of the specified
QueryID.
6) BE returns the query results in Arrow format in the queue to the ADBC
Client, and the ADBC Client completes after verifying the Schema of the results.
## 5.2 Detailed design
Arrow version: 13.0.0
Take the ADBC Low-Level API execution process as an example:
### 5.2.1 ADBC Client
**1.1** db =
adbc_driver_flightsql.connect(uri="grpc://ip:port?user=&password=")
Create a Database connector that can maintain multiple shared Connections at
the same time. Parameters: Arrow Flight Server IP, port, username, password
**1.2** conn = adbc_driver_manager.AdbcConnection(db)
Creating a Database link will trigger authentication and obtain
FlightSqlInfo.
1. Auth
The authentication operation will be triggered when Arrow Flight Server is
requested for the first time.
The return value is a Bearer Token. Each subsequent request to Arrow Flight
Server will bring this Token.
2. getFlightInfoSqlInfo
Request Arrow Flight Server to return SQL Info, including the SQL syntax
supported by the database, etc. The return value is the schema and endpoint of
SQL Info. SQL Info is also data in arrow format, and the endpoint is still the
current doris fe flight server.
All data interacted in arrow flight are arrows. Usually before obtaining an
arrow data, the first request will obtain its endpoint and schema and
encapsulate it in a FlightInfo. Then the endpoint will be requested again to
obtain the arrow data and verify it. schema
3. getStreamSqlInfo
Request the endpoint to obtain SQL Info. The result is wrapped in
ArrowArrayStream and associated with a ServerStreamListener.
**1.3** stmt = adbc_driver_manager.AdbcStatement(conn)
It is used to maintain the status of the query. It can be a one-time query
or a prepare statement, which can be used repeatedly, but the previous query
results will be invalid.
**1.4** stmt.set_sql_query("select * from tpch.hdf5 limit 10;")
**1.5** stream, _ = stmt.execute_query()
Executing Query returns a RecordBatchReader, wrapped in a RecordBatchStream.
1. getFlightInfoStatement
Returns the Endpoints and Schema where the query resul
Re: [I] [Bug] be The static parameter is modified by the post call to return ok, which does not actually take effect [doris]
xinge-1 commented on issue #25390: URL: https://github.com/apache/doris/issues/25390#issuecomment-1765823894 > The latest version has fixed this problem最新版本已解决此问题 get it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](backup) Add build version in backup snapshot info [doris]
github-actions[bot] commented on PR #25513: URL: https://github.com/apache/doris/pull/25513#issuecomment-1765828552 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](backup) Add build version in backup snapshot info [doris]
github-actions[bot] commented on PR #25513: URL: https://github.com/apache/doris/pull/25513#issuecomment-1765828619 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nereids)in physical plan, print join class simple name not full name [doris]
englefly opened a new pull request, #25515: URL: https://github.com/apache/doris/pull/25515 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)in physical plan, print join class simple name not full name [doris]
englefly commented on PR #25515: URL: https://github.com/apache/doris/pull/25515#issuecomment-1765830722 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] Update newsletter.data.ts [doris-website]
KassieZ opened a new pull request, #327: URL: https://github.com/apache/doris-website/pull/327 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](delete) refactor FE DeleteHandler related logic [doris]
TangSiyang2001 commented on PR #25497: URL: https://github.com/apache/doris/pull/25497#issuecomment-1765841141 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Correct the alignment process when the table has sequence column and add cases [doris]
doris-robot commented on PR #25346: URL: https://github.com/apache/doris/pull/25346#issuecomment-1765843330 TeamCity be ut coverage result: Function Coverage: 37.02% (8284/22376) Line Coverage: 29.07% (66450/228614) Region Coverage: 27.75% (34473/124230) Branch Coverage: 24.38% (17521/71878) Coverage Report: http://coverage.selectdb-in.cc/coverage/de1190a37b147592c2a5a260370549840943ce0d_de1190a37b147592c2a5a260370549840943ce0d/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [test](case) add test case to improve code coverage [doris]
zhangstar333 opened a new pull request, #25516: URL: https://github.com/apache/doris/pull/25516 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](case) add test case to improve code coverage [doris]
zhangstar333 commented on PR #25516: URL: https://github.com/apache/doris/pull/25516#issuecomment-1765844279 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipeline](refactor) simplify runtime state constructor [doris]
Gabriel39 commented on PR #25506: URL: https://github.com/apache/doris/pull/25506#issuecomment-1765845321 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [typo](doc)Modify the default value of Stale rowset cleanup policy [doris]
caoliang-web opened a new pull request, #25517: URL: https://github.com/apache/doris/pull/25517 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip function ipv4stringtonum(ordefault, ornull), inet_aton [doris]
github-actions[bot] commented on PR #25510: URL: https://github.com/apache/doris/pull/25510#issuecomment-1765850889 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](Nereids): fix regression test test_cast_as_time in Branch 2.0 [doris]
jackwener opened a new pull request, #25518: URL: https://github.com/apache/doris/pull/25518 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](merge-on-write) Fix some bugs about sequence column [doris]
bobhan1 commented on PR #24915: URL: https://github.com/apache/doris/pull/24915#issuecomment-1765854732 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [only test] [doris]
Mryange opened a new pull request, #25519: URL: https://github.com/apache/doris/pull/25519 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
Mryange commented on PR #25519: URL: https://github.com/apache/doris/pull/25519#issuecomment-1765856515 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Outfile) Export `DateTimev2` type of doris to ORC's `TimeStamp` type [doris]
BePPPower commented on PR #25470: URL: https://github.com/apache/doris/pull/25470#issuecomment-1765856741 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ipv6)fix can not resolve host and port [doris]
morningman merged PR #25254: URL: https://github.com/apache/doris/pull/25254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (31a5e072e76 -> f38f5f50eb1)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 31a5e072e76 [refactor](pipelineX) Simplify set operation (#25502) add f38f5f50eb1 [fix](ipv6)fix can not resolve host and port (#25254) No new revisions were added by this update. Summary of changes: .../java/org/apache/doris/alter/SystemHandler.java | 6 ++-- .../analysis/AdminCancelRebalanceDiskStmt.java | 4 ++- .../apache/doris/analysis/AdminCleanTrashStmt.java | 5 +++- .../doris/analysis/AdminRebalanceDiskStmt.java | 5 +++- .../apache/doris/analysis/ShowTrashDiskStmt.java | 3 +- .../java/org/apache/doris/catalog/BrokerMgr.java | 7 +++-- .../main/java/org/apache/doris/catalog/Env.java| 6 ++-- .../java/org/apache/doris/catalog/FsBroker.java| 3 +- .../org/apache/doris/common/proc/TrashProcDir.java | 3 +- .../org/apache/doris/common/util/BrokerReader.java | 3 +- .../org/apache/doris/common/util/BrokerUtil.java | 6 ++-- .../doris/deploy/impl/AmbariDeployManager.java | 4 ++- .../doris/fs/operations/BrokerFileOperations.java | 4 ++- .../doris/httpv2/controller/BaseController.java| 7 +++-- .../doris/httpv2/rest/RestBaseController.java | 4 ++- .../doris/httpv2/rest/TableQueryPlanAction.java| 4 ++- .../doris/httpv2/rest/manager/ClusterAction.java | 10 +-- .../doris/httpv2/rest/manager/NodeAction.java | 35 ++ .../httpv2/rest/manager/QueryProfileAction.java| 4 ++- .../apache/doris/journal/bdbje/BDBJEJournal.java | 8 +++-- .../java/org/apache/doris/ldap/LdapClient.java | 7 +++-- .../org/apache/doris/master/ReportHandler.java | 4 ++- .../java/org/apache/doris/metric/MetricRepo.java | 7 +++-- .../java/org/apache/doris/mysql/MysqlChannel.java | 4 ++- .../java/org/apache/doris/qe/ShowExecutor.java | 6 ++-- .../org/apache/doris/system/SystemInfoService.java | 13 .../ExternalFileTableValuedFunction.java | 4 ++- 27 files changed, 121 insertions(+), 55 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](case) add test case to improve code coverage [doris]
github-actions[bot] commented on PR #25516: URL: https://github.com/apache/doris/pull/25516#issuecomment-1765858847 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) add projection time in total time [doris]
doris-robot commented on PR #25482: URL: https://github.com/apache/doris/pull/25482#issuecomment-1765863851 TeamCity be ut coverage result: Function Coverage: 37.02% (8284/22378) Line Coverage: 29.06% (66442/228615) Region Coverage: 27.74% (34464/124228) Branch Coverage: 24.37% (17514/71870) Coverage Report: http://coverage.selectdb-in.cc/coverage/27f270cbf3908395468fc9a6bbab3d06902c9b08_27f270cbf3908395468fc9a6bbab3d06902c9b08/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](dynamic) Fix error reporting when dynamic partition properties contain incorrect attributes [doris]
Zhiyu-h commented on PR #25373: URL: https://github.com/apache/doris/pull/25373#issuecomment-1765867069 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
github-actions[bot] commented on PR #25519: URL: https://github.com/apache/doris/pull/25519#issuecomment-1765870135 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [typo](doc)Modify the default value of Stale rowset cleanup policy [doris]
caoliang-web commented on PR #25517: URL: https://github.com/apache/doris/pull/25517#issuecomment-1765870412 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](iceberg)fix read partitioned iceberg without partition path [doris]
wuwenchi commented on PR #25503: URL: https://github.com/apache/doris/pull/25503#issuecomment-1765871544 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Enhancement] Add IPv6 address and or CIDR address process function like ClickHouse [doris]
superdiaodiao commented on issue #21489: URL: https://github.com/apache/doris/issues/21489#issuecomment-1765874372 After this pr(https://github.com/apache/doris/pull/25510) merged, I can have a try. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Enhancement] Add IP address process function for SQL like ClickHouse [doris]
superdiaodiao commented on issue #20584: URL: https://github.com/apache/doris/issues/20584#issuecomment-1765875049 I added another ip function and the PR is https://github.com/apache/doris/pull/25510. @TangSiyang2001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) fix can't choosing best plan for tpcds [doris]
doris-robot commented on PR #25511: URL: https://github.com/apache/doris/pull/25511#issuecomment-1765880126 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.43 seconds stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.7 seconds inserted 1000 Rows, about 348K ops/s storage size: 17162319328 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Enhancement] Add IPv6 address and or CIDR address process function like ClickHouse [doris]
superdiaodiao commented on issue #21489: URL: https://github.com/apache/doris/issues/21489#issuecomment-1765880749 The func named IPv4StringToNum is in the PR(https://github.com/apache/doris/pull/25510). After this PR, I can try the rest. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](backup) Add build version in backup snapshot info [doris]
doris-robot commented on PR #25513: URL: https://github.com/apache/doris/pull/25513#issuecomment-1765881125 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.7 seconds stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.6 seconds inserted 1000 Rows, about 337K ops/s storage size: 17161998727 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](case) add test case to improve code coverage [doris]
doris-robot commented on PR #25516: URL: https://github.com/apache/doris/pull/25516#issuecomment-1765879511 TeamCity be ut coverage result: Function Coverage: 37.03% (8284/22372) Line Coverage: 29.07% (66441/228542) Region Coverage: 27.75% (34465/124193) Branch Coverage: 24.37% (17514/71856) Coverage Report: http://coverage.selectdb-in.cc/coverage/dfbdea3e0a7b0b266ce144987b24c029d2b31c7b_dfbdea3e0a7b0b266ce144987b24c029d2b31c7b/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Correct the alignment process when the table has sequence column and add cases [doris]
doris-robot commented on PR #25346: URL: https://github.com/apache/doris/pull/25346#issuecomment-1765880674 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.1 seconds stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17162117359 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](delete) refactor FE DeleteHandler related logic [doris]
doris-robot commented on PR #25497: URL: https://github.com/apache/doris/pull/25497#issuecomment-1765880732 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.98 seconds stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162262476 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](nereids) Support aggregate functions without from clause [doris]
doris-robot commented on PR #25500: URL: https://github.com/apache/doris/pull/25500#issuecomment-1765880915 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.47 seconds stream load tsv: 558 seconds loaded 74807831229 Bytes, about 127 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.3 seconds inserted 1000 Rows, about 341K ops/s storage size: 17162039074 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)in physical plan, print join class simple name not full name [doris]
doris-robot commented on PR #25515: URL: https://github.com/apache/doris/pull/25515#issuecomment-1765882301 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.47 seconds stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.2 seconds inserted 1000 Rows, about 342K ops/s storage size: 17162197805 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](config) turnoff fuzzy for `enable_simdjson_reader` [doris]
eldenmoon opened a new pull request, #25521: URL: https://github.com/apache/doris/pull/25521 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [docker][fix] add kafka file for p0 regression [doris]
XuJianxu opened a new pull request, #25522: URL: https://github.com/apache/doris/pull/25522 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Outfile) Export `DateTimev2` type of doris to ORC's `TimeStamp` type [doris]
doris-robot commented on PR #25470: URL: https://github.com/apache/doris/pull/25470#issuecomment-1765895335 TeamCity be ut coverage result: Function Coverage: 37.02% (8282/22374) Line Coverage: 29.06% (66423/228536) Region Coverage: 27.75% (34463/124188) Branch Coverage: 24.37% (17507/71850) Coverage Report: http://coverage.selectdb-in.cc/coverage/b1a40e07550bcecbf47c817c7a6cbb045c0f8b4e_b1a40e07550bcecbf47c817c7a6cbb045c0f8b4e/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docker][fix] add kafka file for p0 regression [doris]
github-actions[bot] commented on PR #25522: URL: https://github.com/apache/doris/pull/25522#issuecomment-1765895274 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids): fix regression test test_cast_as_time in Branch 2.0 [doris]
jackwener commented on PR #25518: URL: https://github.com/apache/doris/pull/25518#issuecomment-1765904279 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] Query cannot be cancelled [doris]
zhiqiang- opened a new issue, #25523: URL: https://github.com/apache/doris/issues/25523 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? * ### What You Expected? * ### How to Reproduce? in test_array_map_function, test for non-pipeline will hang. ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](config) turnoff fuzzy for `enable_simdjson_reader` [doris]
github-actions[bot] commented on PR #25521: URL: https://github.com/apache/doris/pull/25521#issuecomment-1765904641 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](query execution) Fix result sink fragment can't be cancelled in non-pipeline [doris]
zhiqiang- opened a new pull request, #25524: URL: https://github.com/apache/doris/pull/25524 close https://github.com/apache/doris/issues/25523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipeline](refactor) simplify runtime state constructor [doris]
doris-robot commented on PR #25506: URL: https://github.com/apache/doris/pull/25506#issuecomment-1765906570 TeamCity be ut coverage result: Function Coverage: 37.01% (8282/22375) Line Coverage: 29.06% (66423/228555) Region Coverage: 27.75% (34462/124198) Branch Coverage: 24.36% (17508/71858) Coverage Report: http://coverage.selectdb-in.cc/coverage/2c2538fdd50cfa774b1f66724103318a64720871_2c2538fdd50cfa774b1f66724103318a64720871/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](Nereids) non-slot filter should not be push through aggregate [doris]
morrySnow opened a new pull request, #25525: URL: https://github.com/apache/doris/pull/25525 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) non-slot filter should not be push through aggregate [doris]
morrySnow commented on PR #25525: URL: https://github.com/apache/doris/pull/25525#issuecomment-1765908423 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](config) turnoff fuzzy for `enable_simdjson_reader` [doris]
github-actions[bot] commented on PR #25521: URL: https://github.com/apache/doris/pull/25521#issuecomment-1765906480 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](query execution) Fix result sink fragment can't be cancelled in non-pipeline [doris]
zhiqiang- commented on PR #25524: URL: https://github.com/apache/doris/pull/25524#issuecomment-1765909973 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Performance](sink) SIMD the tablet sink valied data function [doris]
kaijchen commented on code in PR #25480:
URL: https://github.com/apache/doris/pull/25480#discussion_r1361713194
##
be/src/vec/sink/vtablet_block_convertor.h:
##
@@ -66,15 +66,15 @@ class OlapTableBlockConvertor {
Status _validate_column(RuntimeState* state, const TypeDescriptor& type,
bool is_nullable,
vectorized::ColumnPtr column, size_t slot_index,
bool* stop_processing,
-fmt::memory_buffer& error_prefix,
+fmt::memory_buffer& error_prefix, const uint32_t
row_count,
vectorized::IColumn::Permutation* rows = nullptr);
// make input data valid for OLAP table
// return number of invalid/filtered rows.
// invalid row number is set in Bitmap
// set stop_processing if we want to stop the whole process now.
-Status _validate_data(RuntimeState* state, vectorized::Block* block,
int64_t& filtered_rows,
- bool* stop_processing);
+Status _validate_data(RuntimeState* state, vectorized::Block* block, const
uint32_t rows,
Review Comment:
Remove this `const` in header file.
##
be/src/vec/sink/vtablet_block_convertor.h:
##
@@ -66,15 +66,15 @@ class OlapTableBlockConvertor {
Status _validate_column(RuntimeState* state, const TypeDescriptor& type,
bool is_nullable,
vectorized::ColumnPtr column, size_t slot_index,
bool* stop_processing,
-fmt::memory_buffer& error_prefix,
+fmt::memory_buffer& error_prefix, const uint32_t
row_count,
Review Comment:
Remove this `const` in header file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Performance](sink) SIMD the tablet sink valied data function [doris]
HappenLee merged PR #25480: URL: https://github.com/apache/doris/pull/25480 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](delete) refactor FE DeleteHandler related logic [doris]
doris-robot commented on PR #25497: URL: https://github.com/apache/doris/pull/25497#issuecomment-1765912012 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.19 seconds stream load tsv: 553 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162052088 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](case) add test case to improve code coverage [doris]
doris-robot commented on PR #25516: URL: https://github.com/apache/doris/pull/25516#issuecomment-1765912179 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.19 seconds stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 28.8 seconds inserted 1000 Rows, about 347K ops/s storage size: 17162142548 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Performance](sink) SIMD the tablet sink valied data function (#25480)
This is an automated email from the ASF dual-hosted git repository.
lihaopeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 06ff59bc03d [Performance](sink) SIMD the tablet sink valied data
function (#25480)
06ff59bc03d is described below
commit 06ff59bc03db294508bb0d169292d4a3302ea974
Author: HappenLee
AuthorDate: Tue Oct 17 16:21:08 2023 +0800
[Performance](sink) SIMD the tablet sink valied data function (#25480)
---
be/src/vec/sink/vtablet_block_convertor.cpp | 173 ++--
be/src/vec/sink/vtablet_block_convertor.h | 12 +-
be/src/vec/sink/vtablet_sink_v2.cpp | 5 +-
be/src/vec/sink/writer/vtablet_writer.cpp | 13 +--
4 files changed, 104 insertions(+), 99 deletions(-)
diff --git a/be/src/vec/sink/vtablet_block_convertor.cpp
b/be/src/vec/sink/vtablet_block_convertor.cpp
index da9e3fc743c..436eb3639de 100644
--- a/be/src/vec/sink/vtablet_block_convertor.cpp
+++ b/be/src/vec/sink/vtablet_block_convertor.cpp
@@ -72,12 +72,12 @@ Status OlapTableBlockConvertor::validate_and_convert_block(
RETURN_IF_ERROR(_fill_auto_inc_cols(block.get(), rows));
}
-int64_t filtered_rows = 0;
+int filtered_rows = 0;
{
SCOPED_RAW_TIMER(&_validate_data_ns);
-_filter_bitmap.Reset(block->rows());
+_filter_map.resize(rows, 0);
bool stop_processing = false;
-RETURN_IF_ERROR(_validate_data(state, block.get(), filtered_rows,
&stop_processing));
+RETURN_IF_ERROR(_validate_data(state, block.get(), rows,
filtered_rows, &stop_processing));
_num_filtered_rows += filtered_rows;
has_filtered_rows = filtered_rows > 0;
if (stop_processing) {
@@ -161,11 +161,12 @@ Status
OlapTableBlockConvertor::_validate_column(RuntimeState* state, const Type
bool is_nullable,
vectorized::ColumnPtr column,
size_t slot_index, bool*
stop_processing,
fmt::memory_buffer&
error_prefix,
+ const uint32_t row_count,
vectorized::IColumn::Permutation* rows) {
-DCHECK((rows == nullptr) || (rows->size() == column->size()));
+DCHECK((rows == nullptr) || (rows->size() == row_count));
fmt::memory_buffer error_msg;
auto set_invalid_and_append_error_msg = [&](int row) {
-_filter_bitmap.Set(row, true);
+_filter_map[row] = true;
auto ret = state->append_error_msg_to_file([]() -> std::string {
return ""; },
[&error_prefix,
&error_msg]() -> std::string {
return
fmt::to_string(error_prefix) +
@@ -180,10 +181,9 @@ Status
OlapTableBlockConvertor::_validate_column(RuntimeState* state, const Type
auto& real_column_ptr = column_ptr == nullptr ? column :
(column_ptr->get_nested_column_ptr());
auto null_map = column_ptr == nullptr ? nullptr :
column_ptr->get_null_map_data().data();
auto need_to_validate = [&null_map, this](size_t j, size_t row) {
-return !_filter_bitmap.Get(row) && (null_map == nullptr || null_map[j]
== 0);
+return !_filter_map[row] && (null_map == nullptr || null_map[j] == 0);
};
-ssize_t last_invalid_row = -1;
switch (type.type) {
case TYPE_CHAR:
case TYPE_VARCHAR:
@@ -196,43 +196,49 @@ Status
OlapTableBlockConvertor::_validate_column(RuntimeState* state, const Type
if (type.len > 0) {
limit = std::min(config::string_type_length_soft_limit_bytes,
type.len);
}
-for (size_t j = 0; j < column->size(); ++j) {
-auto row = rows ? (*rows)[j] : j;
-if (row == last_invalid_row) {
-continue;
-}
-if (need_to_validate(j, row)) {
-auto str_val = column_string->get_data_at(j);
-bool invalid = str_val.size > limit;
-if (invalid) {
-last_invalid_row = row;
-if (str_val.size > type.len) {
-fmt::format_to(error_msg, "{}",
- "the length of input is too long than
schema. ");
-fmt::format_to(error_msg, "first 32 bytes of input
str: [{}] ",
- str_val.to_prefix(32));
-fmt::format_to(error_msg, "schema length: {}; ",
type.len);
-fmt::format_to(error_msg, "actual length: {}; ",
str_val.size);
-} else if (str_val.size > limit) {
-fmt::format_to(error_msg, "{}",
- "the length of input string is too long
than vec sch
Re: [PR] [Fix](merge-on-write) Fix some bugs about sequence column [doris]
doris-robot commented on PR #24915: URL: https://github.com/apache/doris/pull/24915#issuecomment-1765916823 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.24 seconds stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162130203 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Outfile) Export `DateTimev2` type of doris to ORC's `TimeStamp` type [doris]
doris-robot commented on PR #25470: URL: https://github.com/apache/doris/pull/25470#issuecomment-1765918777 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.73 seconds stream load tsv: 576 seconds loaded 74807831229 Bytes, about 123 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 28.9 seconds inserted 1000 Rows, about 346K ops/s storage size: 17162239245 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](config) turnoff fuzzy for `enable_simdjson_reader` [doris]
eldenmoon commented on PR #25521: URL: https://github.com/apache/doris/pull/25521#issuecomment-1765915424 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docker][fix] add kafka file for p0 regression [doris]
github-actions[bot] commented on PR #25522: URL: https://github.com/apache/doris/pull/25522#issuecomment-1765916241 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) add projection time in total time [doris]
doris-robot commented on PR #25482: URL: https://github.com/apache/doris/pull/25482#issuecomment-1765916344 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.59 seconds stream load tsv: 553 seconds loaded 74807831229 Bytes, about 129 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162410663 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docker][fix] add kafka file for p0 regression [doris]
xiaokang commented on PR #25522: URL: https://github.com/apache/doris/pull/25522#issuecomment-1765917423 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](config) turnoff fuzzy for `enable_simdjson_reader` [doris]
eldenmoon commented on PR #25521: URL: https://github.com/apache/doris/pull/25521#issuecomment-1765919774 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](dynamic) Fix error reporting when dynamic partition properties contain incorrect attributes [doris]
doris-robot commented on PR #25373: URL: https://github.com/apache/doris/pull/25373#issuecomment-1765924489 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 46.19 seconds stream load tsv: 561 seconds loaded 74807831229 Bytes, about 127 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.0 seconds inserted 1000 Rows, about 344K ops/s storage size: 17162194762 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](query execution) Fix result sink fragment can't be cancelled in non-pipeline [doris]
github-actions[bot] commented on code in PR #25524:
URL: https://github.com/apache/doris/pull/25524#discussion_r1361723807
##
be/src/runtime/fragment_mgr.cpp:
##
@@ -951,6 +951,14 @@ void FragmentMgr::_set_scan_concurrency(const Param&
params, QueryContext* query
#endif
}
+bool FragmentMgr::is_pipeline_fragment(const TUniqueId& id) {
+std::unique_lock state_lock(_lock);
+if (_pipeline_map.contains(id)) {
+return true;
+}
Review Comment:
warning: redundant boolean literal in conditional return statement
[readability-simplify-boolean-expr]
be/src/runtime/fragment_mgr.cpp:956:
```diff
- if (_pipeline_map.contains(id)) {
- return true;
- }
- return false;
+ return _pipeline_map.contains(id);
```
##
be/src/runtime/fragment_mgr.cpp:
##
@@ -951,6 +951,14 @@ void FragmentMgr::_set_scan_concurrency(const Param&
params, QueryContext* query
#endif
}
+bool FragmentMgr::is_pipeline_fragment(const TUniqueId& id) {
+std::unique_lock state_lock(_lock);
Review Comment:
warning: method 'is_pipeline_fragment' can be made static
[readability-convert-member-functions-to-static]
```suggestion
static bool FragmentMgr::is_pipeline_fragment(const TUniqueId& id) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [refactor](mysql result format) use new serde framework to tuple conv… [doris]
zhiqiang- opened a new pull request, #25526: URL: https://github.com/apache/doris/pull/25526 pick [#25006](https://github.com/apache/doris/pull/25006) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](partial-update) Correct the alignment process when the table has sequence column and add cases [doris]
bobhan1 commented on PR #25346: URL: https://github.com/apache/doris/pull/25346#issuecomment-1765926302 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](config) turnoff fuzzy for `enable_simdjson_reader` [doris]
github-actions[bot] commented on PR #25521: URL: https://github.com/apache/doris/pull/25521#issuecomment-1765930299 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](query execution) Fix result sink fragment can't be cancelled in non-pipeline [doris]
github-actions[bot] commented on code in PR #25524:
URL: https://github.com/apache/doris/pull/25524#discussion_r1361728413
##
be/src/runtime/fragment_mgr.cpp:
##
@@ -951,6 +951,14 @@ void FragmentMgr::_set_scan_concurrency(const Param&
params, QueryContext* query
#endif
}
+bool FragmentMgr::is_pipeline_fragment(const TUniqueId& id) {
Review Comment:
warning: method 'is_pipeline_fragment' can be made static
[readability-convert-member-functions-to-static]
```suggestion
static bool FragmentMgr::is_pipeline_fragment(const TUniqueId& id) {
```
##
be/src/runtime/fragment_mgr.cpp:
##
@@ -951,6 +951,14 @@ void FragmentMgr::_set_scan_concurrency(const Param&
params, QueryContext* query
#endif
}
+bool FragmentMgr::is_pipeline_fragment(const TUniqueId& id) {
+std::unique_lock state_lock(_lock);
+if (_pipeline_map.contains(id)) {
+return true;
Review Comment:
warning: redundant boolean literal in conditional return statement
[readability-simplify-boolean-expr]
be/src/runtime/fragment_mgr.cpp:955:
```diff
- if (_pipeline_map.contains(id)) {
- return true;
- }
- return false;
+ return _pipeline_map.contains(id);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](iceberg)fix read partitioned iceberg without partition path [doris]
doris-robot commented on PR #25503: URL: https://github.com/apache/doris/pull/25503#issuecomment-1765931941 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 45.45 seconds stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 29.1 seconds inserted 1000 Rows, about 343K ops/s storage size: 17162177714 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
