Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
github-actions[bot] commented on PR #37744: URL: https://github.com/apache/doris/pull/37744#issuecomment-2226921119 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cherry-pick](branch-2.0) Pick "[Enhancement](merge-on-write) Support dynamic delete bitmap cache (#32991)" [doris]
dataroaring merged PR #37602: URL: https://github.com/apache/doris/pull/37602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [cherry-pick](branch-2.0) Pick "[Enhancement](merge-on-write) Support dynamic delete bitmap cache (#32991)" (#37602)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new a805e0b366c [cherry-pick](branch-2.0) Pick
"[Enhancement](merge-on-write) Support dynamic delete bitmap cache (#32991)"
(#37602)
a805e0b366c is described below
commit a805e0b366c34c1e4606ab7477c46cbc44761228
Author: abmdocrt
AuthorDate: Sat Jul 13 22:09:25 2024 +0800
[cherry-pick](branch-2.0) Pick "[Enhancement](merge-on-write) Support
dynamic delete bitmap cache (#32991)" (#37602)
The default delete bitmap cache is set to 100MB, which can be
insufficient and cause performance issues when the amount of user data
is large. To mitigate the problem of an inadequate cache, we will take
the larger of 5% of the total memory and 100MB as the delete bitmap
cache size.
## Proposed changes
Issue Number: close #xxx
---
be/src/common/config.cpp| 6 ++
be/src/common/config.h | 2 ++
be/src/olap/tablet_meta.cpp | 15 ++-
3 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp
index ee9201bf6b5..54f90675610 100644
--- a/be/src/common/config.cpp
+++ b/be/src/common/config.cpp
@@ -777,6 +777,12 @@ DEFINE_Int64(delete_bitmap_agg_cache_capacity,
"104857600");
// s3 config
DEFINE_mInt32(max_remote_storage_count, "10");
+// The default delete bitmap cache is set to 100MB,
+// which can be insufficient and cause performance issues when the amount of
user data is large.
+// To mitigate the problem of an inadequate cache,
+// we will take the larger of 0.5% of the total memory and 100MB as the delete
bitmap cache size.
+DEFINE_String(delete_bitmap_dynamic_agg_cache_limit, "0.5%");
+DEFINE_mInt32(delete_bitmap_agg_cache_stale_sweep_time_sec, "1800");
// reference
https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#broker-version-compatibility
// If the dependent kafka broker version older than 0.10.0.0,
diff --git a/be/src/common/config.h b/be/src/common/config.h
index dc750fc511f..050987a8398 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -828,6 +828,8 @@ DECLARE_mInt32(jdbc_connection_pool_cache_clear_time_sec);
// Global bitmap cache capacity for aggregation cache, size in bytes
DECLARE_Int64(delete_bitmap_agg_cache_capacity);
+DECLARE_String(delete_bitmap_dynamic_agg_cache_limit);
+DECLARE_mInt32(delete_bitmap_agg_cache_stale_sweep_time_sec);
// s3 config
DECLARE_mInt32(max_remote_storage_count);
diff --git a/be/src/olap/tablet_meta.cpp b/be/src/olap/tablet_meta.cpp
index 22cfdce462f..adbfd4d6f38 100644
--- a/be/src/olap/tablet_meta.cpp
+++ b/be/src/olap/tablet_meta.cpp
@@ -39,6 +39,8 @@
#include "olap/tablet_meta_manager.h"
#include "olap/utils.h"
#include "util/debug_points.h"
+#include "util/mem_info.h"
+#include "util/parse_util.h"
#include "util/string_util.h"
#include "util/time.h"
#include "util/uid_util.h"
@@ -926,7 +928,18 @@ bool operator!=(const TabletMeta& a, const TabletMeta& b) {
}
DeleteBitmap::DeleteBitmap(int64_t tablet_id) : _tablet_id(tablet_id) {
-_agg_cache.reset(new AggCache(config::delete_bitmap_agg_cache_capacity));
+// The default delete bitmap cache is set to 100MB,
+// which can be insufficient and cause performance issues when the amount
of user data is large.
+// To mitigate the problem of an inadequate cache,
+// we will take the larger of 0.5% of the total memory and 100MB as the
delete bitmap cache size.
+bool is_percent = false;
+int64_t delete_bitmap_agg_cache_cache_limit =
+
ParseUtil::parse_mem_spec(config::delete_bitmap_dynamic_agg_cache_limit,
+ MemInfo::mem_limit(),
MemInfo::physical_mem(), &is_percent);
+_agg_cache.reset(new AggCache(delete_bitmap_agg_cache_cache_limit >
+
config::delete_bitmap_agg_cache_capacity
+ ? delete_bitmap_agg_cache_cache_limit
+ :
config::delete_bitmap_agg_cache_capacity));
}
DeleteBitmap::DeleteBitmap(const DeleteBitmap& o) {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cherry-pick](branch-2.0) pick "[Enhancement](mor) Add unique mor table min max push down case #32196" [doris]
dataroaring merged PR #37599: URL: https://github.com/apache/doris/pull/37599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [cherry-pick](branch-2.0) pick "[Enhancement](mor) Add unique mor table min max push down case #32196" (#37599)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 3cd0d365994 [cherry-pick](branch-2.0) pick "[Enhancement](mor) Add
unique mor table min max push down case #32196" (#37599)
3cd0d365994 is described below
commit 3cd0d3659941047541acebf9370cd33bf818a13d
Author: abmdocrt
AuthorDate: Sat Jul 13 22:09:57 2024 +0800
[cherry-pick](branch-2.0) pick "[Enhancement](mor) Add unique mor table min
max push down case #32196" (#37599)
## Proposed changes
Issue Number: close #xxx
---
.../nereids_p0/explain/test_pushdown_explain.out | 74 +++
.../explain/test_pushdown_explain.groovy | 86 ++
2 files changed, 160 insertions(+)
diff --git a/regression-test/data/nereids_p0/explain/test_pushdown_explain.out
b/regression-test/data/nereids_p0/explain/test_pushdown_explain.out
index 2f364bf9bec..eaac687e2d5 100644
--- a/regression-test/data/nereids_p0/explain/test_pushdown_explain.out
+++ b/regression-test/data/nereids_p0/explain/test_pushdown_explain.out
@@ -101,6 +101,80 @@ qdf
-- !select_18 --
zzz
+-- !select_mor_0 --
+1 asd cc
+2 qwe vvx
+3 ffsdmnm
+4 qdf ll
+5 cvfvvff
+
+-- !select_mor_1 --
+1
+
+-- !select_mor_2 --
+5
+
+-- !select_mor_3 --
+asd
+
+-- !select_mor_4 --
+qwe
+
+-- !select_mor_5 --
+cc
+
+-- !select_mor_6 --
+vvx
+
+-- !select_mor_00 --
+1 asd zzz
+2 qwe vvx
+3 ffsdmnm
+4 qdf ll
+5 cvfvvff
+
+-- !select_mor_7 --
+1
+
+-- !select_mor_8 --
+5
+
+-- !select_mor_9 --
+asd
+
+-- !select_mor_10 --
+qwe
+
+-- !select_mor_11 --
+cc
+
+-- !select_mor_12 --
+zzz
+
+-- !select_mor_000 --
+1 asd zzz
+3 ffsdmnm
+4 qdf ll
+5 cvfvvff
+
+-- !select_mor_13 --
+1
+
+-- !select_mor_14 --
+5
+
+-- !select_mor_15 --
+asd
+
+-- !select_mor_16 --
+qdf
+
+-- !select_mor_17 --
+cc
+
+-- !select_mor_18 --
+zzz
+
-- !select_19 --
1
diff --git
a/regression-test/suites/nereids_p0/explain/test_pushdown_explain.groovy
b/regression-test/suites/nereids_p0/explain/test_pushdown_explain.groovy
index ec8d9b5d8ae..ce74d60687c 100644
--- a/regression-test/suites/nereids_p0/explain/test_pushdown_explain.groovy
+++ b/regression-test/suites/nereids_p0/explain/test_pushdown_explain.groovy
@@ -183,6 +183,92 @@ suite("test_pushdown_explain") {
qt_select_16 "select max(username) from table_unique;"
qt_select_18 "select max(val) from table_unique;"
+sql "DROP TABLE IF EXISTS table_unique11"
+sql """
+CREATE TABLE `table_unique11` (
+`user_id` LARGEINT NOT NULL COMMENT '\"用户id\"',
+`username` VARCHAR(50) NOT NULL COMMENT '\"用户昵称\"',
+`val` VARCHAR(50) NULL
+) ENGINE=OLAP
+UNIQUE KEY(`user_id`, `username`)
+COMMENT 'OLAP'
+DISTRIBUTED BY HASH(`user_id`) BUCKETS 1
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 1",
+"disable_auto_compaction" = "true",
+"enable_unique_key_merge_on_write" = "false"
+);
+"""
+sql """
+insert into table_unique11
values(1,"asd","cc"),(2,"qwe","vvx"),(3,"ffsd","mnm"),(4,"qdf","ll"),(5,"cvfv","vff");
+"""
+
+sql "set enable_pushdown_minmax_on_unique = false;"
+explain {
+sql("select min(user_id) from table_unique11;")
+contains "pushAggOp=NONE"
+}
+explain {
+sql("select max(user_id) from table_unique11;")
+contains "pushAggOp=NONE"
+}
+explain {
+sql("select min(username) from table_unique11;")
+contains "pushAggOp=NONE"
+}
+explain {
+sql("select max(username) from table_unique11;")
+contains "pushAggOp=NONE"
+}
+
+
+// set seession variables
+sql "set enable_pushdown_minmax_on_unique = true;"
+explain {
+sql("select min(user_id) from table_unique11;")
+contains "pushAggOp=MINMAX"
+}
+explain {
+sql("select max(user_id) from table_unique11;")
+contains "pushAggOp=MINMAX"
+}
+explain {
+sql("select min(username) from table_unique11;")
+contains "pushAggOp=MINMAX"
+}
+explain {
+sql("select max(username) from table_unique11;")
+contains "pushAggOp=MINMAX"
+}
+qt_select_mor_0 "select * from table_unique11 order by user_id;"
+qt_select_mor_1 "select min(user_id) from table_unique11;"
+qt_select_mor_2 "select max(user_id) from table_unique11;"
+qt_select_mor_3 "select min(username) from table_unique11;"
+qt_select_mor_4 "select max(username) from table_unique11;"
+qt_select_mor_5 "select min(val) from table_unique11;"
+qt_select_mor_6 "select max(val) from table_unique11;"
+sql """
+update tab
Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
github-actions[bot] commented on PR #37745: URL: https://github.com/apache/doris/pull/37745#issuecomment-2226922062 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auto bucket) Fix hit not support alter estimate_partition_size #33670 [doris]
dataroaring merged PR #37633: URL: https://github.com/apache/doris/pull/37633 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [fix](auto bucket) Fix hit not support alter estimate_partition_size #33670 (#37633)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new ec8467f57bd [fix](auto bucket) Fix hit not support alter
estimate_partition_size #33670 (#37633)
ec8467f57bd is described below
commit ec8467f57bd6990b922a207a1f00500355496be4
Author: deardeng <565620...@qq.com>
AuthorDate: Sat Jul 13 22:12:38 2024 +0800
[fix](auto bucket) Fix hit not support alter estimate_partition_size #33670
(#37633)
cherry pick from #33670
---
.../java/org/apache/doris/analysis/ModifyTablePropertiesClause.java | 2 ++
1 file changed, 2 insertions(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
index b5e4c46d91a..3a1d7ccb648 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyTablePropertiesClause.java
@@ -307,6 +307,8 @@ public class ModifyTablePropertiesClause extends
AlterTableClause {
}
this.needTableStable = false;
this.opType = AlterOpType.MODIFY_TABLE_PROPERTY_SYNC;
+} else if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_ESTIMATE_PARTITION_SIZE)) {
+throw new AnalysisException("You can not modify estimate partition
size");
} else {
throw new AnalysisException("Unknown table property: " +
properties.keySet());
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
dataroaring merged PR #37744: URL: https://github.com/apache/doris/pull/37744 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition) Fix be tablet partition id eq 0 By report tablet #32179 [doris]
dataroaring merged PR #37588: URL: https://github.com/apache/doris/pull/37588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](partition) Fix be tablet partition id eq 0 By report tablet #32179 (#37588)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 15a94ba691d [fix](partition) Fix be tablet partition id eq 0 By report
tablet #32179 (#37588)
15a94ba691d is described below
commit 15a94ba691d282c7dab28bd4004875da9806e225
Author: deardeng <565620...@qq.com>
AuthorDate: Sat Jul 13 22:13:06 2024 +0800
[fix](partition) Fix be tablet partition id eq 0 By report tablet #32179
(#37588)
cherry pick from #32179
---
be/src/agent/task_worker_pool.cpp| 17 +
be/src/olap/tablet_manager.cpp | 20
be/src/olap/tablet_manager.h | 3 +++
be/src/olap/tablet_meta.cpp | 4 ++--
.../main/java/org/apache/doris/common/Config.java| 2 ++
.../apache/doris/catalog/TabletInvertedIndex.java| 9 +
6 files changed, 53 insertions(+), 2 deletions(-)
diff --git a/be/src/agent/task_worker_pool.cpp
b/be/src/agent/task_worker_pool.cpp
index 5d67a5bac08..ea56c2a8a22 100644
--- a/be/src/agent/task_worker_pool.cpp
+++ b/be/src/agent/task_worker_pool.cpp
@@ -417,6 +417,23 @@ void
TaskWorkerPool::_update_tablet_meta_worker_thread_callback() {
continue;
}
bool need_to_save = false;
+if (tablet_meta_info.__isset.partition_id) {
+// for fix partition_id = 0
+LOG(WARNING) << "change be tablet id: " <<
tablet->tablet_meta()->tablet_id()
+ << "partition id from : " <<
tablet->tablet_meta()->partition_id()
+ << " to : " << tablet_meta_info.partition_id;
+auto succ =
StorageEngine::instance()->tablet_manager()->update_tablet_partition_id(
+tablet_meta_info.partition_id,
tablet->tablet_meta()->tablet_id());
+if (!succ) {
+std::string err_msg = fmt::format(
+"change be tablet id : {} partition_id : {}
failed",
+tablet->tablet_meta()->tablet_id(),
tablet_meta_info.partition_id);
+LOG(WARNING) << err_msg;
+status = Status::InvalidArgument(err_msg);
+continue;
+}
+need_to_save = true;
+}
if (tablet_meta_info.__isset.storage_policy_id) {
tablet->tablet_meta()->set_storage_policy_id(tablet_meta_info.storage_policy_id);
need_to_save = true;
diff --git a/be/src/olap/tablet_manager.cpp b/be/src/olap/tablet_manager.cpp
index 61c7809cf54..573ec920160 100644
--- a/be/src/olap/tablet_manager.cpp
+++ b/be/src/olap/tablet_manager.cpp
@@ -1653,4 +1653,24 @@ std::set
TabletManager::check_all_tablet_segment(bool repair) {
return bad_tablets;
}
+bool TabletManager::update_tablet_partition_id(::doris::TPartitionId
partition_id,
+ ::doris::TTabletId tablet_id) {
+std::shared_lock rdlock(_get_tablets_shard_lock(tablet_id));
+TabletSharedPtr tablet = _get_tablet_unlocked(tablet_id);
+if (tablet == nullptr) {
+LOG(WARNING) << "get tablet err partition_id: " << partition_id
+ << " tablet_id:" << tablet_id;
+return false;
+}
+_remove_tablet_from_partition(tablet);
+auto st = tablet->tablet_meta()->set_partition_id(partition_id);
+if (!st.ok()) {
+LOG(WARNING) << "set partition id err partition_id: " << partition_id
+ << " tablet_id:" << tablet_id;
+return false;
+}
+_add_tablet_to_partition(tablet);
+return true;
+}
+
} // end namespace doris
diff --git a/be/src/olap/tablet_manager.h b/be/src/olap/tablet_manager.h
index 07c9d563b87..d33cd6e4bc4 100644
--- a/be/src/olap/tablet_manager.h
+++ b/be/src/olap/tablet_manager.h
@@ -166,6 +166,9 @@ public:
std::set check_all_tablet_segment(bool repair);
+bool update_tablet_partition_id(::doris::TPartitionId partition_id,
+::doris::TTabletId tablet_id);
+
private:
// Add a tablet pointer to StorageEngine
// If force, drop the existing tablet add this new one
diff --git a/be/src/olap/tablet_meta.cpp b/be/src/olap/tablet_meta.cpp
index adbfd4d6f38..0d9eb5316f1 100644
--- a/be/src/olap/tablet_meta.cpp
+++ b/be/src/olap/tablet_meta.cpp
@@ -879,8 +879,8 @@ std::string TabletMeta::full_name() const {
Status TabletMeta::set_partition_id(int64_t partition_id) {
if ((_partition_id > 0 && _partition_id != partition_id) || partition_id <
1) {
-LOG(FATAL) << "cur partition id=" << _partition_id << " new partition
id=" << partition_id
- << " not equal";
+LOG(WARNING) << "cur partition
(doris) branch branch-3.0 updated: [Fix](inverted index) fix fast execute for not_in expr (#37744)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 99c601d989c [Fix](inverted index) fix fast execute for not_in expr
(#37744)
99c601d989c is described below
commit 99c601d989ce3da9917558e63d80cbae344a8b8c
Author: airborne12
AuthorDate: Sat Jul 13 22:14:29 2024 +0800
[Fix](inverted index) fix fast execute for not_in expr (#37744)
## Proposed changes
Pick from #37745
---
be/src/vec/exprs/vexpr.cpp | 2 +-
be/src/vec/functions/function.h| 2 +-
.../data/inverted_index_p0/test_index_rqg_bug3.out | 43 ++
.../inverted_index_p0/test_index_rqg_bug3.groovy | 97 ++
4 files changed, 142 insertions(+), 2 deletions(-)
diff --git a/be/src/vec/exprs/vexpr.cpp b/be/src/vec/exprs/vexpr.cpp
index 3a965c27c92..a0ba24115d1 100644
--- a/be/src/vec/exprs/vexpr.cpp
+++ b/be/src/vec/exprs/vexpr.cpp
@@ -627,7 +627,7 @@ std::string VExpr::gen_predicate_result_sign(Block& block,
const ColumnNumbers&
std::string column_name = block.get_by_position(arguments[0]).name;
pred_result_sign +=
BeConsts::BLOCK_TEMP_COLUMN_PREFIX + column_name + "_" +
function_name + "_";
-if (function_name == "in") {
+if (function_name == "in" || function_name == "not_in") {
// Generating 'result_sign' from 'inlist' requires sorting the values.
std::set values;
for (size_t i = 1; i < arguments.size(); i++) {
diff --git a/be/src/vec/functions/function.h b/be/src/vec/functions/function.h
index f3af3870c2e..36558c11df0 100644
--- a/be/src/vec/functions/function.h
+++ b/be/src/vec/functions/function.h
@@ -521,7 +521,7 @@ public:
auto function_name = function->get_name();
return function_name == "eq" || function_name == "ne" || function_name
== "lt" ||
function_name == "gt" || function_name == "le" || function_name
== "ge" ||
- function_name == "in";
+ function_name == "in" || function_name == "not_in";
}
Status eval_inverted_index(FunctionContext* context,
diff --git a/regression-test/data/inverted_index_p0/test_index_rqg_bug3.out
b/regression-test/data/inverted_index_p0/test_index_rqg_bug3.out
new file mode 100644
index 000..cd01bedc787
--- /dev/null
+++ b/regression-test/data/inverted_index_p0/test_index_rqg_bug3.out
@@ -0,0 +1,43 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !select_bug_1 --
+-102023-12-11
+-102023-12-12
+-102023-12-13
+-102023-12-15
+-102023-12-15
+-102023-12-19
+-102023-12-19
+-102024-01-17
+-102024-02-18
+-102024-02-18
+-102025-02-18
+-102026-01-18
+-102026-02-18
+-4 2023-12-10
+-4 2023-12-11
+-4 2023-12-16
+-4 2024-01-31
+0 2024-01-19
+1 2023-12-16
+1 2024-01-09
+2 2023-12-10
+2 2023-12-11
+2 2024-01-08
+2 2024-01-31
+3 2023-12-20
+3 2024-01-19
+3 2025-06-18
+3 2026-02-18
+3 2027-01-16
+4 2023-12-12
+4 2023-12-12
+4 2024-01-08
+5 2023-12-16
+6 2024-02-18
+7 2023-12-17
+7 2023-12-20
+7 2027-01-09
+8 2025-02-18
+9 2024-02-18
+9 2024-02-18
+
diff --git
a/regression-test/suites/inverted_index_p0/test_index_rqg_bug3.groovy
b/regression-test/suites/inverted_index_p0/test_index_rqg_bug3.groovy
new file mode 100644
index 000..4179ad3940e
--- /dev/null
+++ b/regression-test/suites/inverted_index_p0/test_index_rqg_bug3.groovy
@@ -0,0 +1,97 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+suite("test_index_rqg_bug3", "test_index_rqg_bug3"){
+def table1 = "test_index_rqg_bug3"
+
+sql "drop table if exists ${table1}"
+
+sql """
+CREATE TABLE ${table1} (
+ `col_int_undef_signed_not_null` INT NOT NULL,
+ `col_date_undef_signed_not_null` DATE NOT NULL,
+ `col_bigint_undef_signed_not_null_index_inverted` BIGINT NOT NULL,
+ `col_bigint_undef_signed_not_null` BIGI
Re: [PR] [fix](readconsistency) avoid table not exist error #37593 [doris]
dataroaring merged PR #37709: URL: https://github.com/apache/doris/pull/37709 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](readconsistency) avoid table not exist error #37593 (#37709)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new c7e492deb44 [fix](readconsistency) avoid table not exist error #37593
(#37709)
c7e492deb44 is described below
commit c7e492deb44a7cfd9b0e90f0055ce5188cca9f1b
Author: Yongqiang YANG <98214048+dataroar...@users.noreply.github.com>
AuthorDate: Sat Jul 13 22:18:53 2024 +0800
[fix](readconsistency) avoid table not exist error #37593 (#37709)
pick #37593
## Proposed changes
Issue Number: close #xxx
---
.../java/org/apache/doris/qe/StmtExecutor.java | 31 --
1 file changed, 29 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
index 03ec39818b8..342b38892da 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
@@ -588,6 +588,13 @@ public class StmtExecutor {
return;
}
}
+// Query following createting table would throw table not exist
error.
+// For example.
+// t1: client issues create table to master fe
+// t2: client issues query sql to observer fe, the query would
fail due to not exist table in
+// plan phase.
+// t3: observer fe receive editlog creating the table from the
master fe
+syncJournalIfNeeded();
try {
((Command) logicalPlan).run(context, this);
} catch (QueryStateException e) {
@@ -611,6 +618,13 @@ public class StmtExecutor {
ConnectContext.get().setStatsErrorEstimator(new
StatsErrorEstimator());
}
// create plan
+// Query following createting table would throw table not exist
error.
+// For example.
+// t1: client issues create table to master fe
+// t2: client issues query sql to observer fe, the query would
fail due to not exist table in
+// plan phase.
+// t3: observer fe receive editlog creating the table from the
master fe
+syncJournalIfNeeded();
planner = new NereidsPlanner(statementContext);
try {
planner.plan(parsedStmt,
context.getSessionVariable().toThrift());
@@ -643,7 +657,6 @@ public class StmtExecutor {
private void handleQueryWithRetry(TUniqueId queryId) throws Exception {
// queue query here
-syncJournalIfNeeded();
QueueOfferToken offerRet = null;
QueryQueue queryQueue = null;
if (!parsedStmt.isExplain() && Config.enable_workload_group &&
Config.enable_query_queue
@@ -760,6 +773,13 @@ public class StmtExecutor {
LOG.debug("no need to transfer to Master. stmt: {}",
context.getStmtId());
}
} else {
+// Query following createting table would throw table not
exist error.
+// For example.
+// t1: client issues create table to master fe
+// t2: client issues query sql to observer fe, the query would
fail due to not exist table
+// in plan phase.
+// t3: observer fe receive editlog creating the table from the
master fe
+syncJournalIfNeeded();
analyzer = new Analyzer(context.getEnv(), context);
parsedStmt.analyze(analyzer);
parsedStmt.checkPriv();
@@ -947,7 +967,7 @@ public class StmtExecutor {
}
// Analyze one statement to structure in memory.
-public void analyze(TQueryOptions tQueryOptions) throws UserException,
InterruptedException {
+public void analyze(TQueryOptions tQueryOptions) throws UserException,
InterruptedException, Exception {
if (LOG.isDebugEnabled()) {
LOG.debug("begin to analyze stmt: {}, forwarded stmt id: {}",
context.getStmtId(),
context.getForwardedStmtId());
@@ -988,6 +1008,13 @@ public class StmtExecutor {
return;
}
+// Query following createting table would throw table not exist error.
+// For example.
+// t1: client issues create table to master fe
+// t2: client issues query sql to observer fe, the query would fail
due to not exist table in
+// plan phase.
+// t3: observer fe receive editlog creating the table from the master
fe
+syncJournalIfNeeded();
analyzer = new Analyzer(context.getEnv(), context);
if (parsedStmt instanceof PrepareStmt || context.getCommand() ==
MysqlCommand.COM_STMT_PREPARE) {
-
Re: [PR] [chore](UT) Add UT for cloud string util [doris]
github-actions[bot] commented on PR #37552: URL: https://github.com/apache/doris/pull/37552#issuecomment-2226926942 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](regression-test) dup modify key case1 [doris]
dataroaring merged PR #37475: URL: https://github.com/apache/doris/pull/37475 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [enhancement](regression-test) dup modify key case1 (#37475)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 8ba68d48c39 [enhancement](regression-test) dup modify key case1
(#37475)
8ba68d48c39 is described below
commit 8ba68d48c3945c08fbbffb38efb8d8f9570cd9f7
Author: kkop <2449402...@qq.com>
AuthorDate: Sat Jul 13 22:22:23 2024 +0800
[enhancement](regression-test) dup modify key case1 (#37475)
---
.../test_dup_schema_key_change_modify1.groovy | 2422
1 file changed, 2422 insertions(+)
diff --git
a/regression-test/suites/schema_change_p0/test_dup_schema_key_change_modify1.groovy
b/regression-test/suites/schema_change_p0/test_dup_schema_key_change_modify1.groovy
new file mode 100644
index 000..3f6a3def1f5
--- /dev/null
+++
b/regression-test/suites/schema_change_p0/test_dup_schema_key_change_modify1.groovy
@@ -0,0 +1,2422 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_dup_schema_key_change_modify1","p0") {
+ def tbName1 = "test_dup_schema_key_change_modify1"
+ def tbName2 = "test_dup_schema_key_change_modify_1"
+ /**
+ * Test the dup model by modify a value type
+ */
+
+ sql """ DROP TABLE IF EXISTS ${tbName1} """
+ def getTableStatusSql = " SHOW ALTER TABLE COLUMN WHERE
IndexName='${tbName1}' ORDER BY createtime DESC LIMIT 1 "
+ def initTable = " CREATE TABLE IF NOT EXISTS ${tbName1}\n" +
+ " (\n" +
+ " `user_id` LARGEINT NOT NULL COMMENT \"用户id\",\n" +
+ " `username` VARCHAR(50) NOT NULL COMMENT
\"用户昵称\",\n" +
+ " `is_teacher` BOOLEAN COMMENT \"是否是老师\",\n" +
+ " `city` VARCHAR(20) COMMENT \"用户所在城市\",\n" +
+ " `age` SMALLINT COMMENT \"用户年龄\",\n" +
+ " `sex` TINYINT COMMENT \"用户性别\",\n" +
+ " `phone` LARGEINT COMMENT \"用户电话\",\n" +
+ " `address` VARCHAR(500) COMMENT \"用户地址\",\n" +
+ " `register_time` DATETIME COMMENT \"用户注册时间\"\n" +
+ " )\n" +
+ " DUPLICATE KEY(`user_id`, `username`, `is_teacher`)\n" +
+ " DISTRIBUTED BY HASH(`user_id`) BUCKETS 1\n" +
+ " PROPERTIES (\n" +
+ " \"replication_allocation\" = \"tag.location.default:
1\"\n" +
+ " );"
+
+ def initTableData = "insert into ${tbName1} values(123456789, 'Alice', 0,
'Beijing', 25, 0, 13812345678, 'No. 123 Street, Beijing', '2022-01-01
10:00:00')," +
+ " (234567890, 'Bob', 0, 'Shanghai', 30, 1,
13998765432, 'No. 456 Street, Shanghai', '2022-02-02 12:00:00')," +
+ " (345678901, 'Carol', 1, 'Guangzhou', 28, 0,
13724681357, 'No. 789 Street, Guangzhou', '2022-03-03 14:00:00')," +
+ " (456789012, 'Dave', 0, 'Shenzhen', 35, 1,
13680864279, 'No. 987 Street, Shenzhen', '2022-04-04 16:00:00')," +
+ " (567890123, 'Eve', 0, 'Chengdu', 27, 0,
13572468091, 'No. 654 Street, Chengdu', '2022-05-05 18:00:00')," +
+ " (678901234, 'Frank', 1, 'Hangzhou', 32, 1,
13467985213, 'No. 321 Street, Hangzhou', '2022-06-06 20:00:00')," +
+ " (789012345, 'Grace', 0, 'Xian', 29, 0,
133, 'No. 222 Street, Xian', '2022-07-07 22:00:00');"
+ def initTable1 = ""
+ def initTableData1 = ""
+
+ /**
+ * Test the dup model by modify a key type from LARGEINT to other type
+ */
+ sql """ DROP TABLE IF EXISTS ${tbName1} """
+ initTable = " CREATE TABLE IF NOT EXISTS ${tbName1}\n" +
+ " (\n" +
+ " `user_id` LARGEINT NOT NULL COMMENT \"用户id\",\n" +
+ " `username` VARCHAR(50) NOT NULL COMMENT
\"用户昵称\",\n" +
+ " `sn_number` LARGEINT COMMENT \"sn卡\",\n" +
+ " `city` VARCHAR(20) COMMENT \"用户所在城市\",\n" +
+
Re: [PR] [fix]fix be core when migration tablet to other disk [doris]
github-actions[bot] commented on PR #37712: URL: https://github.com/apache/doris/pull/37712#issuecomment-2226929606 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](delete) fix the error message for valid decimal data for 2.1 [doris]
dataroaring commented on PR #37710: URL: https://github.com/apache/doris/pull/37710#issuecomment-2226929686 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix]fix be core when migration tablet to other disk [doris]
github-actions[bot] commented on PR #37712: URL: https://github.com/apache/doris/pull/37712#issuecomment-2226929611 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix]fix be core when migration tablet to other disk [doris]
bobhan1 commented on PR #37712: URL: https://github.com/apache/doris/pull/37712#issuecomment-2226930570 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](regression-test) schema reordering case [doris]
dataroaring commented on PR #37521: URL: https://github.com/apache/doris/pull/37521#issuecomment-2226931534 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Test](regression-test): do not use path style to access s3 [doris]
hello-stephen merged PR #35787: URL: https://github.com/apache/doris/pull/35787 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [Test](regression-test): do not use path style to access s3 (#35787)
This is an automated email from the ASF dual-hosted git repository.
hellostephen pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new ec0bd03108c [Test](regression-test): do not use path style to access
s3 (#35787)
ec0bd03108c is described below
commit ec0bd03108ce243cea3c2a0222d0e261073250f2
Author: Thearas
AuthorDate: Sat Jul 13 22:38:33 2024 +0800
[Test](regression-test): do not use path style to access s3 (#35787)
## Proposed changes
CP from #35666 and #35725.
-
Co-authored-by: Kang
---
.../suites/export_p0/export/test_show_export.groovy | 4 ++--
.../outfile/csv/test_outfile_empty_data.groovy | 4 ++--
.../suites/export_p0/test_outfile_file_suffix.groovy | 4 ++--
.../suites/export_p0/test_with_bom.groovy| 8
.../suites/export_p2/test_export_with_s3.groovy | 2 +-
.../tvf/test_insert_from_tvf_with_common_user.groovy | 2 +-
.../suites/external_table_p0/tvf/test_s3_tvf.groovy | 6 +++---
.../tvf/test_s3_tvf_compression.groovy | 20 ++--
.../tvf/test_path_partition_keys.groovy | 16
.../suites/load_p0/tvf/test_tvf_error_url.groovy | 8
10 files changed, 37 insertions(+), 37 deletions(-)
diff --git a/regression-test/suites/export_p0/export/test_show_export.groovy
b/regression-test/suites/export_p0/export/test_show_export.groovy
index fb90e111c91..ea60b1ebe83 100644
--- a/regression-test/suites/export_p0/export/test_show_export.groovy
+++ b/regression-test/suites/export_p0/export/test_show_export.groovy
@@ -119,7 +119,7 @@ suite("test_show_export", "p0") {
def outfile_url = waiting_export.call(label)
order_qt_select_load1 """ select * from s3(
-"uri" =
"http://${s3_endpoint}${outfile_url.substring(4)}0.parquet",
+"uri" =
"http://${bucket}.${s3_endpoint}${outfile_url.substring(5 + bucket.length(),
outfile_url.length())}0.parquet",
"s3.access_key"= "${ak}",
"s3.secret_key" = "${sk}",
"format" = "parquet",
@@ -154,7 +154,7 @@ suite("test_show_export", "p0") {
def outfile_url = waiting_export.call(label)
order_qt_select_load1 """ select * from s3(
-"uri" =
"http://${s3_endpoint}${outfile_url.substring(4)}0.parquet",
+"uri" =
"http://${bucket}.${s3_endpoint}${outfile_url.substring(5 + bucket.length(),
outfile_url.length())}0.parquet",
"s3.access_key"= "${ak}",
"s3.secret_key" = "${sk}",
"format" = "parquet",
diff --git
a/regression-test/suites/export_p0/outfile/csv/test_outfile_empty_data.groovy
b/regression-test/suites/export_p0/outfile/csv/test_outfile_empty_data.groovy
index 1804fff2a11..25e0dbbeec9 100644
---
a/regression-test/suites/export_p0/outfile/csv/test_outfile_empty_data.groovy
+++
b/regression-test/suites/export_p0/outfile/csv/test_outfile_empty_data.groovy
@@ -152,12 +152,12 @@ suite("test_outfile_empty_data",
"external,hive,tvf,external_docker") {
"""
qt_select_tvf3 """ SELECT * FROM S3 (
-"uri" =
"http://${s3_endpoint}${outfile_to_s3_directly_url.substring(4,
outfile_to_s3_directly_url.length())}0.csv",
+"uri" =
"http://${bucket}.${s3_endpoint}${outfile_to_s3_directly_url.substring(5 +
bucket.length(), outfile_to_s3_directly_url.length())}0.csv",
"ACCESS_KEY"= "${ak}",
"SECRET_KEY" = "${sk}",
"format" = "${format}",
"region" = "${region}",
-"use_path_style" = "true"
+"use_path_style" = "false" -- aliyun does not support
path_style
);
"""
diff --git a/regression-test/suites/export_p0/test_outfile_file_suffix.groovy
b/regression-test/suites/export_p0/test_outfile_file_suffix.groovy
index 30f9fea23de..bbd791052c9 100644
--- a/regression-test/suites/export_p0/test_outfile_file_suffix.groovy
+++ b/regression-test/suites/export_p0/test_outfile_file_suffix.groovy
@@ -60,9 +60,9 @@ suite("test_outfile_file_suffix", "p0") {
def file_suffix = "txt";
def file_format = "csv";
def outfile_url = csv_suffix_result(file_suffix, file_format);
-print("http://${s3_endpoint}${outfile_url.substring(4)}0.${file_suffix}")
+print("http://${bucket}.${s3_endpoint}${outfile_url.substring(5 +
bucket.length(), outfile_url.length())}0.${file_suffix}")
qt_select """ select * from s3(
-"uri" =
"http://${s3_endpoint}${outfile_url.substring(4)}0.${file_suffix}",
+"uri" =
"http://${bucket}.${s3_endpoint}${outfile_url.substring(5 + bucket.length(),
outfile_url.length())}0.${file_suffix}",
"ACCESS_KEY"= "${ak}",
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
gavinchou commented on code in PR #37677:
URL: https://github.com/apache/doris/pull/37677#discussion_r1676836679
##
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##
@@ -2491,11 +2491,15 @@ private boolean createOlapTable(Database db,
CreateTableStmt stmt) throws UserEx
if (info != null) {
storageVaultName = info.first;
storageVaultId = info.second;
+} else {
+throw new DdlException("No Default Storage Vault. You can
use show storage vault stmt to"
++ "get full vaults, and set one as default storage
vault");
Review Comment:
```suggestion
throw new DdlException("No default storage vault."
+ " You can use stmt `SHOW STORAGE VAULT` to get
all available vaults,"
+ " and set default vault with `SET
AS DEFAULT STORAGE VAULT`");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
gavinchou commented on code in PR #37677:
URL: https://github.com/apache/doris/pull/37677#discussion_r1676837312
##
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##
@@ -2491,11 +2491,15 @@ private boolean createOlapTable(Database db,
CreateTableStmt stmt) throws UserEx
if (info != null) {
storageVaultName = info.first;
storageVaultId = info.second;
+} else {
+throw new DdlException("No Default Storage Vault. You can
use show storage vault stmt to"
++ "get full vaults, and set one as default storage
vault");
}
}
if (storageVaultName == null || storageVaultName.isEmpty()) {
-throw new DdlException("Invalid Storage Vault, please set one
useful storage vault");
+throw new DdlException("Invalid Storage Vault. You can use
show storage vault stmt to"
++ "get full vaults, and pick one to set the table's
property");
Review Comment:
```suggestion
throw new DdlException("Invalid Storage Vault. "
+ " You can use `SHOW STORAGE VAULT` to get all
available vaults,"
+ ", and pick one to set the table property
`\"storage_vault_name\" = \"\"`");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
gavinchou commented on code in PR #37677:
URL: https://github.com/apache/doris/pull/37677#discussion_r1676836679
##
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##
@@ -2491,11 +2491,15 @@ private boolean createOlapTable(Database db,
CreateTableStmt stmt) throws UserEx
if (info != null) {
storageVaultName = info.first;
storageVaultId = info.second;
+} else {
+throw new DdlException("No Default Storage Vault. You can
use show storage vault stmt to"
++ "get full vaults, and set one as default storage
vault");
Review Comment:
```suggestion
throw new DdlException("No default storage vault."
+ " You can use `SHOW STORAGE VAULT` to get all
available vaults,"
+ " and pick one set default vault with `SET
AS DEFAULT STORAGE VAULT`");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
gavinchou commented on code in PR #37677:
URL: https://github.com/apache/doris/pull/37677#discussion_r1676837312
##
fe/fe-core/src/main/java/org/apache/doris/datasource/InternalCatalog.java:
##
@@ -2491,11 +2491,15 @@ private boolean createOlapTable(Database db,
CreateTableStmt stmt) throws UserEx
if (info != null) {
storageVaultName = info.first;
storageVaultId = info.second;
+} else {
+throw new DdlException("No Default Storage Vault. You can
use show storage vault stmt to"
++ "get full vaults, and set one as default storage
vault");
}
}
if (storageVaultName == null || storageVaultName.isEmpty()) {
-throw new DdlException("Invalid Storage Vault, please set one
useful storage vault");
+throw new DdlException("Invalid Storage Vault. You can use
show storage vault stmt to"
++ "get full vaults, and pick one to set the table's
property");
Review Comment:
```suggestion
throw new DdlException("Invalid Storage Vault. "
+ " You can use `SHOW STORAGE VAULT` to get all
available vaults,"
+ " and pick one to set the table property
`\"storage_vault_name\" = \"\"`");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
gavinchou commented on PR #37677: URL: https://github.com/apache/doris/pull/37677#issuecomment-2226937968 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](UT) Add UT for cloud string util [doris]
gavinchou commented on code in PR #37552:
URL: https://github.com/apache/doris/pull/37552#discussion_r1676838938
##
cloud/test/util_test.cpp:
##
@@ -0,0 +1,91 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+
+#include "common/config.h"
+#include "common/string_util.h"
+#include "glog/logging.h"
+#include "gtest/gtest.h"
+
+int main(int argc, char** argv) {
+doris::cloud::config::init(nullptr, true);
+::testing::InitGoogleTest(&argc, argv);
+return RUN_ALL_TESTS();
+}
+
+TEST(StringUtilTest, test_string_strip) {
+// clang-format off
+// str expect to_drop
+std::vector>
leading_inputs {
+{"" , "" , ""},
+{"" , "" , "/" },
+{"/" , "" , "/" },
+{"\t" , "" , "/ \t"},
+{"/a///" , "a///" , "/" },
+{"/a/b/c/", "a/b/c/", "/" },
+{"a/b/c/" , "a/b/c/", "/" },
+{"a/b/c/" , "/b/c/" , "a" },
+};
+int idx = 0;
+for (auto&& i : leading_inputs) {
+doris::cloud::strip_leading(std::get<0>(i), std::get<2>(i));
+EXPECT_EQ(std::get<0>(i), std::get<1>(i)) << " index=" << idx;
+++idx;
+}
+
+idx = 0;
+std::vector>
trailing_inputs {
+{"" , "" , ""},
+{"/" , "" , "/" },
+{"\t" , "" , "/ \t"},
+{"/a///" , "/a", "/" },
+{"/a/b/c/", "/a/b/c", "/" },
+{"a/b/c/" , "a/b/c" , "/" },
+{"a/b/c" , "a/b/c" , "/" },
+{"a/b/c" , "a/b/" , "c" },
+};
+for (auto&& i : trailing_inputs) {
+doris::cloud::strip_trailing(std::get<0>(i), std::get<2>(i));
+EXPECT_EQ(std::get<0>(i), std::get<1>(i)) << " index=" << idx;
+++idx;
+}
+
+idx = 0;
+std::vector> trim_inputs {
+{"" , "" },
+{"" , "" },
+{"/" , "" },
+{"\t", "" },
+{"/a ///", "a " },
Review Comment:
```suggestion
{"/a ///", "a" },
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](UT) Add UT for cloud string util [doris]
gavinchou commented on PR #37552: URL: https://github.com/apache/doris/pull/37552#issuecomment-2226941681 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) reduce memory reserved in memtable limiter (#37511) [doris]
kaijchen commented on PR #37699: URL: https://github.com/apache/doris/pull/37699#issuecomment-2226941782 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) reduce memory reserved in memtable limiter (#37511) [doris]
github-actions[bot] commented on PR #37699: URL: https://github.com/apache/doris/pull/37699#issuecomment-2226945175 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](OSS) Retry when encountering TooManyRequest response [doris]
gavinchou commented on code in PR #37199:
URL: https://github.com/apache/doris/pull/37199#discussion_r1676840335
##
be/src/io/fs/s3_file_reader.cpp:
##
@@ -120,65 +112,25 @@ Status S3FileReader::read_at_impl(size_t offset, Slice
result, size_t* bytes_rea
if (!client) {
return Status::InternalError("init s3 client error");
}
-// // clang-format off
-// auto resp = client->get_object( { .bucket = _bucket, .key = _key, },
-// to, offset, bytes_req, bytes_read);
-// // clang-format on
-// if (resp.status.code != ErrorCode::OK) {
-// return std::move(Status(resp.status.code,
std::move(resp.status.msg))
-// .append(fmt::format("failed to read from
{}", _path.native(;
-// }
-// if (*bytes_read != bytes_req) {
-// return Status::InternalError("failed to read from {}(bytes read:
{}, bytes req: {})",
-// _path.native(), *bytes_read,
bytes_req);
-SCOPED_BVAR_LATENCY(s3_bvar::s3_get_latency);
-
-int retry_count = 0;
-const int base_wait_time = config::s3_read_base_wait_time_ms; // Base wait
time in milliseconds
-const int max_wait_time = config::s3_read_max_wait_time_ms; // Maximum
wait time in milliseconds
-const int max_retries = config::max_s3_client_retry; // wait 1s, 2s, 4s,
8s for each backoff
-
-int total_sleep_time = 0;
-while (retry_count <= max_retries) {
-s3_file_reader_read_counter << 1;
-// clang-format off
-auto resp = client->get_object( { .bucket = _bucket, .key = _key, },
-to, offset, bytes_req, bytes_read);
-// clang-format on
-_s3_stats.total_get_request_counter++;
-if (resp.status.code != ErrorCode::OK) {
-if (resp.http_code ==
-
static_cast(Aws::Http::HttpResponseCode::TOO_MANY_REQUESTS)) {
-s3_file_reader_too_many_request_counter << 1;
-retry_count++;
-int wait_time = std::min(base_wait_time * (1 << retry_count),
- max_wait_time); // Exponential backoff
-
std::this_thread::sleep_for(std::chrono::milliseconds(wait_time));
-_s3_stats.too_many_request_err_counter++;
-_s3_stats.too_many_request_sleep_time_ms += wait_time;
-total_sleep_time += wait_time;
-continue;
-} else {
-// Handle other errors
-return std::move(Status(resp.status.code,
std::move(resp.status.msg))
- .append("failed to read"));
-}
-}
-if (*bytes_read != bytes_req) {
-return Status::InternalError("failed to read (bytes read: {},
bytes req: {})",
- *bytes_read, bytes_req);
-}
-_s3_stats.total_bytes_read += bytes_req;
-s3_bytes_read_total << bytes_req;
-s3_bytes_per_read << bytes_req;
-DorisMetrics::instance()->s3_bytes_read_total->increment(bytes_req);
-if (retry_count > 0) {
-LOG(INFO) << fmt::format("read s3 file {} succeed after {} times
with {} ms sleeping",
- _path.native(), retry_count,
total_sleep_time);
-}
-return Status::OK();
+// clang-format off
+auto resp = client->get_object( { .bucket = _bucket, .key = _key, },
Review Comment:
Considering create a new client with `CustomRetryStrategy` which can carry
enough context
such as number retires and duration of all retires.
##
be/src/util/s3_util.cpp:
##
@@ -73,6 +73,20 @@ bool to_int(std::string_view str, int& res) {
return ec == std::errc {};
}
+class CustomRetryStrategy final : public Aws::Client::DefaultRetryStrategy {
+public:
+CustomRetryStrategy(int maxRetries) : DefaultRetryStrategy(maxRetries) {}
+
+bool ShouldRetry(const Aws::Client::AWSError&
error,
+ long attemptedRetries) const override {
+if (attemptedRetries < m_maxRetries &&
+error.GetResponseCode() ==
Aws::Http::HttpResponseCode::TOO_MANY_REQUESTS) {
+return true;
Review Comment:
We should record something here if we are doing retry.
e.g. a retry counter or something useful for us to observe the status of
requesting the object storage service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](fix) replace hardcode s3BucketName [doris]
hello-stephen commented on PR #37739: URL: https://github.com/apache/doris/pull/37739#issuecomment-2226946070 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Vault) Support alter s3 storage vault using http interface [doris]
gavinchou commented on code in PR #37537:
URL: https://github.com/apache/doris/pull/37537#discussion_r1676840768
##
cloud/src/meta-service/meta_service_resource.cpp:
##
@@ -509,6 +509,80 @@ static void set_default_vault_log_helper(const
InstanceInfoPB& instance,
LOG(INFO) << vault_msg;
}
+static int alter_storage_vault(InstanceInfoPB& instance,
std::unique_ptr txn,
+ const StorageVaultPB& vault, MetaServiceCode&
code,
+ std::string& msg) {
+if (!vault.has_obj_info()) {
+code = MetaServiceCode::INVALID_ARGUMENT;
+std::stringstream ss;
+ss << "Only s3 vault can be altered";
Review Comment:
Why not HDFS?
##
cloud/src/meta-service/meta_service_resource.cpp:
##
@@ -509,6 +509,80 @@ static void set_default_vault_log_helper(const
InstanceInfoPB& instance,
LOG(INFO) << vault_msg;
}
+static int alter_storage_vault(InstanceInfoPB& instance,
std::unique_ptr txn,
Review Comment:
Add an UT to test it.
And refer to `MetaServiceImpl::alter_obj_store_info` and
`MetaServiceImpl::alter_instance` to abstract the behavior and make as flexible
as possible.
##
cloud/src/meta-service/meta_service_resource.cpp:
##
@@ -575,22 +649,20 @@ void
MetaServiceImpl::alter_obj_store_info(google::protobuf::RpcController* cont
}
break;
}
-case AlterObjStoreInfoRequest::ADD_BUILT_IN_VAULT: {
Review Comment:
Do not delete it, it will be used in the future.
##
cloud/src/meta-service/meta_service_http.cpp:
##
@@ -449,6 +449,7 @@ void
MetaServiceImpl::http(::google::protobuf::RpcController* controller,
{"show_storage_vaults", process_get_obj_store_info},
{"add_hdfs_vault", process_alter_obj_store_info},
{"add_s3_vault", process_alter_obj_store_info},
+{"alter_s3_vault", process_alter_obj_store_info},
Review Comment:
It should be also a RPC, which can be called by FE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
doris-robot commented on PR #37745: URL: https://github.com/apache/doris/pull/37745#issuecomment-2226957616 TPC-H: Total hot run time: 39894 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit df1cecfb40914720470069186696f0964dfe7b0c, data reload: false -- Round 1 -- q1 17695 440143214321 q2 2022189 190 189 q3 10497 121010781078 q4 10222 832 741 741 q5 7568273326912691 q6 230 141 139 139 q7 968 591 589 589 q8 9227211321012101 q9 8644658365526552 q10 8836383537893789 q11 458 241 233 233 q12 397 223 221 221 q13 18868 294729902947 q14 273 238 229 229 q15 539 498 493 493 q16 511 370 372 370 q17 979 647 711 647 q18 8158753674857485 q19 7918145613721372 q20 700 326 325 325 q21 4854310631503106 q22 340 292 276 276 Total cold run time: 119904 ms Total hot run time: 39894 ms - Round 2, with runtime_filter_mode=off - q1 4418426541834183 q2 360 279 263 263 q3 3136293529232923 q4 1943171716811681 q5 5510556454475447 q6 220 134 141 134 q7 2250182618361826 q8 3304348234263426 q9 8736886287788778 q10 4132380637773777 q11 585 500 507 500 q12 818 634 639 634 q13 17064 317431643164 q14 323 288 289 288 q15 520 477 497 477 q16 498 425 433 425 q17 1811150215021502 q18 8074799578487848 q19 1757162815011501 q20 2156186518531853 q21 5111466547724665 q22 623 498 499 498 Total cold run time: 73349 ms Total hot run time: 55793 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Recycler) Add rate limit for all obj storage client implementation [doris]
ByteYue commented on PR #37663: URL: https://github.com/apache/doris/pull/37663#issuecomment-2226958270 > BE may use the same rate limiter? Yes. I'll add it into BE later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Vault) Support alter s3 storage vault using http interface [doris]
ByteYue commented on code in PR #37537:
URL: https://github.com/apache/doris/pull/37537#discussion_r1676845267
##
cloud/src/meta-service/meta_service_resource.cpp:
##
@@ -509,6 +509,80 @@ static void set_default_vault_log_helper(const
InstanceInfoPB& instance,
LOG(INFO) << vault_msg;
}
+static int alter_storage_vault(InstanceInfoPB& instance,
std::unique_ptr txn,
+ const StorageVaultPB& vault, MetaServiceCode&
code,
+ std::string& msg) {
+if (!vault.has_obj_info()) {
+code = MetaServiceCode::INVALID_ARGUMENT;
+std::stringstream ss;
+ss << "Only s3 vault can be altered";
Review Comment:
Currently i'm not sure which property of hdfs vault can be altered. As the
pr title says, this pr is for `S3 Vault`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Vault) Support alter s3 storage vault using http interface [doris]
ByteYue commented on code in PR #37537:
URL: https://github.com/apache/doris/pull/37537#discussion_r1676845357
##
cloud/src/meta-service/meta_service_http.cpp:
##
@@ -449,6 +449,7 @@ void
MetaServiceImpl::http(::google::protobuf::RpcController* controller,
{"show_storage_vaults", process_get_obj_store_info},
{"add_hdfs_vault", process_alter_obj_store_info},
{"add_s3_vault", process_alter_obj_store_info},
+{"alter_s3_vault", process_alter_obj_store_info},
Review Comment:
The rpc procedure is included in #37606.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Vault) Support alter s3 storage vault using http interface [doris]
ByteYue commented on code in PR #37537:
URL: https://github.com/apache/doris/pull/37537#discussion_r1676845509
##
cloud/src/meta-service/meta_service_resource.cpp:
##
@@ -575,22 +649,20 @@ void
MetaServiceImpl::alter_obj_store_info(google::protobuf::RpcController* cont
}
break;
}
-case AlterObjStoreInfoRequest::ADD_BUILT_IN_VAULT: {
Review Comment:
It seems useless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Vault) Support alter s3 storage vault using http interface [doris]
ByteYue commented on code in PR #37537: URL: https://github.com/apache/doris/pull/37537#discussion_r1676845465 ## cloud/src/meta-service/meta_service_resource.cpp: ## @@ -509,6 +509,80 @@ static void set_default_vault_log_helper(const InstanceInfoPB& instance, LOG(INFO) << vault_msg; } +static int alter_storage_vault(InstanceInfoPB& instance, std::unique_ptr txn, Review Comment: Ut and regression cases would be involved in #37606. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](function) remove folding constant implementation of convert_tz [doris]
zclllyybb opened a new pull request, #37746: URL: https://github.com/apache/doris/pull/37746 ## Proposed changes Issue Number: close #xxx Because what we support for timezone format is not same with mysql or Java. function like `convert_tz` will generate different result when meet like `Asia/SHANGHAI` or `UTC+8`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](function) remove folding constant implementation of convert_tz [doris]
zclllyybb commented on PR #37746: URL: https://github.com/apache/doris/pull/37746#issuecomment-2226960427 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
doris-robot commented on PR #37745: URL: https://github.com/apache/doris/pull/37745#issuecomment-2226960426 TPC-DS: Total hot run time: 172687 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit df1cecfb40914720470069186696f0964dfe7b0c, data reload: false query1 910 375 364 364 query2 6682179218081792 query3 6644208 217 208 query4 28309 17600 17288 17288 query5 3655501 480 480 query6 270 190 161 161 query7 4584291 288 288 query8 247 204 201 201 query9 8534240123762376 query10 455 269 264 264 query11 10589 10172 10032 10032 query12 114 85 80 80 query13 1624365 350 350 query14 9781684075036840 query15 228 161 166 161 query16 7406327 308 308 query17 1366537 517 517 query18 1946273 268 268 query19 195 142 147 142 query20 93 85 78 78 query21 206 122 122 122 query22 4433409841084098 query23 34052 33740 33553 33553 query24 11292 293327702770 query25 676 406 408 406 query26 1187151 148 148 query27 3025280 284 280 query28 7342206220472047 query29 926 673 637 637 query30 257 157 160 157 query31 985 783 798 783 query32 93 55 61 55 query33 746 296 292 292 query34 1061481 496 481 query35 693 563 577 563 query36 1153981 982 981 query37 158 86 83 83 query38 2957296828922892 query39 885 858 870 858 query40 207 125 120 120 query41 46 46 47 46 query42 112 99 98 98 query43 504 456 437 437 query44 1196707 707 707 query45 201 158 161 158 query46 1088725 726 725 query47 1879176418141764 query48 369 281 291 281 query49 847 403 435 403 query50 768 380 378 378 query51 6982689267266726 query52 109 90 91 90 query53 364 287 290 287 query54 853 458 439 439 query55 76 72 74 72 query56 288 266 270 266 query57 1127105710631057 query58 239 265 246 246 query59 2823251026082510 query60 322 274 276 274 query61 97 96 92 92 query62 799 625 633 625 query63 304 280 278 278 query64 9307225534042255 query65 3144307430883074 query66 813 326 328 326 query67 15380 15256 15235 15235 query68 4503529 528 528 query69 476 333 313 313 query70 1171113610751075 query71 405 266 273 266 query72 7030528458475284 query73 760 326 326 326 query74 5983551155625511 query75 3401271626512651 query76 2224997 892 892 query77 422 283 294 283 query78 9583982191739173 query79 2782517 507 507 query80 1920470 458 458 query81 576 219 224 219 query82 760 129 129 129 query83 322 164 172 164 query84 272 87 88 87 query85 1520385 295 295 query86 330 327 307 307 query87 3268308030823080 query88 4155233323322332 query89 461 382 380 380 query90 1706191 189 189 query91 127 95 101 95 query92 55 50 49 49 query93 3291501 495 495 query94 801 216 216 216 query95 416 311 316 311 query96 617 276 267 267 query97 3211301530103010 query98 216 223 190 190 query99 1686126712321232 Total cold run time: 280453 ms Total hot run time: 172687 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Fix](function) remove folding constant implementation of convert_tz [doris]
doris-robot commented on PR #37746: URL: https://github.com/apache/doris/pull/37746#issuecomment-2226960459 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix]fix be core when migration tablet to other disk [doris]
doris-robot commented on PR #37712: URL: https://github.com/apache/doris/pull/37712#issuecomment-2226961198 TPC-H: Total hot run time: 39629 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a29b688dd3e55ee8d5426bf20eeb36a8ea01b2cc, data reload: false -- Round 1 -- q1 17618 437742664266 q2 2005190 190 190 q3 10455 120210651065 q4 10187 929 738 738 q5 7529273026152615 q6 225 134 139 134 q7 962 600 585 585 q8 9293208820722072 q9 8878657265466546 q10 8821385637653765 q11 464 230 234 230 q12 424 223 216 216 q13 17906 294429992944 q14 271 233 224 224 q15 529 477 467 467 q16 509 375 368 368 q17 973 579 622 579 q18 8130758274547454 q19 6321140314191403 q20 671 320 314 314 q21 4939319531713171 q22 346 283 287 283 Total cold run time: 117456 ms Total hot run time: 39629 ms - Round 2, with runtime_filter_mode=off - q1 4439425142464246 q2 363 262 258 258 q3 3021291229192912 q4 2035173217691732 q5 5656557054965496 q6 230 135 129 129 q7 2250193118361836 q8 3240340534383405 q9 8717885888018801 q10 4188372938273729 q11 573 498 495 495 q12 799 656 651 651 q13 16014 310832253108 q14 320 292 287 287 q15 530 486 488 486 q16 519 437 448 437 q17 1805150315091503 q18 8237790378117811 q19 1736155817111558 q20 2172189218791879 q21 8872481248504812 q22 588 515 499 499 Total cold run time: 76304 ms Total hot run time: 56070 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
doris-robot commented on PR #37677: URL: https://github.com/apache/doris/pull/37677#issuecomment-2226961384 TPC-H: Total hot run time: 39807 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8cce6d206c2f9e385b0d28aa776b5c9e593fd1e9, data reload: false -- Round 1 -- q1 17644 434243134313 q2 2031191 186 186 q3 10510 119011331133 q4 10197 843 721 721 q5 7547267027002670 q6 219 135 137 135 q7 958 594 610 594 q8 9223206420902064 q9 8714655165546551 q10 8858378337863783 q11 459 239 234 234 q12 477 224 224 224 q13 17757 298029552955 q14 291 235 239 235 q15 529 480 498 480 q16 523 383 373 373 q17 962 674 634 634 q18 8071748473637363 q19 6569151214061406 q20 697 330 320 320 q21 4883315032803150 q22 347 295 283 283 Total cold run time: 117466 ms Total hot run time: 39807 ms - Round 2, with runtime_filter_mode=off - q1 4381425342574253 q2 377 272 267 267 q3 3022292429362924 q4 2001165717281657 q5 5651550555145505 q6 227 133 135 133 q7 2239191918581858 q8 3268344434383438 q9 8787878788028787 q10 4077382838283828 q11 578 493 508 493 q12 809 633 647 633 q13 15815 317232083172 q14 318 278 309 278 q15 540 476 502 476 q16 488 439 427 427 q17 1848150615001500 q18 8084796678227822 q19 1746158615341534 q20 2166186918621862 q21 9658459348374593 q22 554 555 503 503 Total cold run time: 76634 ms Total hot run time: 55943 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
doris-robot commented on PR #37745: URL: https://github.com/apache/doris/pull/37745#issuecomment-2226961666 ClickBench: Total hot run time: 30.02 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit df1cecfb40914720470069186696f0964dfe7b0c, data reload: false query1 0.040.030.02 query2 0.080.030.04 query3 0.220.050.05 query4 1.670.060.06 query5 0.500.490.49 query6 1.130.720.73 query7 0.010.010.02 query8 0.050.040.04 query9 0.550.490.49 query10 0.530.550.55 query11 0.160.110.11 query12 0.150.130.13 query13 0.580.590.58 query14 0.760.770.79 query15 0.840.800.81 query16 0.370.360.35 query17 1.030.981.02 query18 0.220.210.22 query19 1.781.721.74 query20 0.010.000.02 query21 15.31 0.720.66 query22 4.567.731.51 query23 18.24 1.351.27 query24 1.870.270.21 query25 0.150.080.08 query26 0.290.210.21 query27 0.460.230.22 query28 13.33 1.021.00 query29 12.60 3.273.25 query30 0.250.060.06 query31 2.860.390.38 query32 3.280.460.46 query33 2.902.882.92 query34 16.99 4.284.31 query35 4.404.394.42 query36 0.650.470.49 query37 0.190.160.15 query38 0.160.150.15 query39 0.040.030.04 query40 0.150.120.12 query41 0.090.050.05 query42 0.060.050.06 query43 0.050.040.03 Total cold run time: 109.56 s Total hot run time: 30.02 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix]fix be core when migration tablet to other disk [doris]
doris-robot commented on PR #37712: URL: https://github.com/apache/doris/pull/37712#issuecomment-2226967998 TPC-DS: Total hot run time: 173346 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a29b688dd3e55ee8d5426bf20eeb36a8ea01b2cc, data reload: false query1 914 378 366 366 query2 6453190918911891 query3 6636206 213 206 query4 28620 17591 17329 17329 query5 3611503 482 482 query6 265 174 167 167 query7 4580293 285 285 query8 237 192 203 192 query9 8668240623892389 query10 444 305 279 279 query11 11967 10104 10048 10048 query12 117 93 82 82 query13 1632386 372 372 query14 10208 782777287728 query15 210 171 167 167 query16 7691320 313 313 query17 1807567 524 524 query18 1909287 291 287 query19 210 156 157 156 query20 91 83 82 82 query21 218 126 127 126 query22 4244405939123912 query23 34093 33882 33654 33654 query24 10934 292029302920 query25 604 427 377 377 query26 782 153 159 153 query27 2306278 283 278 query28 6218205820582058 query29 886 685 625 625 query30 261 153 155 153 query31 979 762 751 751 query32 94 53 56 53 query33 767 329 292 292 query34 961 505 510 505 query35 682 572 593 572 query36 11541004971 971 query37 144 94 81 81 query38 2969283128772831 query39 926 806 801 801 query40 208 120 121 120 query41 51 44 44 44 query42 119 95 104 95 query43 492 467 458 458 query44 1242750 753 750 query45 207 161 158 158 query46 1085727 724 724 query47 1840173217401732 query48 375 297 294 294 query49 841 418 424 418 query50 794 397 394 394 query51 6987674067756740 query52 108 92 96 92 query53 357 292 293 292 query54 889 453 449 449 query55 74 73 75 73 query56 307 267 275 267 query57 1147105510551055 query58 242 253 245 245 query59 2913261125752575 query60 302 283 280 280 query61 101 125 94 94 query62 786 653 627 627 query63 314 292 296 292 query64 9301221816741674 query65 3185307630893076 query66 756 323 328 323 query67 15344 14993 14992 14992 query68 6102542 542 542 query69 741 463 357 357 query70 1189117310581058 query71 472 293 281 281 query72 7272531556855315 query73 801 328 325 325 query74 5970554054845484 query75 3824267227182672 query76 4014940 953 940 query77 671 314 304 304 query78 9675915489828982 query79 2196529 523 523 query80 1874485 482 482 query81 589 226 220 220 query82 1396136 134 134 query83 268 172 179 172 query84 278 86 88 86 query85 1297306 298 298 query86 464 323 325 323 query87 3242309032083090 query88 4053252524612461 query89 483 391 385 385 query90 2211196 197 196 query91 136 104 101 101 query92 60 53 52 52 query93 2309492 502 492 query94 1048212 210 210 query95 409 322 324 322 query96 627 278 274 274 query97 3216303730313031 query98 230 196 197 196 query99 1688126512471247 Total cold run time: 284664 ms Total hot run time: 173346 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
doris-robot commented on PR #37677: URL: https://github.com/apache/doris/pull/37677#issuecomment-2226968206 TPC-DS: Total hot run time: 174002 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8cce6d206c2f9e385b0d28aa776b5c9e593fd1e9, data reload: false query1 930 374 373 373 query2 6439197118541854 query3 6641205 213 205 query4 28334 17457 17466 17457 query5 3598498 472 472 query6 255 166 161 161 query7 4587294 279 279 query8 240 189 201 189 query9 8567237023552355 query10 450 275 274 274 query11 10592 10041 10066 10041 query12 124 88 85 85 query13 1642389 372 372 query14 10306 845780178017 query15 223 162 169 162 query16 7262321 311 311 query17 1795553 541 541 query18 1886287 286 286 query19 203 152 154 152 query20 91 86 88 86 query21 213 129 131 129 query22 4228397739803977 query23 34027 33870 33662 33662 query24 10647 291129022902 query25 603 417 423 417 query26 693 158 150 150 query27 2181276 286 276 query28 5627206020542054 query29 892 655 661 655 query30 252 162 158 158 query31 955 784 767 767 query32 101 59 59 59 query33 663 323 313 313 query34 874 494 502 494 query35 712 596 597 596 query36 1153978 993 978 query37 165 87 90 87 query38 2981287528622862 query39 951 841 837 837 query40 208 127 123 123 query41 51 48 49 48 query42 106 98 101 98 query43 539 476 466 466 query44 1072716 742 716 query45 197 163 163 163 query46 1074742 717 717 query47 1874175917691759 query48 360 295 297 295 query49 828 407 413 407 query50 777 387 382 382 query51 6859670467426704 query52 104 97 91 91 query53 357 290 282 282 query54 954 453 461 453 query55 71 72 73 72 query56 278 266 265 265 query57 1102106910681068 query58 248 239 252 239 query59 2987262627682626 query60 313 280 268 268 query61 98 96 117 96 query62 793 636 650 636 query63 324 281 283 281 query64 9119222816521652 query65 3218310830973097 query66 714 331 343 331 query67 15849 14977 14979 14977 query68 8456538 534 534 query69 729 499 367 367 query70 1443114111291129 query71 519 280 276 276 query72 8371558555165516 query73 2204324 318 318 query74 5872549155335491 query75 4741267427072674 query76 4696968 856 856 query77 795 318 306 306 query78 10779 908489468946 query79 11765 513 523 513 query80 985 470 467 467 query81 570 228 224 224 query82 281 130 133 130 query83 313 168 162 162 query84 280 84 88 84 query85 899 318 348 318 query86 363 304 302 302 query87 3361309531493095 query88 5102246724502450 query89 518 386 384 384 query90 2077194 196 194 query91 133 100 98 98 query92 65 48 48 48 query93 6429501 506 501 query94 1384214 213 213 query95 405 321 317 317 query96 621 281 273 273 query97 3242299630292996 query98 216 199 192 192 query99 1504127112791271 Total cold run time: 301675 ms Total hot run time: 174002 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](Recycler) Retry object storage request when meeting 429 http error code [doris]
ByteYue commented on PR #37680: URL: https://github.com/apache/doris/pull/37680#issuecomment-2226968289 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](cloud) Accelerate creating table by batching RPC In cloud [doris]
gavinchou commented on PR #36786: URL: https://github.com/apache/doris/pull/36786#issuecomment-2226968462 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Recycler) Retry object storage request when meeting 429 http error code [doris]
github-actions[bot] commented on PR #37680: URL: https://github.com/apache/doris/pull/37680#issuecomment-2226968830 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
ByteYue commented on PR #37677: URL: https://github.com/apache/doris/pull/37677#issuecomment-2226968922 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](cloud) Accelerate creating table by batching RPC In cloud [doris]
github-actions[bot] commented on PR #36786: URL: https://github.com/apache/doris/pull/36786#issuecomment-2226969465 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](cloud) Accelerate creating table by batching RPC In cloud [doris]
github-actions[bot] commented on PR #36786: URL: https://github.com/apache/doris/pull/36786#issuecomment-2226969534 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix]fix be core when migration tablet to other disk [doris]
doris-robot commented on PR #37712: URL: https://github.com/apache/doris/pull/37712#issuecomment-2226969966 ClickBench: Total hot run time: 30.18 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a29b688dd3e55ee8d5426bf20eeb36a8ea01b2cc, data reload: false query1 0.040.030.03 query2 0.080.040.03 query3 0.220.050.05 query4 1.680.080.08 query5 0.510.500.48 query6 1.130.720.73 query7 0.020.020.01 query8 0.050.040.04 query9 0.560.500.50 query10 0.540.540.54 query11 0.150.110.11 query12 0.150.110.12 query13 0.580.580.58 query14 0.760.770.83 query15 0.850.820.82 query16 0.370.360.36 query17 0.980.940.97 query18 0.220.220.21 query19 1.811.691.65 query20 0.010.010.02 query21 15.43 0.790.67 query22 4.607.191.49 query23 18.27 1.341.35 query24 2.160.230.24 query25 0.170.080.09 query26 0.320.210.21 query27 0.460.240.22 query28 13.19 1.010.99 query29 12.62 3.353.30 query30 0.250.060.05 query31 2.860.370.38 query32 3.310.490.48 query33 2.852.972.87 query34 17.07 4.334.30 query35 4.434.484.49 query36 0.660.490.48 query37 0.190.160.15 query38 0.150.150.15 query39 0.040.030.04 query40 0.160.120.13 query41 0.090.040.05 query42 0.070.050.05 query43 0.040.040.04 Total cold run time: 110.1 s Total hot run time: 30.18 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Recycler) Retry object storage request when meeting 429 http error code [doris]
github-actions[bot] commented on PR #37680: URL: https://github.com/apache/doris/pull/37680#issuecomment-2226970954 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
github-actions[bot] commented on PR #37745: URL: https://github.com/apache/doris/pull/37745#issuecomment-2226975180 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](inverted index) fix fast execute for not_in expr [doris]
github-actions[bot] commented on PR #37745: URL: https://github.com/apache/doris/pull/37745#issuecomment-2226975176 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](index) rollback in comopund index opt [doris]
dataroaring merged PR #37742: URL: https://github.com/apache/doris/pull/37742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-3.0 updated: [fix](index) rollback in comopund index opt (#37742)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new d728555f646 [fix](index) rollback in comopund index opt (#37742)
d728555f646 is described below
commit d728555f6468e41362dcc8c2a630ba769783ff85
Author: Kang
AuthorDate: Sun Jul 14 00:10:43 2024 +0800
[fix](index) rollback in comopund index opt (#37742)
## Proposed changes
rollback opt in #34134
---
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 21 +++--
1 file changed, 3 insertions(+), 18 deletions(-)
diff --git a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
index a8a9758afcd..a1cb81de2cc 100644
--- a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
+++ b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
@@ -814,16 +814,9 @@ Status
SegmentIterator::_execute_predicates_except_leafnode_of_andnode(
} else if (_is_literal_node(node_type)) {
auto v_literal_expr =
std::dynamic_pointer_cast(expr);
_column_predicate_info->query_values.insert(v_literal_expr->value());
-} else if (node_type == TExprNodeType::BINARY_PRED || node_type ==
TExprNodeType::MATCH_PRED ||
- node_type == TExprNodeType::IN_PRED) {
+} else if (node_type == TExprNodeType::BINARY_PRED || node_type ==
TExprNodeType::MATCH_PRED) {
if (node_type == TExprNodeType::MATCH_PRED) {
_column_predicate_info->query_op = "match";
-} else if (node_type == TExprNodeType::IN_PRED) {
-if (expr->op() == TExprOpcode::type::FILTER_IN) {
-_column_predicate_info->query_op = "in";
-} else {
-_column_predicate_info->query_op = "not_in";
-}
} else {
_column_predicate_info->query_op = expr->fn().name.function_name;
}
@@ -961,9 +954,7 @@ Status
SegmentIterator::_apply_index_except_leafnode_of_andnode() {
bool is_support = pred_type == PredicateType::EQ || pred_type ==
PredicateType::NE ||
pred_type == PredicateType::LT || pred_type ==
PredicateType::LE ||
pred_type == PredicateType::GT || pred_type ==
PredicateType::GE ||
- pred_type == PredicateType::MATCH ||
- pred_type == PredicateType::IN_LIST ||
- pred_type == PredicateType::NOT_IN_LIST;
+ pred_type == PredicateType::MATCH;
if (!is_support) {
_need_read_data_indices[column_id] = true;
continue;
@@ -1118,7 +1109,7 @@ Status
SegmentIterator::_apply_inverted_index_on_column_predicate(
}
auto pred_type = pred->type();
-if (pred_type == PredicateType::MATCH || pred_type ==
PredicateType::IN_LIST) {
+if (pred_type == PredicateType::MATCH) {
std::string pred_result_sign = _gen_predicate_result_sign(pred);
_rowid_result_for_index.emplace(pred_result_sign,
std::make_pair(false, _row_bitmap));
}
@@ -2813,12 +2804,6 @@ void
SegmentIterator::_calculate_pred_in_remaining_conjunct_root(
} else {
if (node_type == TExprNodeType::MATCH_PRED) {
_column_predicate_info->query_op = "match";
-} else if (node_type == TExprNodeType::IN_PRED) {
-if (expr->op() == TExprOpcode::type::FILTER_IN) {
-_column_predicate_info->query_op = "in";
-} else {
-_column_predicate_info->query_op = "not_in";
-}
} else if (node_type != TExprNodeType::COMPOUND_PRED) {
_column_predicate_info->query_op = expr->fn().name.function_name;
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](inverted index) Error parsing NULL_LITERAL [doris]
zzzxl1993 opened a new pull request, #37747: URL: https://github.com/apache/doris/pull/37747 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](inverted index) Error parsing NULL_LITERAL [doris]
doris-robot commented on PR #37747: URL: https://github.com/apache/doris/pull/37747#issuecomment-2226977266 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](compaction) optimizing memory usage for compaction (#37099) [doris]
luwei16 commented on PR #37486: URL: https://github.com/apache/doris/pull/37486#issuecomment-2226977277 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](index) rollback in compound opt [doris]
xiaokang opened a new pull request, #37748: URL: https://github.com/apache/doris/pull/37748 rollback opt in https://github.com/apache/doris/pull/34134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](index) rollback in compound opt [doris]
doris-robot commented on PR #37748: URL: https://github.com/apache/doris/pull/37748#issuecomment-2226977453 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](function) remove folding constant implementation of convert_tz [doris]
doris-robot commented on PR #37746: URL: https://github.com/apache/doris/pull/37746#issuecomment-2226977792 TPC-H: Total hot run time: 39784 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit dc5b7e662b5445b00b51eadb0dccf98f5ca19cfe, data reload: false -- Round 1 -- q1 17635 427142584258 q2 2018187 185 185 q3 10457 119810741074 q4 10183 742 725 725 q5 7547269026302630 q6 218 135 134 134 q7 953 594 590 590 q8 9212205120572051 q9 8714652466126524 q10 8814377137683768 q11 460 235 237 235 q12 470 222 220 220 q13 17771 297629712971 q14 268 232 244 232 q15 521 485 492 485 q16 495 374 377 374 q17 955 673 690 673 q18 7955762673627362 q19 8269153114731473 q20 697 316 328 316 q21 4987322832313228 q22 348 291 276 276 Total cold run time: 118947 ms Total hot run time: 39784 ms - Round 2, with runtime_filter_mode=off - q1 4438425342654253 q2 360 271 263 263 q3 3070291229382912 q4 1934171018071710 q5 5585549154385438 q6 229 133 130 130 q7 2238185918211821 q8 3294342634283426 q9 8763878488158784 q10 4128376737753767 q11 587 497 495 495 q12 795 645 655 645 q13 15941 314631553146 q14 314 290 289 289 q15 520 497 485 485 q16 507 426 436 426 q17 1800154115191519 q18 7986800478437843 q19 1818169116601660 q20 2245186818771868 q21 5130488947444744 q22 619 511 522 511 Total cold run time: 72301 ms Total hot run time: 56135 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
doris-robot commented on PR #37677: URL: https://github.com/apache/doris/pull/37677#issuecomment-2226979942 TPC-H: Total hot run time: 39789 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 19cc325f7f0318d084f9415c43e4792080d9569c, data reload: false -- Round 1 -- q1 17863 445143734373 q2 2479198 193 193 q3 12738 119811281128 q4 10487 759 803 759 q5 7667275927042704 q6 216 138 142 138 q7 960 601 625 601 q8 9214205520912055 q9 8536652864556455 q10 8645378037503750 q11 480 239 247 239 q12 408 233 232 232 q13 17823 297029702970 q14 286 241 237 237 q15 536 478 483 478 q16 492 412 389 389 q17 971 634 712 634 q18 8235750872987298 q19 7724147913001300 q20 696 325 330 325 q21 4883323739453237 q22 364 294 297 294 Total cold run time: 121703 ms Total hot run time: 39789 ms - Round 2, with runtime_filter_mode=off - q1 4513429842454245 q2 385 277 259 259 q3 3041283528152815 q4 1890159216351592 q5 5286535052955295 q6 222 134 134 134 q7 2145176517361736 q8 3185331132643264 q9 8272831783628317 q10 3827369837113698 q11 571 497 515 497 q12 775 603 620 603 q13 16317 294330162943 q14 300 286 263 263 q15 513 460 483 460 q16 469 411 425 411 q17 1750145914511451 q18 7567750574087408 q19 3517163415081508 q20 2069179117811781 q21 4827459946104599 q22 548 493 511 493 Total cold run time: 71989 ms Total hot run time: 53772 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](inverted index) Error parsing NULL_LITERAL [doris]
github-actions[bot] commented on PR #37747: URL: https://github.com/apache/doris/pull/37747#issuecomment-2226983027 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](index) rollback in compound opt [doris]
github-actions[bot] commented on PR #37748: URL: https://github.com/apache/doris/pull/37748#issuecomment-2226983151 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](index) rollback in compound opt [doris]
github-actions[bot] commented on PR #37748: URL: https://github.com/apache/doris/pull/37748#issuecomment-2226985525 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](function) remove folding constant implementation of convert_tz [doris]
doris-robot commented on PR #37746: URL: https://github.com/apache/doris/pull/37746#issuecomment-2226989055 TPC-DS: Total hot run time: 172531 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit dc5b7e662b5445b00b51eadb0dccf98f5ca19cfe, data reload: false query1 914 371 361 361 query2 6917184919041849 query3 6647213 220 213 query4 23489 17490 17350 17350 query5 3641479 474 474 query6 279 168 168 168 query7 4569287 281 281 query8 236 188 199 188 query9 8503234523372337 query10 442 296 267 267 query11 11448 999810114 9998 query12 109 82 84 82 query13 1634364 349 349 query14 10111 740866486648 query15 216 167 165 165 query16 7480310 327 310 query17 1346543 523 523 query18 1922271 265 265 query19 194 147 144 144 query20 85 81 83 81 query21 210 133 131 131 query22 4282398239563956 query23 34100 33702 33617 33617 query24 10957 291829162916 query25 612 380 383 380 query26 717 152 144 144 query27 2313280 274 274 query28 5708202820472028 query29 897 644 631 631 query30 259 150 147 147 query31 1035768 753 753 query32 106 53 57 53 query33 661 299 297 297 query34 901 488 496 488 query35 697 586 575 575 query36 1159979 954 954 query37 140 93 90 90 query38 2995281028362810 query39 912 852 842 842 query40 222 132 120 120 query41 55 46 48 46 query42 114 104 113 104 query43 547 475 477 475 query44 1100725 734 725 query45 192 165 165 165 query46 1083750 751 750 query47 1849180017911791 query48 373 282 298 282 query49 837 406 412 406 query50 773 383 399 383 query51 6779683366826682 query52 108 94 93 93 query53 359 296 285 285 query54 900 437 438 437 query55 73 72 72 72 query56 281 263 270 263 query57 1133107910511051 query58 234 240 250 240 query59 2989269327242693 query60 290 276 282 276 query61 98 96 132 96 query62 773 661 644 644 query63 308 285 287 285 query64 9126223074232230 query65 3137311131263111 query66 766 331 327 327 query67 15319 14895 14906 14895 query68 4759522 520 520 query69 669 497 328 328 query70 1183110611601106 query71 426 280 274 274 query72 8046543055445430 query73 745 319 320 319 query74 6031547454955474 query75 3391268126882681 query76 29021001912 912 query77 625 305 302 302 query78 10891 968689668966 query79 4321521 511 511 query80 2059464 467 464 query81 621 218 228 218 query82 1030145 135 135 query83 318 172 162 162 query84 271 91 86 86 query85 1395318 289 289 query86 473 318 297 297 query87 3304308730883087 query88 4627245524602455 query89 490 382 382 382 query90 1769188 186 186 query91 129 100 98 98 query92 64 55 50 50 query93 4734483 486 483 query94 1181220 207 207 query95 401 316 314 314 query96 634 279 268 268 query97 3273299429942994 query98 217 205 195 195 query99 1697127212891272 Total cold run time: 281337 ms Total hot run time: 172531 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
doris-robot commented on PR #37677: URL: https://github.com/apache/doris/pull/37677#issuecomment-2226989625 TPC-DS: Total hot run time: 171565 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 19cc325f7f0318d084f9415c43e4792080d9569c, data reload: false query1 909 369 361 361 query2 6502190818031803 query3 6678208 225 208 query4 28446 17511 17106 17106 query5 4231490 514 490 query6 265 177 173 173 query7 4592297 289 289 query8 248 202 201 201 query9 8502237323542354 query10 439 302 268 268 query11 11969 10022 10210 10022 query12 129 84 81 81 query13 1637357 372 357 query14 10180 739778357397 query15 228 170 172 170 query16 7624315 307 307 query17 1836556 513 513 query18 1564273 272 272 query19 194 148 157 148 query20 92 82 78 78 query21 207 137 127 127 query22 4284398040013980 query23 33922 33046 33668 33046 query24 11028 293029242924 query25 628 370 394 370 query26 1351151 148 148 query27 2862262 265 262 query28 7634200819851985 query29 929 654 650 650 query30 290 151 148 148 query31 959 740 757 740 query32 97 52 55 52 query33 775 288 288 288 query34 941 494 490 490 query35 681 587 569 569 query36 1102937 920 920 query37 151 78 80 78 query38 2885275127202720 query39 844 785 790 785 query40 276 119 120 119 query41 46 43 48 43 query42 118 101 100 100 query43 510 482 463 463 query44 1247748 747 747 query45 200 161 159 159 query46 1066714 731 714 query47 1867179217731773 query48 360 287 294 287 query49 1076409 441 409 query50 778 394 407 394 query51 6823676267916762 query52 100 95 93 93 query53 364 292 283 283 query54 872 447 444 444 query55 73 73 75 73 query56 301 275 269 269 query57 1137105910331033 query58 256 239 242 239 query59 2970268526372637 query60 315 269 278 269 query61 97 93 94 93 query62 830 664 672 664 query63 316 291 289 289 query64 9643220616461646 query65 3196309731043097 query66 1068341 339 339 query67 15493 14864 14793 14793 query68 4615532 524 524 query69 481 325 331 325 query70 1198113611421136 query71 393 282 277 277 query72 7115520859665208 query73 738 321 322 321 query74 5962554054825482 query75 3408264726992647 query76 2537964 901 901 query77 500 304 309 304 query78 9584911490219021 query79 2224527 518 518 query80 2393480 482 480 query81 580 222 224 222 query82 799 138 131 131 query83 298 171 179 171 query84 276 96 94 94 query85 2208348 305 305 query86 488 323 302 302 query87 3297311631383116 query88 4007246424392439 query89 476 378 378 378 query90 1835200 197 197 query91 130 103 98 98 query92 66 50 54 50 query93 2457517 510 510 query94 1249211 278 211 query95 405 318 320 318 query96 588 287 275 275 query97 3248302730783027 query98 229 199 196 196 query99 1659127012751270 Total cold run time: 284676 ms Total hot run time: 171565 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Fix](function) remove folding constant implementation of convert_tz [doris]
doris-robot commented on PR #37746: URL: https://github.com/apache/doris/pull/37746#issuecomment-2226990459 ClickBench: Total hot run time: 30.84 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit dc5b7e662b5445b00b51eadb0dccf98f5ca19cfe, data reload: false query1 0.050.030.03 query2 0.070.040.04 query3 0.220.040.05 query4 1.730.070.06 query5 0.500.470.48 query6 1.130.730.73 query7 0.020.010.02 query8 0.050.050.04 query9 0.550.500.49 query10 0.550.560.54 query11 0.150.120.11 query12 0.150.120.12 query13 0.590.600.58 query14 0.750.780.82 query15 0.840.820.82 query16 0.360.350.35 query17 1.020.981.04 query18 0.220.210.22 query19 1.801.751.67 query20 0.010.010.01 query21 15.37 0.750.65 query22 4.716.412.21 query23 18.65 1.291.31 query24 2.110.220.22 query25 0.160.090.09 query26 0.290.200.21 query27 0.470.230.23 query28 13.32 1.031.00 query29 12.65 3.333.25 query30 0.250.060.06 query31 2.880.380.40 query32 3.280.480.48 query33 2.922.872.92 query34 16.95 4.334.40 query35 4.394.434.47 query36 0.660.470.49 query37 0.200.160.17 query38 0.170.160.16 query39 0.040.040.04 query40 0.160.120.13 query41 0.090.040.05 query42 0.050.040.05 query43 0.040.040.04 Total cold run time: 110.57 s Total hot run time: 30.84 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Vault) Provide guidance log when user encounters no default vault situation. [doris]
doris-robot commented on PR #37677: URL: https://github.com/apache/doris/pull/37677#issuecomment-2226991003 ClickBench: Total hot run time: 30.75 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 19cc325f7f0318d084f9415c43e4792080d9569c, data reload: false query1 0.040.030.03 query2 0.070.040.03 query3 0.220.040.04 query4 1.690.080.08 query5 0.520.490.52 query6 1.130.730.73 query7 0.020.020.01 query8 0.050.050.05 query9 0.540.500.48 query10 0.540.540.54 query11 0.150.120.11 query12 0.150.120.12 query13 0.590.590.58 query14 0.760.770.76 query15 0.850.810.81 query16 0.360.370.36 query17 1.031.041.02 query18 0.230.230.22 query19 1.811.751.67 query20 0.000.010.01 query21 15.42 0.730.67 query22 4.117.022.11 query23 18.30 1.411.27 query24 2.040.220.22 query25 0.150.080.09 query26 0.290.210.20 query27 0.450.230.23 query28 13.35 1.021.01 query29 12.65 3.293.24 query30 0.250.060.05 query31 2.880.390.38 query32 3.270.470.47 query33 2.912.942.85 query34 16.71 4.364.36 query35 4.464.444.50 query36 0.630.460.48 query37 0.190.170.16 query38 0.160.150.15 query39 0.040.040.04 query40 0.150.120.11 query41 0.090.050.04 query42 0.060.050.04 query43 0.040.030.04 Total cold run time: 109.35 s Total hot run time: 30.75 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](inverted index) Error parsing NULL_LITERAL [doris]
github-actions[bot] commented on PR #37747: URL: https://github.com/apache/doris/pull/37747#issuecomment-2226991137 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](UT) Add UT for cloud string util [doris]
gavinchou commented on PR #37552: URL: https://github.com/apache/doris/pull/37552#issuecomment-2226991611 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](timeout) unify query_timeout, use setting from user property first [doris]
liutang123 commented on PR #37626: URL: https://github.com/apache/doris/pull/37626#issuecomment-2226991980 run compile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cherry-pick](branch-2.0) pick "[Fix](schema change) Fix schema change fault when add complex type column (#31824)" [doris]
dataroaring merged PR #37598: URL: https://github.com/apache/doris/pull/37598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [cherry-pick](branch-2.0) pick "[Fix](schema change) Fix schema change fault when add complex type column (#31824)" (#37598)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 914b6a77597 [cherry-pick](branch-2.0) pick "[Fix](schema change) Fix
schema change fault when add complex type column (#31824)" (#37598)
914b6a77597 is described below
commit 914b6a775970f617d3615d02b206fe7fa84d1d14
Author: abmdocrt
AuthorDate: Sun Jul 14 00:50:40 2024 +0800
[cherry-pick](branch-2.0) pick "[Fix](schema change) Fix schema change
fault when add complex type column (#31824)" (#37598)
Problem: An error is encountered when executing a schema change on a
unique table to add a column with a complex type, such as bitmap, as
documented in https://github.com/apache/doris/issues/31365
Reason: The error arises because the schema change logic erroneously
applies an aggregation check for new columns across all table types,
demanding an explicit aggregation type declaration. However, unique and
duplicate tables inherently assume default aggregation types for newly
added columns, leading to this misstep.
Solution: The schema change process for introducing new columns needs to
distinguish between table types accurately. For unique and duplicate
tables, it should automatically assign the appropriate aggregation type,
which, for the purpose of smooth integration with subsequent processes,
should be set to NONE.
pick #31824
## Proposed changes
Issue Number: close #xxx
---
.../java/org/apache/doris/analysis/AddColumnClause.java | 4
.../src/main/java/org/apache/doris/analysis/ColumnDef.java | 13 +++--
.../java/org/apache/doris/analysis/ModifyColumnClause.java | 4
.../schema_change_p0/test_schema_change_duplicate.groovy| 7 +++
.../schema_change_p0/test_schema_change_unique.groovy | 8
.../schema_change_p0/test_schema_change_unique_mow.groovy | 7 +++
6 files changed, 41 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/AddColumnClause.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/AddColumnClause.java
index 2d472cd5495..aa8e189b104 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/AddColumnClause.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/AddColumnClause.java
@@ -80,7 +80,11 @@ public class AddColumnClause extends AlterTableClause {
&& columnDef.getAggregateType() == null) {
columnDef.setIsKey(true);
}
+if (table instanceof OlapTable) {
+columnDef.setKeysType(((OlapTable) table).getKeysType());
+}
}
+
columnDef.analyze(true);
if (colPos != null) {
colPos.analyze();
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ColumnDef.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ColumnDef.java
index 2cd6b1e2870..da911c989af 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ColumnDef.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ColumnDef.java
@@ -22,6 +22,7 @@ package org.apache.doris.analysis;
import org.apache.doris.catalog.AggregateType;
import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.KeysType;
import org.apache.doris.catalog.PrimitiveType;
import org.apache.doris.catalog.ScalarType;
import org.apache.doris.catalog.Type;
@@ -178,6 +179,8 @@ public class ColumnDef {
private boolean isKey;
private boolean isAllowNull;
private boolean isAutoInc;
+private long autoIncInitValue;
+private KeysType keysType;
private DefaultValue defaultValue;
private String comment;
private boolean visible;
@@ -290,6 +293,10 @@ public class ColumnDef {
this.isKey = isKey;
}
+public void setKeysType(KeysType keysType) {
+this.keysType = keysType;
+}
+
public TypeDef getTypeDef() {
return typeDef;
}
@@ -326,8 +333,10 @@ public class ColumnDef {
if (isKey) {
throw new AnalysisException("Key column can not set complex
type:" + name);
}
-if (aggregateType == null) {
-throw new AnalysisException("complex type have to use
aggregate function: " + name);
+if (keysType == null || keysType == KeysType.AGG_KEYS) {
+if (aggregateType == null) {
+throw new AnalysisException("complex type have to use
aggregate function: " + name);
+}
}
isAllowNull = false;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyColumnClause.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ModifyColumnClause.java
index 5477614006b..49f5447163d 100644
--- a
Re: [PR] [improvement](cloud) Accelerate creating table by batching RPC In cloud [doris]
doris-robot commented on PR #36786: URL: https://github.com/apache/doris/pull/36786#issuecomment-2226996199 TPC-H: Total hot run time: 40417 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8503881ee03d2a6966860c677d5500ae2a67d23d, data reload: false -- Round 1 -- q1 18038 463943084308 q2 2024194 187 187 q3 10528 121812121212 q4 10231 831 886 831 q5 7610273526792679 q6 238 141 140 140 q7 964 601 598 598 q8 9235209321252093 q9 8631660465596559 q10 8776380038223800 q11 454 234 241 234 q12 397 238 226 226 q13 18960 297030272970 q14 288 234 242 234 q15 532 493 494 493 q16 506 380 378 378 q17 1000722 685 685 q18 8204754375827543 q19 5182138714611387 q20 666 335 327 327 q21 4924324433033244 q22 353 291 289 289 Total cold run time: 117741 ms Total hot run time: 40417 ms - Round 2, with runtime_filter_mode=off - q1 4422425042604250 q2 366 274 266 266 q3 3001274227802742 q4 1891164316711643 q5 5343536853385338 q6 224 133 134 133 q7 2117174417271727 q8 3233336733403340 q9 8522841684228416 q10 3865372737503727 q11 594 480 504 480 q12 762 634 606 606 q13 17498 294529432943 q14 297 274 273 273 q15 509 488 483 483 q16 511 446 432 432 q17 1804149814741474 q18 7802770973647364 q19 1669159715341534 q20 1962178017881780 q21 8275480247884788 q22 599 486 503 486 Total cold run time: 75266 ms Total hot run time: 54225 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](cloud) Accelerate creating table by batching RPC In cloud [doris]
doris-robot commented on PR #36786: URL: https://github.com/apache/doris/pull/36786#issuecomment-2226998884 TPC-DS: Total hot run time: 172109 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8503881ee03d2a6966860c677d5500ae2a67d23d, data reload: false query1 912 376 362 362 query2 6483188018041804 query3 6649205 216 205 query4 28202 17626 17479 17479 query5 4337481 503 481 query6 283 173 166 166 query7 4602290 281 281 query8 246 197 189 189 query9 8353236223382338 query10 446 291 269 269 query11 10644 10020 10022 10020 query12 127 87 78 78 query13 1642363 361 361 query14 10173 766476957664 query15 212 166 164 164 query16 7699313 303 303 query17 1723539 521 521 query18 1933268 273 268 query19 195 145 146 145 query20 87 83 80 80 query21 206 124 125 124 query22 4362393939703939 query23 33625 33010 32943 32943 query24 11981 288528332833 query25 688 359 374 359 query26 1773147 148 147 query27 2895269 267 267 query28 7485198819961988 query29 1107626 605 605 query30 286 150 148 148 query31 954 745 773 745 query32 99 56 58 56 query33 781 319 308 308 query34 912 497 484 484 query35 714 583 568 568 query36 1083926 954 926 query37 190 82 82 82 query38 2848275227852752 query39 862 802 805 802 query40 281 124 122 122 query41 49 47 49 47 query42 122 106 108 106 query43 506 474 453 453 query44 1203747 734 734 query45 193 163 164 163 query46 1093712 738 712 query47 1816173917571739 query48 365 309 295 295 query49 1237417 404 404 query50 790 403 397 397 query51 6924683267816781 query52 107 95 91 91 query53 350 295 286 286 query54 883 449 448 448 query55 75 73 73 73 query56 283 261 263 261 query57 1158107710641064 query58 246 234 248 234 query59 2924258924842484 query60 326 273 272 272 query61 96 95 93 93 query62 837 648 665 648 query63 316 290 289 289 query64 10449 226916871687 query65 3533310431113104 query66 1397346 339 339 query67 15674 15417 14945 14945 query68 4606525 519 519 query69 521 373 339 339 query70 1154108611721086 query71 400 283 276 276 query72 7076564456335633 query73 746 328 324 324 query74 5957547356095473 query75 3388268826782678 query76 2765987 896 896 query77 465 316 303 303 query78 9610906788428842 query79 2191517 516 516 query80 2139465 481 465 query81 586 223 226 223 query82 805 144 144 144 query83 285 165 166 165 query84 263 95 86 86 query85 2206325 303 303 query86 491 308 321 308 query87 3316308731143087 query88 4085246824542454 query89 463 373 377 373 query90 1876195 195 195 query91 133 102 103 102 query92 65 50 50 50 query93 2458493 498 493 query94 1359220 212 212 query95 407 315 320 315 query96 593 275 280 275 query97 3137302230513022 query98 219 204 190 190 query99 1535130712371237 Total cold run time: 286332 ms Total hot run time: 172109 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [improvement](cloud) Accelerate creating table by batching RPC In cloud [doris]
doris-robot commented on PR #36786: URL: https://github.com/apache/doris/pull/36786#issuecomment-222741 ClickBench: Total hot run time: 30.86 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 8503881ee03d2a6966860c677d5500ae2a67d23d, data reload: false query1 0.040.030.03 query2 0.080.040.03 query3 0.230.050.05 query4 1.670.090.09 query5 0.510.490.49 query6 1.140.720.72 query7 0.020.020.01 query8 0.050.050.05 query9 0.540.500.50 query10 0.550.550.54 query11 0.150.120.11 query12 0.150.120.12 query13 0.590.590.59 query14 0.750.780.81 query15 0.860.820.81 query16 0.360.360.36 query17 1.041.001.02 query18 0.240.220.21 query19 1.871.761.82 query20 0.010.020.01 query21 15.39 0.760.67 query22 4.317.012.16 query23 18.29 1.361.18 query24 2.040.260.22 query25 0.150.090.08 query26 0.290.210.22 query27 0.450.230.23 query28 13.31 1.020.99 query29 12.62 3.313.36 query30 0.260.060.05 query31 2.870.390.38 query32 3.270.470.48 query33 2.892.932.89 query34 17.02 4.304.34 query35 4.494.484.44 query36 0.650.480.45 query37 0.200.160.16 query38 0.160.140.15 query39 0.040.040.04 query40 0.150.120.12 query41 0.090.040.05 query42 0.060.050.04 query43 0.040.040.03 Total cold run time: 109.89 s Total hot run time: 30.86 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](compaction) optimizing memory usage for compaction (#37099) [doris]
doris-robot commented on PR #37486: URL: https://github.com/apache/doris/pull/37486#issuecomment-2227006409 TeamCity be ut coverage result: Function Coverage: 36.63% (9281/25336) Line Coverage: 28.16% (75960/269781) Region Coverage: 26.98% (39053/144761) Branch Coverage: 23.68% (19833/83746) Coverage Report: http://coverage.selectdb-in.cc/coverage/9edf23fa4c7129f3b3f3a49bf0041513df84b22b_9edf23fa4c7129f3b3f3a49bf0041513df84b22b/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](csv)Supports reading CSV data using LF and CRLF as line separators. [doris]
hubgeter commented on PR #37687: URL: https://github.com/apache/doris/pull/37687#issuecomment-2227039753 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](csv)Supports reading CSV data using LF and CRLF as line separators. [doris]
doris-robot commented on PR #37687: URL: https://github.com/apache/doris/pull/37687#issuecomment-2227090231 TPC-H: Total hot run time: 40112 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 7a951eaa70e9c1e53958bb8a6db118b6ea788df4, data reload: false -- Round 1 -- q1 17836 453643014301 q2 2020193 185 185 q3 10527 118710771077 q4 10197 864 837 837 q5 7572269426562656 q6 220 138 141 138 q7 971 606 609 606 q8 9301212720842084 q9 8801662266006600 q10 8852375237943752 q11 456 233 243 233 q12 396 226 226 226 q13 17779 301529992999 q14 272 238 245 238 q15 539 490 494 490 q16 495 396 380 380 q17 984 662 712 662 q18 8173756373307330 q19 7359152114361436 q20 663 333 324 324 q21 4867327239243272 q22 367 286 290 286 Total cold run time: 118647 ms Total hot run time: 40112 ms - Round 2, with runtime_filter_mode=off - q1 4398427242634263 q2 377 267 277 267 q3 3161289129282891 q4 1929170617671706 q5 5627555254445444 q6 227 137 135 135 q7 2293186018681860 q8 3324347934513451 q9 8809882688568826 q10 4152378538803785 q11 605 510 493 493 q12 808 649 603 603 q13 17025 315432093154 q14 310 304 277 277 q15 541 489 478 478 q16 501 430 447 430 q17 1840156215051505 q18 8121805477787778 q19 1768145815861458 q20 2233189018701870 q21 5876489846854685 q22 606 504 515 504 Total cold run time: 74531 ms Total hot run time: 55863 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](csv)Supports reading CSV data using LF and CRLF as line separators. [doris]
doris-robot commented on PR #37687: URL: https://github.com/apache/doris/pull/37687#issuecomment-2227093124 TPC-DS: Total hot run time: 174019 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 7a951eaa70e9c1e53958bb8a6db118b6ea788df4, data reload: false query1 928 378 364 364 query2 6425188917861786 query3 6638205 219 205 query4 28124 17568 17302 17302 query5 3654470 482 470 query6 266 170 167 167 query7 4580297 281 281 query8 249 197 197 197 query9 8445236823482348 query10 429 262 262 262 query11 11931 10019 10135 10019 query12 114 85 85 85 query13 1627359 366 359 query14 10315 754976457549 query15 224 165 163 163 query16 7454311 308 308 query17 1819557 558 557 query18 1851272 275 272 query19 198 157 157 157 query20 86 79 80 79 query21 198 124 123 123 query22 4374405539443944 query23 34073 33714 33510 33510 query24 11088 299229172917 query25 579 388 391 388 query26 700 148 150 148 query27 2302277 285 277 query28 6506202120362021 query29 890 656 651 651 query30 253 155 157 155 query31 993 769 770 769 query32 97 51 55 51 query33 766 310 292 292 query34 997 500 515 500 query35 695 583 580 580 query36 11511003956 956 query37 150 81 84 81 query38 3032286828522852 query39 902 804 818 804 query40 207 127 120 120 query41 51 46 45 45 query42 118 102 108 102 query43 529 453 489 453 query44 1192743 730 730 query45 201 173 159 159 query46 1102714 733 714 query47 1862173818011738 query48 363 294 299 294 query49 850 420 434 420 query50 772 398 393 393 query51 6952675068246750 query52 104 94 93 93 query53 359 291 290 290 query54 873 449 467 449 query55 77 75 73 73 query56 314 288 303 288 query57 11431114 query58 271 271 271 271 query59 2781267825992599 query60 323 297 298 297 query61 118 117 119 117 query62 812 643 641 641 query63 323 302 296 296 query64 9213229275612292 query65 3150308831013088 query66 766 344 341 341 query67 15408 15132 15031 15031 query68 5326524 530 524 query69 717 462 358 358 query70 1164116311651163 query71 479 285 281 281 query72 8675557555695569 query73 769 325 327 325 query74 6090555755525552 query75 4232268026722672 query76 3741937 945 937 query77 676 308 308 308 query78 12384 947889788978 query79 11404 517 568 517 query80 1354475 476 475 query81 573 224 223 223 query82 513 134 137 134 query83 320 170 165 165 query84 274 85 84 84 query85 721 300 291 291 query86 460 311 299 299 query87 3336313730993099 query88 5580246924852469 query89 481 372 389 372 query90 1933191 194 191 query91 131 99 99 99 query92 65 50 49 49 query93 4985495 488 488 query94 1238206 211 206 query95 412 322 323 322 query96 609 276 273 273 query97 3224298630232986 query98 214 196 204 196 query99 1609127013011270 Total cold run time: 299026 ms Total hot run time: 174019 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [feature](csv)Supports reading CSV data using LF and CRLF as line separators. [doris]
doris-robot commented on PR #37687: URL: https://github.com/apache/doris/pull/37687#issuecomment-2227094312 ClickBench: Total hot run time: 30.32 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 7a951eaa70e9c1e53958bb8a6db118b6ea788df4, data reload: false query1 0.040.030.03 query2 0.080.030.04 query3 0.220.050.05 query4 1.660.080.07 query5 0.500.480.48 query6 1.130.720.72 query7 0.020.010.02 query8 0.050.040.05 query9 0.540.490.50 query10 0.550.550.54 query11 0.160.120.12 query12 0.140.120.13 query13 0.590.580.58 query14 0.750.780.77 query15 0.840.810.81 query16 0.360.360.36 query17 1.020.951.04 query18 0.220.220.22 query19 1.911.811.79 query20 0.020.010.00 query21 15.40 0.750.65 query22 4.726.881.63 query23 18.28 1.311.32 query24 2.110.230.22 query25 0.160.090.08 query26 0.290.210.21 query27 0.460.230.23 query28 13.34 1.011.00 query29 12.62 3.333.28 query30 0.260.060.06 query31 2.860.390.39 query32 3.280.470.48 query33 2.902.902.88 query34 17.15 4.344.37 query35 4.414.384.38 query36 0.660.460.46 query37 0.190.150.14 query38 0.160.150.14 query39 0.050.040.03 query40 0.150.110.12 query41 0.100.040.05 query42 0.050.050.05 query43 0.050.040.04 Total cold run time: 110.45 s Total hot run time: 30.32 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](fix) replace hardcode s3BucketName [doris]
hello-stephen commented on PR #37739: URL: https://github.com/apache/doris/pull/37739#issuecomment-2227149887 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](index) rollback in compound opt [doris]
xiaokang commented on PR #37748: URL: https://github.com/apache/doris/pull/37748#issuecomment-2227153589 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [release](version) change branch-2.0 version to 2.0.13 [doris]
xiaokang opened a new pull request, #37749: URL: https://github.com/apache/doris/pull/37749 change branch-2.0 version to 2.0.13 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [release](version) change branch-2.0 version to 2.0.13 [doris]
doris-robot commented on PR #37749: URL: https://github.com/apache/doris/pull/37749#issuecomment-2227154362 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [test](fix) replace hardcode s3BucketName [doris]
hello-stephen opened a new pull request, #37750: URL: https://github.com/apache/doris/pull/37750 ## Proposed changes pick from master #37739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [release](version) change branch-2.0 version to 2.0.13 [doris]
xiaokang commented on PR #37749: URL: https://github.com/apache/doris/pull/37749#issuecomment-2227154876 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](fix) replace hardcode s3BucketName [doris]
doris-robot commented on PR #37750: URL: https://github.com/apache/doris/pull/37750#issuecomment-2227154911 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](fix) replace hardcode s3BucketName [doris]
hello-stephen commented on PR #37750: URL: https://github.com/apache/doris/pull/37750#issuecomment-2227154924 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](index) rollback in compound opt [doris]
doris-robot commented on PR #37748: URL: https://github.com/apache/doris/pull/37748#issuecomment-2227160358 TeamCity be ut coverage result: Function Coverage: 37.84% (8115/21444) Line Coverage: 29.53% (66516/225278) Region Coverage: 28.99% (34275/118234) Branch Coverage: 24.86% (17607/70812) Coverage Report: http://coverage.selectdb-in.cc/coverage/13c798e200af2bd5c3c1e4bb991a97c9fecef885_13c798e200af2bd5c3c1e4bb991a97c9fecef885/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](index) rollback in compound opt [doris]
doris-robot commented on PR #37748: URL: https://github.com/apache/doris/pull/37748#issuecomment-2227160493 TPC-H: Total hot run time: 50204 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 13c798e200af2bd5c3c1e4bb991a97c9fecef885, data reload: false -- Round 1 -- q1 17875 440744534407 q2 2079157 161 157 q3 10466 193320001933 q4 10288 125013721250 q5 8704387239283872 q6 257 130 131 130 q7 2051160716171607 q8 9352276527442744 q9 10724 10446 10254 10254 q10 8680359635733573 q11 417 252 260 252 q12 469 305 314 305 q13 18347 398240923982 q14 357 338 328 328 q15 511 458 456 456 q16 693 567 572 567 q17 1155986 1012986 q18 7364694069036903 q19 1818167516231623 q20 530 314 290 290 q21 4504422141534153 q22 534 432 456 432 Total cold run time: 117175 ms Total hot run time: 50204 ms - Round 2, with runtime_filter_mode=off - q1 4322433142764276 q2 322 230 224 224 q3 4215418741214121 q4 2800279727542754 q5 7130714671087108 q6 241 125 126 125 q7 3228282328642823 q8 4403449645144496 q9 16932 16877 16822 16822 q10 4250438143584358 q11 769 668 707 668 q12 1052874 902 874 q13 6958375037653750 q14 452 433 422 422 q15 512 456 461 456 q16 740 677 679 677 q17 3837389038033803 q18 8851870689478706 q19 1759172516581658 q20 2376216621382138 q21 8720850185178501 q22 1017995 960 960 Total cold run time: 84886 ms Total hot run time: 79720 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
