Re: [PR] [debug](load) check the column type when string column is invalid [doris]
doris-robot commented on PR #39337: URL: https://github.com/apache/doris/pull/39337#issuecomment-2288006822 TeamCity be ut coverage result: Function Coverage: 36.53% (9320/25516) Line Coverage: 28.02% (76366/272541) Region Coverage: 26.82% (39223/146229) Branch Coverage: 23.55% (19913/84554) Coverage Report: http://coverage.selectdb-in.cc/coverage/3c613172a3ae02d4b048e3835f33dcc3232b19a6_3c613172a3ae02d4b048e3835f33dcc3232b19a6/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
Mryange opened a new pull request, #39340:
URL: https://github.com/apache/doris/pull/39340
## Proposed changes
Since some variables are only initialized when enable_java_support is
enabled, not adding this check here would result in accessing a nullptr.
```
*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 0 ***
*** Aborted at 1723618805 (unix time) try "date -d @1723618805" if you are
using GNU date ***
*** Current BE git commitID: 8d64cfa563 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 2579243 (TID
2583082 OR 0x7f07ae942700) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/mnt/disk2/yanxuecheng/doris/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in
/mnt/disk2/yanxuecheng/java17/lib/server/libjvm.so
2# JVM_handle_linux_signal in
/mnt/disk2/yanxuecheng/java17/lib/server/libjvm.so
3# 0x7F0DA7188B50 in /lib64/libc.so.6
4# jni_CallVoidMethodV in /mnt/disk2/yanxuecheng/java17/lib/server/libjvm.so
5# JNIEnv_::CallVoidMethod(_jobject*, _jmethodID*, ...) at
/mnt/disk2/yanxuecheng/java17/include/jni.h:1059
6#
doris::JniUtil::clean_udf_class_load_cache(std::__cxx11::basic_string, std::allocator > const&) in
/mnt/disk2/yanxuecheng/doris/output/be/lib/doris_be
7# doris::clean_udf_cache_callback(doris::TAgentTaskRequest const&) at
/mnt/disk2/yanxuecheng/doris/be/src/agent/task_worker_pool.cpp:2072
8# auto doris::AgentServer::start_workers(doris::StorageEngine&,
doris::ExecEnv*)::$_19::operator()(doris::TAgentTaskRequest const&) const at
/mnt/disk2/yanxuecheng/doris/be/src/agent/agent_server.cpp:184
9# void std::__invoke_impl(std::__invoke_other,
doris::AgentServer::start_workers(doris::StorageEngine&,
doris::ExecEnv*)::$_19&, doris::TAgentTaskRequest const&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61
10# std::enable_if, void>::type
std::__invoke_r(doris::AgentServer::start_workers(doris::StorageEngine&,
doris::ExecEnv*)::$_19&, doris::TAgentTaskRequest const&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117
11# std::_Function_handler::_M_invoke(std::_Any_data const&,
doris::TAgentTaskRequest const&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290
12# std::function::operator()(doris::TAgentTaskRequest const&) const at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591
13# doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest
const&)::$_0::operator()(doris::TAgentTaskRequest const&) const::{lambda()#1}::operator()()
const at /mnt/disk2/yanxuecheng/doris/be/src/agent/task_worker_pool.cpp:541
14# void std::__invoke_impl(doris::TAgentTaskRequest const&)
const::{lambda()#1}&>(std::__invoke_other,
doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest
const&)::$_0::operator()(doris::TAgentTaskRequest const&) const::{lambda()#1}&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61
15# std::enable_if(doris::TAgentTaskRequest const&) const::{lambda()#1}&>, void>::type
std::__invoke_r(doris::TAgentTaskRequest const&)
const::{lambda()#1}&>(doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest
const&)::$_0::operator()(doris::TAgentTaskRequest const&) const::{lambda()#1}&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117
16# std::_Function_handler(doris::TAgentTaskRequest const&)
const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290
17# std::function::operator()() const at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591
18# doris::FunctionRunnable::run() at
/mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:48
19# doris::ThreadPool::dispatch_thread() at
/mnt/disk2/yanxuecheng/doris/be/src/util/threadpool.cpp:543
20# void std::__invoke_impl(std::__invoke_memfun_deref, void
(doris::ThreadPool::*&)(), doris::ThreadPool*&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:74
21# std::__invoke_result::type std::__invoke(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96
22# void std::_Bind::__call(std::tuple<>&&, std::_Index_tuple<0ul>) at
/mnt/disk2/yanxuecheng/ldb_toolchain/bin/../lib/
Re: [PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
doris-robot commented on PR #39340: URL: https://github.com/apache/doris/pull/39340#issuecomment-2288013793 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
Mryange commented on PR #39340: URL: https://github.com/apache/doris/pull/39340#issuecomment-2288013925 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition rebalancer) fix migrate tablets between backends back and forth [doris]
yujun777 commented on PR #39333: URL: https://github.com/apache/doris/pull/39333#issuecomment-2288022172 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Modify thedefault JEMALLOC_CONF and support flush Jemalloc tcache [doris]
xinyiZzz commented on PR #38185: URL: https://github.com/apache/doris/pull/38185#issuecomment-2288027931 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](ES Catalog)Check isArray before parse json to array (#39104) (#39275)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new adfef1d790e [fix](ES Catalog)Check isArray before parse json to array
(#39104) (#39275)
adfef1d790e is described below
commit adfef1d790e2474fabad98a68f8bceaa1ebdec73
Author: qiye
AuthorDate: Wed Aug 14 15:21:27 2024 +0800
[fix](ES Catalog)Check isArray before parse json to array (#39104) (#39275)
---
be/src/exec/es/es_scroll_parser.cpp| 25 +-
.../scripts/data/composite_type_array_bulk.json| 10
.../elasticsearch/scripts/es_init.sh | 58 ++
.../index/array_meta_composite_type_array.json | 9
.../scripts/index/es6_composite_type_array.json| 14 ++
.../scripts/index/es7_composite_type_array.json| 12 +
.../external_table_p0/es/test_es_query.groovy | 25 ++
7 files changed, 141 insertions(+), 12 deletions(-)
diff --git a/be/src/exec/es/es_scroll_parser.cpp
b/be/src/exec/es/es_scroll_parser.cpp
index 138bbb81bc0..22c9d3ee337 100644
--- a/be/src/exec/es/es_scroll_parser.cpp
+++ b/be/src/exec/es/es_scroll_parser.cpp
@@ -100,9 +100,9 @@ static const std::string ERROR_COL_DATA_IS_ARRAY =
static const std::string INVALID_NULL_VALUE =
"Invalid null value occurs: Non-null column `$0` contains NULL";
-#define RETURN_ERROR_IF_COL_IS_ARRAY(col, type) \
+#define RETURN_ERROR_IF_COL_IS_ARRAY(col, type, is_array)\
do { \
-if (col.IsArray()) { \
+if (col.IsArray() == is_array) { \
std::stringstream ss;\
ss << "Expected value of type: " << type_to_string(type) \
<< "; but found type: " << json_type_to_string(col.GetType()) \
@@ -167,7 +167,7 @@ Status get_int_value(const rapidjson::Value& col,
PrimitiveType type, void* slot
return Status::OK();
}
-RETURN_ERROR_IF_COL_IS_ARRAY(col, type);
+RETURN_ERROR_IF_COL_IS_ARRAY(col, type, true);
RETURN_ERROR_IF_COL_IS_NOT_STRING(col, type);
StringParser::ParseResult result;
@@ -294,7 +294,7 @@ Status get_date_int(const rapidjson::Value& col,
PrimitiveType type, bool pure_d
return get_date_value_int(col[0], type, false, slot, time_zone);
} else {
// this would happened just only when `enable_docvalue_scan = false`,
and field has string format date from _source
-RETURN_ERROR_IF_COL_IS_ARRAY(col, type);
+RETURN_ERROR_IF_COL_IS_ARRAY(col, type, true);
RETURN_ERROR_IF_COL_IS_NOT_STRING(col, type);
return get_date_value_int(col, type, true, slot, time_zone);
}
@@ -322,7 +322,7 @@ Status get_float_value(const rapidjson::Value& col,
PrimitiveType type, void* sl
return Status::OK();
}
-RETURN_ERROR_IF_COL_IS_ARRAY(col, type);
+RETURN_ERROR_IF_COL_IS_ARRAY(col, type, true);
RETURN_ERROR_IF_COL_IS_NOT_STRING(col, type);
StringParser::ParseResult result;
@@ -351,7 +351,7 @@ Status insert_float_value(const rapidjson::Value& col,
PrimitiveType type,
return Status::OK();
}
-RETURN_ERROR_IF_COL_IS_ARRAY(col, type);
+RETURN_ERROR_IF_COL_IS_ARRAY(col, type, true);
RETURN_ERROR_IF_COL_IS_NOT_STRING(col, type);
StringParser::ParseResult result;
@@ -390,7 +390,7 @@ Status insert_int_value(const rapidjson::Value& col,
PrimitiveType type,
return Status::OK();
}
-RETURN_ERROR_IF_COL_IS_ARRAY(col, type);
+RETURN_ERROR_IF_COL_IS_ARRAY(col, type, true);
RETURN_ERROR_IF_COL_IS_NOT_STRING(col, type);
StringParser::ParseResult result;
@@ -543,7 +543,7 @@ Status ScrollParser::fill_columns(const TupleDescriptor*
tuple_desc,
val = col[0].GetString();
}
} else {
-RETURN_ERROR_IF_COL_IS_ARRAY(col, type);
+RETURN_ERROR_IF_COL_IS_ARRAY(col, type, true);
if (!col.IsString()) {
val = json_value_to_string(col);
} else {
@@ -618,7 +618,7 @@ Status ScrollParser::fill_columns(const TupleDescriptor*
tuple_desc,
const rapidjson::Value& str_col = is_nested_str ? col[0] : col;
-RETURN_ERROR_IF_COL_IS_ARRAY(col, type);
+RETURN_ERROR_IF_COL_IS_ARRAY(col, type, true);
const std::string& val = str_col.GetString();
size_t val_size = str_col.GetStringLength();
@@ -644,7 +644,7 @@ Status ScrollParser::fill_columns(const TupleDescriptor*
tuple_desc,
val = col[0].GetString();
}
Re: [PR] [fix](ES Catalog)Check isArray before parse json to array (#39104) [doris]
xiaokang merged PR #39275: URL: https://github.com/apache/doris/pull/39275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
doris-robot commented on PR #39341: URL: https://github.com/apache/doris/pull/39341#issuecomment-2288031796 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
zy-kkk opened a new pull request, #39341: URL: https://github.com/apache/doris/pull/39341 In previous versions, we used a method based on JDBC 4.2 to read data, so it was equivalent to abandoning support for ojdbc6. However, we recently found that a large number of users still use Oracle version 11g, which will have some unexpected compatibility issues when using ojdbc8 to connect. Therefore, I use version verification to make it compatible with both ojdbc6 and ojdbc8, so that good compatibility can be obtained through ojdbc6, and better reading efficiency can be obtained through ojdbc8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
zy-kkk commented on PR #39341: URL: https://github.com/apache/doris/pull/39341#issuecomment-2288032077 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bug](schema change)fix schema change cause load failed due to err -215 [doris]
DarvenDuan commented on PR #23836: URL: https://github.com/apache/doris/pull/23836#issuecomment-2288041389 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](query cancel) cancel query if it has pipeline task leakage [doris]
zhiqiang- commented on PR #39223: URL: https://github.com/apache/doris/pull/39223#issuecomment-2288041735 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](doris-future) Support REGR_SXX_SXY_SYY aggregation functions [doris]
wyxxxcat commented on PR #39187: URL: https://github.com/apache/doris/pull/39187#issuecomment-2288043810 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nullif) revert nullif signature change by #38497 [doris]
morrySnow opened a new pull request, #39342: URL: https://github.com/apache/doris/pull/39342 #38497 let second arg follow first arg's type. but the correct way is let second arg convert to a new type get from processing like 1st = 2nd. and then compare this two arg with the 2nd arg type when execution. this pr revert nullif signature. and will let nullif work as expected in future PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nullif) revert nullif signature change by #38497 [doris]
morrySnow commented on PR #39342: URL: https://github.com/apache/doris/pull/39342#issuecomment-2288046834 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [branch-2.0](txn) be dead exceeds 5min abort its txns (#22781, #28662, #35342) (#39317)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new eaffb691d38 [branch-2.0](txn) be dead exceeds 5min abort its txns
(#22781, #28662, #35342) (#39317)
eaffb691d38 is described below
commit eaffb691d38afe0995f073847572564df49b909e
Author: yujun
AuthorDate: Wed Aug 14 15:31:51 2024 +0800
[branch-2.0](txn) be dead exceeds 5min abort its txns (#22781, #28662,
#35342) (#39317)
cherry-pick: #22781, #28662, #35342
-
Co-authored-by: HHoflittlefish777
<77738092+hhoflittlefish...@users.noreply.github.com>
---
.../runtime/stream_load/stream_load_executor.cpp | 4 +
docs/en/docs/admin-manual/config/fe-config.md | 10 ++
docs/zh-CN/docs/admin-manual/config/fe-config.md | 10 ++
.../main/java/org/apache/doris/common/Config.java | 14 +++
.../apache/doris/analysis/NativeInsertStmt.java| 5 +-
.../java/org/apache/doris/common/ClientPool.java | 2 +-
.../java/org/apache/doris/common/FeConstants.java | 1 -
.../org/apache/doris/httpv2/rest/LoadAction.java | 6 ++
.../java/org/apache/doris/load/DeleteHandler.java | 5 +-
.../apache/doris/load/loadv2/BrokerLoadJob.java| 5 +-
.../org/apache/doris/load/loadv2/SparkLoadJob.java | 5 +-
.../load/routineload/RoutineLoadTaskInfo.java | 5 +-
.../doris/load/sync/canal/CanalSyncChannel.java| 7 +-
.../org/apache/doris/nereids/txn/Transaction.java | 5 +-
.../java/org/apache/doris/qe/SimpleScheduler.java | 3 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 8 +-
.../java/org/apache/doris/service/ExecuteEnv.java | 7 ++
.../apache/doris/service/FrontendServiceImpl.java | 12 ++-
.../java/org/apache/doris/system/HeartbeatMgr.java | 18 +++-
.../doris/transaction/DatabaseTransactionMgr.java | 7 +-
.../doris/transaction/GlobalTransactionMgr.java| 32 +--
.../apache/doris/transaction/TransactionState.java | 15 ++-
.../apache/doris/clone/BeDownCancelCloneTest.java | 4 +-
.../doris/cluster/DecommissionBackendTest.java | 2 +-
.../org/apache/doris/qe/SimpleSchedulerTest.java | 8 +-
.../transaction/DatabaseTransactionMgrTest.java| 10 +-
.../transaction/GlobalTransactionMgrTest.java | 11 ++-
.../doris/transaction/TransactionStateTest.java| 5 +-
.../apache/doris/utframe/TestWithFeService.java| 2 +-
gensrc/thrift/FrontendService.thrift | 2 +
.../org/apache/doris/regression/suite/Suite.groovy | 62
.../suites/demo_p0/streamLoad_action.groovy| 5 +
.../stream_load/test_coordidator_be_restart.groovy | 106 +
33 files changed, 349 insertions(+), 54 deletions(-)
diff --git a/be/src/runtime/stream_load/stream_load_executor.cpp
b/be/src/runtime/stream_load/stream_load_executor.cpp
index 4a80eb35ca6..19d25e9ffa1 100644
--- a/be/src/runtime/stream_load/stream_load_executor.cpp
+++ b/be/src/runtime/stream_load/stream_load_executor.cpp
@@ -45,6 +45,7 @@
#include "runtime/stream_load/new_load_stream_mgr.h"
#include "runtime/stream_load/stream_load_context.h"
#include "thrift/protocol/TDebugProtocol.h"
+#include "util/debug_points.h"
#include "util/doris_metrics.h"
#include "util/thrift_rpc_helper.h"
#include "util/time.h"
@@ -242,6 +243,7 @@ Status StreamLoadExecutor::begin_txn(StreamLoadContext*
ctx) {
request.__set_timeout(ctx->timeout_second);
}
request.__set_request_id(ctx->id.to_thrift());
+request.__set_backend_id(_exec_env->master_info()->backend_id);
TLoadTxnBeginResult result;
Status status;
@@ -374,6 +376,8 @@ void
StreamLoadExecutor::get_commit_request(StreamLoadContext* ctx,
}
Status StreamLoadExecutor::commit_txn(StreamLoadContext* ctx) {
+DBUG_EXECUTE_IF("StreamLoadExecutor.commit_txn.block", DBUG_BLOCK);
+
DorisMetrics::instance()->stream_load_txn_commit_request_total->increment(1);
TLoadTxnCommitRequest request;
diff --git a/docs/en/docs/admin-manual/config/fe-config.md
b/docs/en/docs/admin-manual/config/fe-config.md
index 1c2ab9939a1..fbab5cfcd15 100644
--- a/docs/en/docs/admin-manual/config/fe-config.md
+++ b/docs/en/docs/admin-manual/config/fe-config.md
@@ -587,6 +587,16 @@ Is it possible to configure dynamically: true
Whether it is a configuration item unique to the Master FE node: true
+### `abort_txn_after_lost_heartbeat_time_second`
+
+Abort transaction time after lost heartbeat. The default value is 300, which
means transactions of be will be aborted after lost heartbeat 300s.
+
+Default: 300(s)
+
+Is it possible to configure dynamically: true
+
+Whether it is a configuration item unique to the Master FE node: true
+
`enable_access_file_without_broker`
Default：false
diff --git a/docs/zh-CN/docs/admin-manual/config/fe-config.md
b/docs/zh-CN/docs/admin-ma
(doris) branch master updated (8844e03bbda -> d9c8ff041c7)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 8844e03bbda [opt](mtmv) partition rollup support week and quarter (#39286) add d9c8ff041c7 [fix](cloud) fix routine load job progress cache incorrect in cloud mode (#39313) No new revisions were added by this update. Summary of changes: .../doris/load/routineload/KafkaProgress.java | 23 -- 1 file changed, 21 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](txn) be dead exceeds 5min abort its txns (#22781, #28662, #35342) [doris]
dataroaring merged PR #39317: URL: https://github.com/apache/doris/pull/39317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) fix routine load job progress cache incorrect in cloud mode [doris]
dataroaring merged PR #39313: URL: https://github.com/apache/doris/pull/39313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](restore) Support clean_tables/clean_partitions properties for restore job [doris]
github-actions[bot] commented on PR #39028: URL: https://github.com/apache/doris/pull/39028#issuecomment-2288049123 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](cloud) CloudStorageEngine should wait compaction task done when stoping it [doris]
dataroaring merged PR #39302: URL: https://github.com/apache/doris/pull/39302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (d9c8ff041c7 -> 181c202bf64)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from d9c8ff041c7 [fix](cloud) fix routine load job progress cache incorrect in cloud mode (#39313) add 181c202bf64 [Fix](cloud) CloudStorageEngine should wait compaction task done when stoping it (#39302) No new revisions were added by this update. Summary of changes: be/src/cloud/cloud_storage_engine.cpp | 8 1 file changed, 8 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](query cancel) cancel query if it has pipeline task leakage [doris]
zhiqiang- commented on PR #39223: URL: https://github.com/apache/doris/pull/39223#issuecomment-2288053073 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](set operation) adjust two different behaviors with MySQL [doris]
morrySnow closed pull request #39042: [opt](set operation) adjust two different behaviors with MySQL URL: https://github.com/apache/doris/pull/39042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](thirdparty) upgrade arrow to 17.0.0 [doris]
xinyiZzz commented on PR #38572: URL: https://github.com/apache/doris/pull/38572#issuecomment-2288056673 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Cooldown) Add metric to trace cooldown task and unused remote files caused by failed upload and cold compaction [doris]
github-actions[bot] commented on PR #39293: URL: https://github.com/apache/doris/pull/39293#issuecomment-2288057141 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](Cooldown) Add metric to trace cooldown task and unused remote files caused by failed upload and cold compaction [doris]
github-actions[bot] commented on PR #39293: URL: https://github.com/apache/doris/pull/39293#issuecomment-2288057190 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] revert [improvement](mv) Support to use cast when create sync materialized view #38008 [doris]
doris-robot commented on PR #39338: URL: https://github.com/apache/doris/pull/39338#issuecomment-2288059304 TPC-H: Total hot run time: 38197 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit fc1768dfe093b0c8171558f420b71c4431578f85, data reload: false -- Round 1 -- q1 18292 483544294429 q2 2699192 202 192 q3 11823 118512081185 q4 11144 780 778 778 q5 8119289828562856 q6 226 140 140 140 q7 969 596 611 596 q8 9309209220462046 q9 6977653065456530 q10 7061221222692212 q11 467 246 253 246 q12 395 226 218 218 q13 19089 297730142977 q14 284 236 246 236 q15 540 512 491 491 q16 514 389 389 389 q17 1024658 763 658 q18 7622681268326812 q19 43901048954 954 q20 677 344 344 344 q21 3903291130092911 q22 1100997 999 997 Total cold run time: 116624 ms Total hot run time: 38197 ms - Round 2, with runtime_filter_mode=off - q1 4511427142714271 q2 376 270 265 265 q3 2879261126262611 q4 1911158816031588 q5 5413541253615361 q6 229 134 137 134 q7 2054165016911650 q8 32123337 q9 8418839083758375 q10 3420319231553155 q11 592 496 511 496 q12 778 602 592 592 q13 16698 300630003000 q14 306 267 273 267 q15 523 475 474 474 q16 494 418 420 418 q17 1811150414921492 q18 7633753975087508 q19 1667149915451499 q20 2017180017991799 q21 5297517651105110 q22 10831019997 997 Total cold run time: 71322 ms Total hot run time: 54395 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth) Fix ResourceTypeEnum Compatibility Upgrade [doris]
github-actions[bot] commented on PR #39288: URL: https://github.com/apache/doris/pull/39288#issuecomment-2288059476 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](fold) support complex type for constant folding [doris]
WencongLiu commented on PR #32867: URL: https://github.com/apache/doris/pull/32867#issuecomment-2288060031 @zhangstar333 Hi, R、recently, I have been developing some new functions and noticed that they are not being folded even when the constant folding feature is enabled. I would like to seek advice on how to modify them to achieve constant folding. Could you possibly add my WeChat, Liuwenclever, for further discussion? I would greatly appreciate your assistance. Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Txn) Fix wrong columns sequence in txn model [doris]
morrySnow commented on code in PR #39295:
URL: https://github.com/apache/doris/pull/39295#discussion_r1716449846
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalSink.java:
##
@@ -54,4 +54,8 @@ public List computeOutput() {
.map(NamedExpression::toSlot)
.collect(ImmutableList.toImmutableList());
}
+
+public List getOutputExprs() {
+return outputExprs;
+}
Review Comment:
remove override function in
`org.apache.doris.nereids.trees.plans.physical.PhysicalResultSink`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](array) forbid create inverted index with parser for array and add cases for write correctness check [doris]
amorynan commented on PR #39062: URL: https://github.com/apache/doris/pull/39062#issuecomment-2288064121 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) reuse connection in load_stream_stub [doris]
github-actions[bot] commented on PR #39231: URL: https://github.com/apache/doris/pull/39231#issuecomment-2288067360 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) reuse connection in load_stream_stub [doris]
github-actions[bot] commented on PR #39231: URL: https://github.com/apache/doris/pull/39231#issuecomment-2288067414 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](nereids) support extract from disjunction in join on condition [doris]
doris-robot commented on PR #38479: URL: https://github.com/apache/doris/pull/38479#issuecomment-2288071971 TPC-H: Total hot run time: 37972 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d4da9b8744c048477395e396af2525d0189acd3c, data reload: false -- Round 1 -- q1 17626 442743304330 q2 2018180 185 180 q3 11591 109011411090 q4 10505 749 811 749 q5 7743283528222822 q6 224 137 139 137 q7 960 586 595 586 q8 9539207220982072 q9 8627656065676560 q10 7029222021932193 q11 471 244 245 244 q12 391 221 214 214 q13 17771 301830353018 q14 280 260 236 236 q15 520 492 490 490 q16 490 408 377 377 q17 985 718 776 718 q18 7509675568526755 q19 57271102969 969 q20 670 327 324 324 q21 3877297629182918 q22 1098990 1011990 Total cold run time: 115651 ms Total hot run time: 37972 ms - Round 2, with runtime_filter_mode=off - q1 4449426642294229 q2 373 272 270 270 q3 2872266326662663 q4 1918166916721669 q5 5702576156445644 q6 230 140 134 134 q7 2136174217131713 q8 3289349934713471 q9 8831880087918791 q10 3569327932503250 q11 618 509 514 509 q12 869 621 587 587 q13 15753 307231313072 q14 313 298 290 290 q15 533 501 483 483 q16 486 438 451 438 q17 1847157215551555 q18 8167801779287928 q19 2191159915841584 q20 2183190418701870 q21 8399532553425325 q22 1195104610401040 Total cold run time: 75923 ms Total hot run time: 56515 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
doris-robot commented on PR #39340: URL: https://github.com/apache/doris/pull/39340#issuecomment-2288074352 TPC-H: Total hot run time: 37781 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit af86ff5b7c623d41b39d9278dfee8197c9d0b2cb, data reload: false -- Round 1 -- q1 18687 801943284328 q2 2034188 179 179 q3 11648 110911711109 q4 10443 832 704 704 q5 7769285728042804 q6 223 137 139 137 q7 1028624 612 612 q8 9332204821162048 q9 7205654465376537 q10 7008220322112203 q11 454 242 234 234 q12 387 221 220 220 q13 17751 297029772970 q14 285 246 231 231 q15 514 495 487 487 q16 511 393 396 393 q17 1008759 742 742 q18 7548679467846784 q19 60651052964 964 q20 665 324 335 324 q21 3877284127802780 q22 11141005991 991 Total cold run time: 115556 ms Total hot run time: 37781 ms - Round 2, with runtime_filter_mode=off - q1 4488423042574230 q2 381 282 275 275 q3 2818261827292618 q4 2021176117081708 q5 5723566756025602 q6 224 129 131 129 q7 2131177318151773 q8 3307347634343434 q9 8814885988908859 q10 3529333932193219 q11 592 512 519 512 q12 812 657 640 640 q13 16209 319731673167 q14 332 291 277 277 q15 575 493 475 475 q16 503 444 448 444 q17 1837155115311531 q18 8018783079107830 q19 1740152216071522 q20 2160191919051905 q21 5664541852275227 q22 1118103811071038 Total cold run time: 72996 ms Total hot run time: 56415 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](err-msg) Add detailed column and schema info when failed to create a column iterator [doris]
doris-robot commented on PR #38689: URL: https://github.com/apache/doris/pull/38689#issuecomment-2288073084 TPC-H: Total hot run time: 37933 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit babfb5a6e4babb557646e3136f2974efdd900883, data reload: false -- Round 1 -- q1 17641 453342924292 q2 2022180 179 179 q3 11543 107111651071 q4 10500 755 759 755 q5 7751286828262826 q6 222 135 137 135 q7 964 598 608 598 q8 9367206321072063 q9 7243654265136513 q10 7005219021672167 q11 455 257 247 247 q12 397 224 225 224 q13 18834 297729702970 q14 270 253 241 241 q15 518 484 476 476 q16 513 400 381 381 q17 1011762 675 675 q18 7429685267696769 q19 6285107311261073 q20 675 329 325 325 q21 3839305029532953 q22 1127100010381000 Total cold run time: 115611 ms Total hot run time: 37933 ms - Round 2, with runtime_filter_mode=off - q1 4543424443094244 q2 367 281 261 261 q3 2843262626182618 q4 1896169517081695 q5 5709568055875587 q6 227 137 139 137 q7 2100174617301730 q8 3260351834373437 q9 8873885188198819 q10 3585335232563256 q11 595 514 522 514 q12 834 616 622 616 q13 16932 319331663166 q14 327 286 293 286 q15 519 491 482 482 q16 507 431 435 431 q17 1858156215071507 q18 8094805078247824 q19 1759154315171517 q20 2158186619471866 q21 8000524355695243 q22 1128992 1008992 Total cold run time: 76114 ms Total hot run time: 56228 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition rebalancer) fix migrate tablets between backends back and forth [doris]
github-actions[bot] commented on PR #39333: URL: https://github.com/apache/doris/pull/39333#issuecomment-2288075298 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](external)add check of engine and catalog types [doris]
wuwenchi commented on PR #39343: URL: https://github.com/apache/doris/pull/39343#issuecomment-2288076747 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](sort) Free sort blocks if this block is exhausted [doris]
Gabriel39 commented on PR #39306: URL: https://github.com/apache/doris/pull/39306#issuecomment-2288076756 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition rebalancer) fix migrate tablets between backends back and forth [doris]
doris-robot commented on PR #39333: URL: https://github.com/apache/doris/pull/39333#issuecomment-2288078923 TPC-H: Total hot run time: 37411 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8703450260877c131a38fa02f4e680c88214e4d4, data reload: false -- Round 1 -- q1 17600 438042514251 q2 2017182 183 182 q3 11904 101411441014 q4 10514 751 697 697 q5 7754279427782778 q6 219 135 138 135 q7 950 582 591 582 q8 9511202720432027 q9 8716649465096494 q10 7038221221562156 q11 468 241 243 241 q12 388 219 215 215 q13 17765 298829972988 q14 277 237 236 236 q15 535 488 489 488 q16 494 385 383 383 q17 957 703 745 703 q18 7453668468986684 q19 75341137993 993 q20 680 333 332 332 q21 3813284630902846 q22 10631003986 986 Total cold run time: 117650 ms Total hot run time: 37411 ms - Round 2, with runtime_filter_mode=off - q1 4572428642954286 q2 377 266 271 266 q3 2862261625922592 q4 1981161816851618 q5 5632573355685568 q6 224 136 140 136 q7 2127172817061706 q8 3351348334353435 q9 8804865188178651 q10 3567330233163302 q11 629 522 497 497 q12 782 641 604 604 q13 17032 315532103155 q14 308 286 295 286 q15 525 494 508 494 q16 495 451 448 448 q17 1868152315351523 q18 8074784878177817 q19 2822153915241524 q20 2168189218721872 q21 6972547955345479 q22 1144100210271002 Total cold run time: 76316 ms Total hot run time: 56261 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] revert [improvement](mv) Support to use cast when create sync materialized view #38008 [doris]
doris-robot commented on PR #39338: URL: https://github.com/apache/doris/pull/39338#issuecomment-2288079568 TPC-DS: Total hot run time: 185282 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit fc1768dfe093b0c8171558f420b71c4431578f85, data reload: false query1 921 394 388 388 query2 6447190318581858 query3 6657209 225 209 query4 34106 23284 23181 23181 query5 4248513 516 513 query6 286 185 181 181 query7 4595293 289 289 query8 253 208 208 208 query9 8491245424352435 query10 444 300 269 269 query11 16625 15146 15141 15141 query12 144 100 100 100 query13 1635366 366 366 query14 9578685271886852 query15 216 170 169 169 query16 7512447 486 447 query17 1575577 539 539 query18 1419281 283 281 query19 191 147 149 147 query20 119 111 111 111 query21 208 102 99 99 query22 4363402939213921 query23 33955 33156 34907 33156 query24 11988 290628632863 query25 522 387 389 387 query26 1120157 155 155 query27 2685267 273 267 query28 7420205320292029 query29 634 411 405 405 query30 332 152 149 149 query31 994 773 775 773 query32 97 53 56 53 query33 736 289 282 282 query34 912 457 479 457 query35 825 735 760 735 query36 1095914 951 914 query37 217 82 81 81 query38 3880384538413841 query39 1440139313691369 query40 276 116 117 116 query41 48 44 44 44 query42 122 95 99 95 query43 509 473 466 466 query44 1230745 734 734 query45 197 165 162 162 query46 1125784 735 735 query47 1842174517581745 query48 372 293 290 290 query49 1129438 452 438 query50 811 408 408 408 query51 6810674666946694 query52 98 93 89 89 query53 266 186 187 186 query54 816 463 454 454 query55 79 75 75 75 query56 271 256 246 246 query57 1148105210751052 query58 237 222 228 222 query59 2881271226142614 query60 301 271 272 271 query61 100 98 100 98 query62 784 643 637 637 query63 218 188 193 188 query64 6076228817281728 query65 3224316931533153 query66 1083351 348 348 query67 15400 14957 14858 14858 query68 4829546 563 546 query69 399 280 283 280 query70 1135107210941072 query71 380 283 277 277 query72 2602208920332033 query73 734 325 321 321 query74 9202887687658765 query75 3415268227552682 query76 2886108510581058 query77 550 318 311 311 query78 9493908190239023 query79 1631541 543 541 query80 1575495 492 492 query81 577 224 222 222 query82 778 141 134 134 query83 242 152 154 152 query84 273 80 75 75 query85 1443278 273 273 query86 460 309 321 309 query87 4476414242034142 query88 4047231722982298 query89 396 282 294 282 query90 1829201 197 197 query91 123 101 95 95 query92 69 53 51 51 query93 1888542 535 535 query94 746 298 303 298 query95 370 344 267 267 query96 611 268 269 268 query97 3208304430363036 query98 223 205 205 205 query99 1540127612891276 Total cold run time: 288468 ms Total hot run time: 185282 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [Fix](regression) Fix flaky case in test_stream_load_2pc [doris]
liaoxin01 commented on PR #39257: URL: https://github.com/apache/doris/pull/39257#issuecomment-2288080340 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
github-actions[bot] commented on PR #39340: URL: https://github.com/apache/doris/pull/39340#issuecomment-2288087641 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
github-actions[bot] commented on PR #39340: URL: https://github.com/apache/doris/pull/39340#issuecomment-2288087700 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] revert [improvement](mv) Support to use cast when create sync materialized view #38008 [doris]
doris-robot commented on PR #39338: URL: https://github.com/apache/doris/pull/39338#issuecomment-2288088776 ClickBench: Total hot run time: 30.79 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit fc1768dfe093b0c8171558f420b71c4431578f85, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.220.050.05 query4 1.660.090.11 query5 0.500.480.49 query6 1.120.730.73 query7 0.020.010.01 query8 0.050.050.05 query9 0.540.480.49 query10 0.540.550.52 query11 0.160.120.12 query12 0.140.130.13 query13 0.600.600.59 query14 0.790.810.77 query15 0.880.830.82 query16 0.370.370.39 query17 1.021.011.05 query18 0.230.230.21 query19 1.751.751.76 query20 0.010.010.01 query21 15.39 0.760.65 query22 4.527.141.72 query23 18.24 1.431.38 query24 2.120.220.22 query25 0.160.080.08 query26 0.300.220.21 query27 0.460.240.23 query28 13.28 1.021.01 query29 12.57 3.363.33 query30 0.240.050.05 query31 2.890.410.39 query32 3.260.480.48 query33 2.972.972.98 query34 17.01 4.414.31 query35 4.434.484.43 query36 0.660.490.47 query37 0.210.160.15 query38 0.160.150.15 query39 0.040.030.04 query40 0.150.120.13 query41 0.100.050.05 query42 0.060.060.05 query43 0.050.040.04 Total cold run time: 110 s Total hot run time: 30.79 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function) avoid calculating sqrt of negative in agg function CORR [doris]
zclllyybb commented on PR #39324: URL: https://github.com/apache/doris/pull/39324#issuecomment-2288091556 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](nereids) support extract from disjunction in join on condition [doris]
doris-robot commented on PR #38479: URL: https://github.com/apache/doris/pull/38479#issuecomment-2288092881 TPC-DS: Total hot run time: 189667 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d4da9b8744c048477395e396af2525d0189acd3c, data reload: false query1 1267887 848 848 query2 6474195118981898 query3 10624 377936413641 query4 59378 24778 23248 23248 query5 5939487 474 474 query6 498 158 167 158 query7 6522292 294 292 query8 306 224 210 210 query9 9016245424432443 query10 504 268 259 259 query11 18659 15035 15320 15035 query12 170 99 109 99 query13 1599393 407 393 query14 12146 662169146621 query15 241 172 176 172 query16 7671510 512 510 query17 1147589 570 570 query18 1980294 305 294 query19 287 159 157 157 query20 131 115 109 109 query21 220 108 113 108 query22 460842744274 query23 34416 33360 33184 33184 query24 5593291928842884 query25 552 401 409 401 query26 696 160 160 160 query27 1792271 274 271 query28 3781204820812048 query29 702 423 423 423 query30 242 155 148 148 query31 941 734 778 734 query32 97 54 55 54 query33 491 298 292 292 query34 859 467 469 467 query35 814 727 717 717 query36 1056926 930 926 query37 131 89 81 81 query38 3930382538593825 query39 1451139113991391 query40 195 120 115 115 query41 47 43 42 42 query42 122 96 97 96 query43 500 469 478 469 query44 1081742 761 742 query45 195 164 167 164 query46 1099758 736 736 query47 1825178017681768 query48 371 291 294 291 query49 763 425 435 425 query50 798 411 412 411 query51 6883675267276727 query52 106 88 95 88 query53 259 189 185 185 query54 582 566 444 444 query55 75 74 76 74 query56 269 242 240 240 query57 1124109310741074 query58 224 241 229 229 query59 2925279127322732 query60 303 266 289 266 query61 97 98 96 96 query62 753 651 649 649 query63 206 187 185 185 query64 4403227217691769 query65 3223317331263126 query66 664 325 326 325 query67 15551 15054 14930 14930 query68 6871547 544 544 query69 674 367 285 285 query70 1214112010711071 query71 550 276 268 268 query72 7053228920512051 query73 792 317 318 317 query74 9298879287708770 query75 4353270427652704 query76 4297103310181018 query77 776 328 357 328 query78 9901896789748967 query79 8425535 530 530 query80 2238489 495 489 query81 565 226 219 219 query82 1032135 134 134 query83 302 148 143 143 query84 262 78 74 74 query85 1480279 309 279 query86 375 296 293 293 query87 4417421942644219 query88 5160230622972297 query89 540 287 284 284 query90 1987196 191 191 query91 124 96 95 95 query92 66 50 52 50 query93 6237545 536 536 query94 716 287 293 287 query95 345 257 265 257 query96 622 266 265 265 query97 3231302430473024 query98 223 207 203 203 query99 1518127812581258 Total cold run time: 337218 ms Total hot run time: 189667 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [enhancement](err-msg) Add detailed column and schema info when failed to create a column iterator [doris]
doris-robot commented on PR #38689: URL: https://github.com/apache/doris/pull/38689#issuecomment-2288093905 TPC-DS: Total hot run time: 188168 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit babfb5a6e4babb557646e3136f2974efdd900883, data reload: false query1 1256892 847 847 query2 6467188918071807 query3 10745 385638823856 query4 60085 26322 23149 23149 query5 5501493 485 485 query6 490 175 156 156 query7 6407291 287 287 query8 295 202 197 197 query9 8926241623972397 query10 471 261 244 244 query11 18246 14884 15302 14884 query12 164 99 105 99 query13 1537386 377 377 query14 11642 657170456571 query15 225 174 171 171 query16 7566513 465 465 query17 1142565 575 565 query18 2118304 304 304 query19 305 155 154 154 query20 119 113 112 112 query21 209 99 109 99 query22 4610445543344334 query23 34190 33221 33107 33107 query24 5546287328612861 query25 548 396 397 396 query26 691 163 157 157 query27 1791270 271 270 query28 3731204820352035 query29 710 425 427 425 query30 243 155 147 147 query31 950 735 737 735 query32 74 52 55 52 query33 413 291 283 283 query34 858 451 461 451 query35 824 707 700 700 query36 1043932 923 923 query37 130 85 87 85 query38 3899377637673767 query39 1458137913991379 query40 199 118 114 114 query41 48 45 46 45 query42 113 95 100 95 query43 489 461 449 449 query44 1065739 748 739 query45 202 169 161 161 query46 1082711 728 711 query47 1849176617691766 query48 358 289 289 289 query49 760 427 457 427 query50 794 407 410 407 query51 6821667366256625 query52 104 103 89 89 query53 261 184 185 184 query54 572 456 466 456 query55 79 78 75 75 query56 281 255 248 248 query57 1130113610381038 query58 216 234 211 211 query59 2847270026692669 query60 286 259 265 259 query61 101 96 103 96 query62 763 648 646 646 query63 205 181 177 177 query64 426217051705 query65 3236309131423091 query66 671 326 328 326 query67 15641 14920 14796 14796 query68 6105527 529 527 query69 616 266 255 255 query70 1154111710621062 query71 525 279 271 271 query72 7518226220612061 query73 787 312 316 312 query74 9092873387118711 query75 4821268526512651 query76 4107945 1012945 query77 681 306 309 306 query78 9899896989388938 query79 11389 525 521 521 query80 856 502 491 491 query81 581 221 261 221 query82 707 134 131 131 query83 330 143 146 143 query84 270 71 73 71 query85 909 274 266 266 query86 341 305 291 291 query87 4376416240874087 query88 4671228822822282 query89 496 281 286 281 query90 2287190 190 190 query91 119 94 95 94 query92 66 49 49 49 query93 5602531 530 530 query94 1001292 301 292 query95 358 263 258 258 query96 604 263 277 263 query97 3181300329982998 query98 231 246 199 199 query99 1451124412571244 Total cold run time: 335191 ms Total hot run time: 188168 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
doris-robot commented on PR #39340: URL: https://github.com/apache/doris/pull/39340#issuecomment-2288094608 TPC-DS: Total hot run time: 189551 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit af86ff5b7c623d41b39d9278dfee8197c9d0b2cb, data reload: false query1 1269910 858 858 query2 6449186818101810 query3 10598 398838963896 query4 59346 24986 23126 23126 query5 5341505 492 492 query6 492 174 169 169 query7 6414294 291 291 query8 307 203 200 200 query9 8896244924352435 query10 526 267 251 251 query11 17815 14960 15172 14960 query12 175 106 100 100 query13 1645369 364 364 query14 11125 685668716856 query15 232 175 170 170 query16 7536499 467 467 query17 1128565 562 562 query18 2010285 307 285 query19 295 143 146 143 query20 122 110 111 110 query21 221 103 104 103 query22 4705444344944443 query23 34264 33627 33405 33405 query24 5566288028352835 query25 511 394 379 379 query26 681 158 154 154 query27 1789268 273 268 query28 3947206820592059 query29 678 421 393 393 query30 227 154 154 154 query31 905 719 730 719 query32 78 52 53 52 query33 430 283 274 274 query34 837 450 461 450 query35 820 724 700 700 query36 1028909 945 909 query37 139 80 80 80 query38 3846378237583758 query39 1449140014031400 query40 230 114 114 114 query41 48 43 44 43 query42 116 93 93 93 query43 497 448 462 448 query44 1081737 733 733 query45 193 164 160 160 query46 1080722 778 722 query47 1892182517781778 query48 350 282 286 282 query49 745 410 430 410 query50 809 398 404 398 query51 6782676167466746 query52 110 92 84 84 query53 254 178 183 178 query54 570 454 448 448 query55 74 76 74 74 query56 255 245 237 237 query57 1151105910471047 query58 212 220 222 220 query59 2838272427712724 query60 286 258 257 257 query61 136 97 96 96 query62 765 623 658 623 query63 210 180 178 178 query64 3124173416871687 query65 3243311031693110 query66 687 336 328 328 query67 15224 14912 14989 14912 query68 4764544 535 535 query69 414 263 269 263 query70 1180114211611142 query71 432 269 273 269 query72 6525226820752075 query73 777 321 322 321 query74 9096883187788778 query75 3335272826882688 query76 2411913 977 913 query77 551 306 297 297 query78 9704903090489030 query79 2116527 519 519 query80 1025475 480 475 query81 565 219 221 219 query82 986 133 135 133 query83 242 145 149 145 query84 265 79 74 74 query85 1089283 337 283 query86 467 270 302 270 query87 4355422141784178 query88 4291232123172317 query89 399 294 289 289 query90 1958193 189 189 query91 119 94 97 94 query92 58 48 48 48 query93 2891527 517 517 query94 928 291 302 291 query95 359 254 261 254 query96 595 275 270 270 query97 3200306030353035 query98 234 202 203 202 query99 1551122812631228 Total cold run time: 313686 ms Total hot run time: 189551 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Chore](aggregation) remove useless code for aggregation function Covar [doris]
github-actions[bot] commented on PR #39309: URL: https://github.com/apache/doris/pull/39309#issuecomment-2288100168 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition rebalancer) fix migrate tablets between backends back and forth [doris]
doris-robot commented on PR #39333: URL: https://github.com/apache/doris/pull/39333#issuecomment-2288100222 TPC-DS: Total hot run time: 189504 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 8703450260877c131a38fa02f4e680c88214e4d4, data reload: false query1 1257883 843 843 query2 6429192118411841 query3 10591 392139563921 query4 58261 26647 23201 23201 query5 5534499 497 497 query6 468 176 155 155 query7 6231320 291 291 query8 315 215 207 207 query9 8715244824222422 query10 496 270 288 270 query11 17875 15013 15192 15013 query12 157 101 105 101 query13 1567389 389 389 query14 11714 781173747374 query15 265 172 186 172 query16 7272513 552 513 query17 1149605 579 579 query18 1482307 308 307 query19 314 147 148 147 query20 123 109 114 109 query21 221 105 108 105 query22 4561416746094167 query23 34307 33306 33218 33218 query24 5748286528362836 query25 552 405 407 405 query26 687 159 161 159 query27 1781279 279 279 query28 3766207920362036 query29 668 427 427 427 query30 239 151 154 151 query31 921 759 756 756 query32 79 60 57 57 query33 486 288 297 288 query34 863 467 463 463 query35 820 731 721 721 query36 1058950 926 926 query37 144 84 87 84 query38 3879376338273763 query39 1436137713991377 query40 198 121 119 119 query41 49 45 48 45 query42 120 96 94 94 query43 505 457 458 457 query44 1096747 733 733 query45 195 164 170 164 query46 1099710 759 710 query47 1824171817271718 query48 365 290 301 290 query49 771 428 439 428 query50 817 403 423 403 query51 6825662866396628 query52 102 90 91 90 query53 262 183 185 183 query54 584 551 448 448 query55 73 76 75 75 query56 269 245 244 244 query57 1122106210521052 query58 214 228 238 228 query59 2821260327832603 query60 301 265 270 265 query61 95 93 96 93 query62 777 639 643 639 query63 219 184 182 182 query64 3366177017361736 query65 3193312331423123 query66 685 331 359 331 query67 15448 14878 14924 14878 query68 5065542 551 542 query69 510 277 279 277 query70 1168113911411139 query71 494 309 278 278 query72 6745229218721872 query73 777 313 319 313 query74 9338879387998793 query75 4497269527582695 query76 3425993 1020993 query77 762 308 310 308 query78 9938909688108810 query79 9637551 552 551 query80 1046514 503 503 query81 578 230 225 225 query82 629 141 137 137 query83 353 150 143 143 query84 267 73 74 73 query85 871 275 303 275 query86 347 269 307 269 query87 4375420540704070 query88 4531228722962287 query89 531 289 287 287 query90 2209200 199 199 query91 124 98 98 98 query92 63 50 50 50 query93 6216551 548 548 query94 877 296 289 289 query95 344 258 265 258 query96 616 268 267 267 query97 3165303229932993 query98 228 204 193 193 query99 1523125612401240 Total cold run time: 327075 ms Total hot run time: 189504 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [improvement](nereids) support extract from disjunction in join on condition [doris]
doris-robot commented on PR #38479: URL: https://github.com/apache/doris/pull/38479#issuecomment-2288102807 ClickBench: Total hot run time: 31.09 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d4da9b8744c048477395e396af2525d0189acd3c, data reload: false query1 0.040.040.04 query2 0.100.050.04 query3 0.220.050.06 query4 1.970.080.08 query5 0.510.500.49 query6 1.280.740.73 query7 0.020.010.01 query8 0.050.040.04 query9 0.550.480.48 query10 0.540.540.54 query11 0.160.110.12 query12 0.150.130.12 query13 0.600.580.58 query14 0.770.770.78 query15 0.860.820.82 query16 0.370.370.37 query17 0.991.061.02 query18 0.230.210.22 query19 1.791.811.79 query20 0.010.010.01 query21 15.39 0.740.65 query22 3.727.732.01 query23 18.29 1.381.39 query24 2.070.220.23 query25 0.160.070.08 query26 0.300.200.21 query27 0.460.240.23 query28 13.23 1.021.00 query29 12.65 3.343.31 query30 0.240.050.04 query31 2.900.390.39 query32 3.260.490.48 query33 2.933.012.98 query34 16.93 4.354.33 query35 4.464.424.47 query36 0.650.500.48 query37 0.200.160.16 query38 0.160.150.15 query39 0.040.040.04 query40 0.150.120.13 query41 0.090.060.04 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 109.6 s Total hot run time: 31.09 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](err-msg) Add detailed column and schema info when failed to create a column iterator [doris]
doris-robot commented on PR #38689: URL: https://github.com/apache/doris/pull/38689#issuecomment-2288103789 ClickBench: Total hot run time: 30.4 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit babfb5a6e4babb557646e3136f2974efdd900883, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.220.040.04 query4 1.690.080.07 query5 0.500.480.49 query6 1.120.720.72 query7 0.020.010.01 query8 0.060.040.04 query9 0.550.490.47 query10 0.520.550.54 query11 0.160.120.11 query12 0.150.120.13 query13 0.590.590.58 query14 0.760.770.77 query15 0.850.810.81 query16 0.370.370.36 query17 1.031.050.97 query18 0.230.220.22 query19 1.821.681.76 query20 0.010.010.03 query21 15.39 0.730.65 query22 3.857.331.78 query23 18.29 1.421.26 query24 2.090.230.21 query25 0.160.090.09 query26 0.300.210.21 query27 0.450.230.23 query28 13.31 1.011.00 query29 12.61 3.263.22 query30 0.240.060.04 query31 2.890.390.39 query32 3.280.480.48 query33 2.882.972.98 query34 17.05 4.394.36 query35 4.474.454.39 query36 0.670.470.48 query37 0.180.160.15 query38 0.150.160.15 query39 0.040.030.03 query40 0.160.120.12 query41 0.090.040.04 query42 0.060.050.04 query43 0.050.040.04 Total cold run time: 109.44 s Total hot run time: 30.4 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](java udf) fix clean_udf_cache_callback without enable_java_support [doris]
doris-robot commented on PR #39340: URL: https://github.com/apache/doris/pull/39340#issuecomment-2288104497 ClickBench: Total hot run time: 30.91 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit af86ff5b7c623d41b39d9278dfee8197c9d0b2cb, data reload: false query1 0.040.050.04 query2 0.070.040.04 query3 0.220.050.06 query4 1.660.070.08 query5 0.510.480.47 query6 1.140.730.73 query7 0.010.020.02 query8 0.050.040.05 query9 0.520.480.48 query10 0.550.530.53 query11 0.160.120.12 query12 0.160.120.12 query13 0.580.600.57 query14 0.760.770.78 query15 0.860.820.82 query16 0.370.370.38 query17 1.021.071.00 query18 0.220.220.22 query19 1.881.771.76 query20 0.010.010.01 query21 15.39 0.740.66 query22 4.027.312.10 query23 18.31 1.421.28 query24 2.150.230.22 query25 0.150.090.08 query26 0.310.220.21 query27 0.460.230.23 query28 13.17 1.021.00 query29 12.60 3.323.31 query30 0.240.060.04 query31 2.880.380.39 query32 3.280.490.47 query33 2.942.972.94 query34 17.19 4.334.37 query35 4.434.404.38 query36 0.660.460.48 query37 0.190.160.16 query38 0.160.160.16 query39 0.050.030.03 query40 0.150.120.12 query41 0.090.040.04 query42 0.050.040.04 query43 0.040.040.04 Total cold run time: 109.7 s Total hot run time: 30.91 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](thirdparty) upgrade arrow to 17.0.0 [doris]
doris-robot commented on PR #38572: URL: https://github.com/apache/doris/pull/38572#issuecomment-2288104510 TPC-H: Total hot run time: 37328 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a66223089b8600a77ba9495bf7f5457c916adda8, data reload: false -- Round 1 -- q1 18139 439242924292 q2 2014175 175 175 q3 10523 123610881088 q4 10219 731 748 731 q5 7754280627482748 q6 225 145 144 144 q7 956 583 590 583 q8 9308203120122012 q9 7063645065186450 q10 6993216221802162 q11 471 236 248 236 q12 397 223 218 218 q13 19065 296429902964 q14 271 249 230 230 q15 523 470 487 470 q16 500 382 384 382 q17 967 706 731 706 q18 7514679268556792 q19 6689101310411013 q20 675 332 327 327 q21 3796290426022602 q22 1088100910031003 Total cold run time: 115150 ms Total hot run time: 37328 ms - Round 2, with runtime_filter_mode=off - q1 4534426142794261 q2 384 280 267 267 q3 2809264926422642 q4 1870160115721572 q5 5461560156735601 q6 231 134 136 134 q7 2148175517641755 q8 3233340433843384 q9 8772872689458726 q10 3490323132523231 q11 599 514 509 509 q12 810 606 601 601 q13 17522 311931443119 q14 321 291 281 281 q15 524 493 490 490 q16 515 451 479 451 q17 1835152415651524 q18 8074777973587358 q19 1676155515481548 q20 2093180317641764 q21 5287523350125012 q22 10871037999 999 Total cold run time: 73275 ms Total hot run time: 55229 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
doris-robot commented on PR #39341: URL: https://github.com/apache/doris/pull/39341#issuecomment-2288108680 TPC-H: Total hot run time: 37589 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 27139502490b745ea24b2f8bd31671dafd61b6a6, data reload: false -- Round 1 -- q1 17643 432942504250 q2 2019187 187 187 q3 11748 970 1101970 q4 10511 738 771 738 q5 7766280127612761 q6 223 137 136 136 q7 949 590 586 586 q8 9308204520442044 q9 7179651365356513 q10 7020224422352235 q11 443 243 243 243 q12 396 224 225 224 q13 18759 298630052986 q14 286 232 236 232 q15 535 478 487 478 q16 495 405 381 381 q17 963 741 707 707 q18 7259687468196819 q19 72601070943 943 q20 790 340 336 336 q21 3967294928122812 q22 1091100810081008 Total cold run time: 116610 ms Total hot run time: 37589 ms - Round 2, with runtime_filter_mode=off - q1 4426428942534253 q2 372 275 265 265 q3 2837261225992599 q4 1899170316931693 q5 5661577355855585 q6 221 132 137 132 q7 2176176217621762 q8 3312345534453445 q9 8708869887428698 q10 3578326032763260 q11 592 523 520 520 q12 808 612 604 604 q13 16153 315231283128 q14 324 288 295 288 q15 521 490 501 490 q16 492 446 447 446 q17 1817155815011501 q18 8062792679967926 q19 1801170816181618 q20 2140190919021902 q21 5616540753445344 q22 1121102210411022 Total cold run time: 72637 ms Total hot run time: 56481 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](partition rebalancer) fix migrate tablets between backends back and forth [doris]
doris-robot commented on PR #39333: URL: https://github.com/apache/doris/pull/39333#issuecomment-2288110420 ClickBench: Total hot run time: 30.99 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 8703450260877c131a38fa02f4e680c88214e4d4, data reload: false query1 0.050.040.04 query2 0.080.040.05 query3 0.230.050.04 query4 1.680.080.08 query5 0.520.500.50 query6 1.130.720.73 query7 0.020.020.01 query8 0.050.050.04 query9 0.540.480.48 query10 0.550.540.53 query11 0.160.110.11 query12 0.150.130.13 query13 0.620.610.60 query14 0.760.800.78 query15 0.850.820.82 query16 0.370.370.36 query17 1.001.050.95 query18 0.230.220.22 query19 1.871.811.82 query20 0.010.010.01 query21 15.39 0.740.65 query22 4.267.042.09 query23 18.28 1.371.24 query24 2.060.220.21 query25 0.150.090.08 query26 0.300.200.22 query27 0.460.240.23 query28 13.32 1.011.00 query29 12.62 3.373.35 query30 0.240.050.05 query31 2.880.390.39 query32 3.290.500.47 query33 2.962.952.97 query34 16.92 4.374.37 query35 4.464.434.40 query36 0.650.470.45 query37 0.190.160.15 query38 0.160.160.15 query39 0.040.040.03 query40 0.150.120.12 query41 0.090.050.05 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 109.85 s Total hot run time: 30.99 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](restore) Support clean_tables/clean_partitions properties for restore job [doris]
dataroaring merged PR #39028: URL: https://github.com/apache/doris/pull/39028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [feature](restore) Support clean_tables/clean_partitions properties for restore job (#39028)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 73787b68b4d [feature](restore) Support clean_tables/clean_partitions
properties for restore job (#39028)
73787b68b4d is described below
commit 73787b68b4df9e1020316054ea7f19144d3ed972
Author: walter
AuthorDate: Wed Aug 14 16:08:43 2024 +0800
[feature](restore) Support clean_tables/clean_partitions properties for
restore job (#39028)
The restore will keep the existing tables/partitions in which the
restore target is not contained, this PR adds a property clean_restore
to indicate that the restore job needs to recycle those
tables/partitions
The CCR part PR is: selectdb/ccr-syncer#128.
The website PR is: apache/doris-website#999
---
.../org/apache/doris/analysis/RestoreStmt.java | 104 +++---
.../org/apache/doris/backup/BackupHandler.java | 4 +-
.../java/org/apache/doris/backup/RestoreJob.java | 97 +++--
.../java/org/apache/doris/catalog/Partition.java | 4 +-
.../apache/doris/datasource/InternalCatalog.java | 145 +++
.../apache/doris/service/FrontendServiceImpl.java | 12 ++
.../org/apache/doris/backup/RestoreJobTest.java| 2 +-
gensrc/thrift/FrontendService.thrift | 2 +
.../test_backup_restore_clean_restore.groovy | 158 +
9 files changed, 406 insertions(+), 122 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
index 317de11c1f6..45691647001 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
@@ -35,13 +35,15 @@ import java.util.Set;
public class RestoreStmt extends AbstractBackupStmt {
private static final String PROP_ALLOW_LOAD = "allow_load";
-private static final String PROP_REPLICATION_NUM = "replication_num";
private static final String PROP_BACKUP_TIMESTAMP = "backup_timestamp";
private static final String PROP_META_VERSION = "meta_version";
-private static final String PROP_RESERVE_REPLICA = "reserve_replica";
-private static final String PROP_RESERVE_DYNAMIC_PARTITION_ENABLE =
"reserve_dynamic_partition_enable";
private static final String PROP_IS_BEING_SYNCED =
PropertyAnalyzer.PROPERTIES_IS_BEING_SYNCED;
+public static final String PROP_RESERVE_REPLICA = "reserve_replica";
+public static final String PROP_RESERVE_DYNAMIC_PARTITION_ENABLE =
"reserve_dynamic_partition_enable";
+public static final String PROP_CLEAN_TABLES = "clean_tables";
+public static final String PROP_CLEAN_PARTITIONS = "clean_partitions";
+
private boolean allowLoad = false;
private ReplicaAllocation replicaAlloc =
ReplicaAllocation.DEFAULT_ALLOCATION;
private String backupTimestamp = null;
@@ -50,16 +52,18 @@ public class RestoreStmt extends AbstractBackupStmt {
private boolean reserveDynamicPartitionEnable = false;
private boolean isLocal = false;
private boolean isBeingSynced = false;
+private boolean isCleanTables = false;
+private boolean isCleanPartitions = false;
private byte[] meta = null;
private byte[] jobInfo = null;
public RestoreStmt(LabelName labelName, String repoName,
AbstractBackupTableRefClause restoreTableRefClause,
- Map properties) {
+Map properties) {
super(labelName, repoName, restoreTableRefClause, properties);
}
public RestoreStmt(LabelName labelName, String repoName,
AbstractBackupTableRefClause restoreTableRefClause,
- Map properties, byte[] meta, byte[]
jobInfo) {
+Map properties, byte[] meta, byte[] jobInfo) {
super(labelName, repoName, restoreTableRefClause, properties);
this.meta = meta;
this.jobInfo = jobInfo;
@@ -109,6 +113,14 @@ public class RestoreStmt extends AbstractBackupStmt {
return isBeingSynced;
}
+public boolean isCleanTables() {
+return isCleanTables;
+}
+
+public boolean isCleanPartitions() {
+return isCleanPartitions;
+}
+
@Override
public void analyze(Analyzer analyzer) throws UserException {
if (repoName.equals(Repository.KEEP_ON_LOCAL_REPO_NAME)) {
@@ -142,17 +154,7 @@ public class RestoreStmt extends AbstractBackupStmt {
Map copiedProperties = Maps.newHashMap(properties);
// allow load
-if (copiedProperties.containsKey(PROP_ALLOW_LOAD)) {
-if
(copiedProperties.get(PROP_ALLOW_LOAD).equalsIgnoreCase("true")) {
-allowLoad = true;
-} else if
(copiedProperties.get(PROP_ALLOW_LOAD).equalsIgnoreCase("false")) {
Re: [PR] [fix](nullif) revert nullif signature change by #38497 [doris]
doris-robot commented on PR #39342: URL: https://github.com/apache/doris/pull/39342#issuecomment-2288110793 TPC-H: Total hot run time: 37419 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b75b3faa64ede7caa3bbcddb6b68a688d85a3e97, data reload: false -- Round 1 -- q1 17623 431542874287 q2 2018183 182 182 q3 11697 963 1155963 q4 10512 719 696 696 q5 7738277927802779 q6 218 137 139 137 q7 958 593 611 593 q8 9306204320682043 q9 7200650965266509 q10 7004219622192196 q11 459 248 246 246 q12 391 224 222 222 q13 18169 296730122967 q14 288 231 246 231 q15 525 483 497 483 q16 491 404 387 387 q17 966 664 706 664 q18 7366677667066706 q19 7101972 1047972 q20 857 343 339 339 q21 3907281729442817 q22 1091100010171000 Total cold run time: 115885 ms Total hot run time: 37419 ms - Round 2, with runtime_filter_mode=off - q1 4545437042434243 q2 373 278 271 271 q3 2841259126072591 q4 1906170116461646 q5 5670564756095609 q6 226 146 143 143 q7 2174175616821682 q8 3322348734873487 q9 8852871187238711 q10 3611329932493249 q11 596 525 493 493 q12 825 658 663 658 q13 15875 304431653044 q14 327 288 278 278 q15 540 502 510 502 q16 475 430 460 430 q17 1835149515221495 q18 8148807178267826 q19 1734140714361407 q20 2094190819111908 q21 6989535653085308 q22 1183102710191019 Total cold run time: 74141 ms Total hot run time: 56000 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Chore](exec) remove some thread pool and rename fragmentMgrThreadPool [doris]
BiteThet opened a new pull request, #39344: URL: https://github.com/apache/doris/pull/39344 ## Proposed changes remove some thread pool and rename fragmentMgrThreadPool -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Chore](exec) remove some thread pool and rename fragmentMgrThreadPool [doris]
BiteThet commented on PR #39344: URL: https://github.com/apache/doris/pull/39344#issuecomment-2288111394 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](ip)pick ip rowstore [doris]
amorynan opened a new pull request, #39345: URL: https://github.com/apache/doris/pull/39345 ## Proposed changes backport: https://github.com/apache/doris/pull/39258 Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ip)pick ip rowstore [doris]
doris-robot commented on PR #39345: URL: https://github.com/apache/doris/pull/39345#issuecomment-2288113361 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ip)pick ip rowstore [doris]
amorynan commented on PR #39345: URL: https://github.com/apache/doris/pull/39345#issuecomment-2288113455 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mtmv)fix refresh failed when catalog rebuild [doris]
zddr closed pull request #39283: [fix](mtmv)fix refresh failed when catalog rebuild URL: https://github.com/apache/doris/pull/39283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) fix group commit core if be inject FragmentMgr.exec_plan_fragment.failed [doris]
mymeiyi commented on PR #39339: URL: https://github.com/apache/doris/pull/39339#issuecomment-2288119728 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](inverted index)Add exception check when write bkd index (#39248) [doris]
xiaokang merged PR #39279: URL: https://github.com/apache/doris/pull/39279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](inverted index)Add exception check when write bkd index (#39248) (#39279)
This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 9e9bdfb6721 [fix](inverted index)Add exception check when write bkd
index (#39248) (#39279)
9e9bdfb6721 is described below
commit 9e9bdfb672102a1c90f04dd0db72ecb088a4c276
Author: qiye
AuthorDate: Wed Aug 14 16:15:04 2024 +0800
[fix](inverted index)Add exception check when write bkd index (#39248)
(#39279)
---
.../rowset/segment_v2/inverted_index_writer.cpp| 40 +
.../test_index_bkd_writer_fault_injection.groovy | 51 ++
2 files changed, 72 insertions(+), 19 deletions(-)
diff --git a/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp
b/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp
index 9b452b6c765..cdfc0d7fdd4 100644
--- a/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp
+++ b/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp
@@ -395,7 +395,7 @@ public:
_rid++;
}
} else if constexpr (field_is_numeric_type(field_type)) {
-add_numeric_values(values, count);
+RETURN_IF_ERROR(add_numeric_values(values, count));
}
return Status::OK();
}
@@ -487,11 +487,7 @@ public:
continue;
}
const CppType* p = &reinterpret_cast(value_ptr)[j];
-std::string new_value;
-size_t value_length = sizeof(CppType);
-
-_value_key_coder->full_encode_ascending(p, &new_value);
-_bkd_writer->add((const uint8_t*)new_value.c_str(),
value_length, _rid);
+RETURN_IF_ERROR(add_value(*p));
}
start_off += array_elem_size;
_row_ids_seen_for_bkd++;
@@ -535,11 +531,7 @@ public:
if (values->is_null_at(j)) {
// bkd do not index null values, so we do nothing here.
} else {
-std::string new_value;
-size_t value_length = sizeof(CppType);
-
-_value_key_coder->full_encode_ascending(p, &new_value);
-_bkd_writer->add((const uint8_t*)new_value.c_str(),
value_length, _rid);
+RETURN_IF_ERROR(add_value(*p));
}
item_data_ptr = (uint8_t*)item_data_ptr + field_size;
}
@@ -551,23 +543,33 @@ public:
return Status::OK();
}
-void add_numeric_values(const void* values, size_t count) {
+Status add_numeric_values(const void* values, size_t count) {
auto p = reinterpret_cast(values);
for (size_t i = 0; i < count; ++i) {
-add_value(*p);
+RETURN_IF_ERROR(add_value(*p));
+_rid++;
p++;
_row_ids_seen_for_bkd++;
}
+return Status::OK();
}
-void add_value(const CppType& value) {
-std::string new_value;
-size_t value_length = sizeof(CppType);
+Status add_value(const CppType& value) {
+try {
+std::string new_value;
+size_t value_length = sizeof(CppType);
-_value_key_coder->full_encode_ascending(&value, &new_value);
-_bkd_writer->add((const uint8_t*)new_value.c_str(), value_length,
_rid);
+
DBUG_EXECUTE_IF("InvertedIndexColumnWriterImpl::add_value_bkd_writer_add_throw_error",
{
+_CLTHROWA(CL_ERR_IllegalArgument, ("packedValue should be
length=xxx"));
+});
-_rid++;
+_value_key_coder->full_encode_ascending(&value, &new_value);
+_bkd_writer->add((const uint8_t*)new_value.c_str(), value_length,
_rid);
+} catch (const CLuceneError& e) {
+return Status::Error(
+"CLuceneError add_value: {}", e.what());
+}
+return Status::OK();
}
int64_t size() const override {
diff --git
a/regression-test/suites/fault_injection_p0/test_index_bkd_writer_fault_injection.groovy
b/regression-test/suites/fault_injection_p0/test_index_bkd_writer_fault_injection.groovy
new file mode 100644
index 000..7df72ebeaf1
--- /dev/null
+++
b/regression-test/suites/fault_injection_p0/test_index_bkd_writer_fault_injection.groovy
@@ -0,0 +1,51 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/lic
Re: [PR] [opt](thirdparty) upgrade arrow to 17.0.0 [doris]
doris-robot commented on PR #38572: URL: https://github.com/apache/doris/pull/38572#issuecomment-2288127870 TPC-DS: Total hot run time: 184603 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a66223089b8600a77ba9495bf7f5457c916adda8, data reload: false query1 908 380 362 362 query2 6460198719011901 query3 6680204 227 204 query4 34128 23149 23248 23149 query5 4177502 510 502 query6 296 166 159 159 query7 4599301 286 286 query8 250 213 221 213 query9 8492244024242424 query10 435 289 274 274 query11 17728 14953 14930 14930 query12 146 99 103 99 query13 1631368 366 366 query14 11861 672272546722 query15 231 168 166 166 query16 7278456 471 456 query17 1547592 556 556 query18 1169285 283 283 query19 189 157 152 152 query20 115 104 107 104 query21 211 102 104 102 query22 4399428138913891 query23 33923 33250 33382 33250 query24 11926 295829112911 query25 683 400 393 393 query26 1812159 158 158 query27 2775275 282 275 query28 7232202920142014 query29 987 430 427 427 query30 306 155 158 155 query31 997 733 767 733 query32 97 59 58 58 query33 760 300 292 292 query34 930 469 481 469 query35 842 706 729 706 query36 1106951 926 926 query37 155 82 86 82 query38 3942379538143795 query39 1444139114021391 query40 280 126 120 120 query41 53 48 47 47 query42 117 98 103 98 query43 537 472 467 467 query44 1204747 750 747 query45 207 169 165 165 query46 1109781 745 745 query47 1822175917291729 query48 360 295 305 295 query49 1149448 441 441 query50 820 418 424 418 query51 7195676966776677 query52 108 89 94 89 query53 260 191 189 189 query54 1084451 440 440 query55 76 72 76 72 query56 282 248 276 248 query57 1154102410781024 query58 229 231 227 227 query59 3044279726312631 query60 301 274 270 270 query61 111 93 98 93 query62 811 654 667 654 query63 220 191 189 189 query64 6351225317561756 query65 3252316131383138 query66 1254332 343 332 query67 15403 14939 14713 14713 query68 6660554 549 549 query69 660 338 293 293 query70 1233116010391039 query71 555 277 272 272 query72 7600223120142014 query73 816 322 320 320 query74 9361869687988696 query75 4797270926402640 query76 46891089972 972 query77 807 318 316 316 query78 9814908190009000 query79 7289529 548 529 query80 2410509 511 509 query81 588 237 224 224 query82 1176144 138 138 query83 281 150 148 148 query84 266 77 81 77 query85 1408273 268 268 query86 418 312 277 277 query87 4333420742184207 query88 4721230322962296 query89 443 297 297 297 query90 2065206 202 202 query91 121 96 95 95 query92 58 50 50 50 query93 5943545 533 533 query94 990 300 283 283 query95 362 258 258 258 query96 626 273 264 264 query97 3201302630203020 query98 228 240 205 205 query99 1577126212791262 Total cold run time: 317136 ms Total hot run time: 184603 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [fix](ip)pick ip rowstore [doris]
github-actions[bot] commented on PR #39345: URL: https://github.com/apache/doris/pull/39345#issuecomment-2288128487 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
doris-robot commented on PR #39341: URL: https://github.com/apache/doris/pull/39341#issuecomment-2288133250 TPC-DS: Total hot run time: 189073 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 27139502490b745ea24b2f8bd31671dafd61b6a6, data reload: false query1 1243902 865 865 query2 6438193018781878 query3 10599 377137973771 query4 59292 23930 23382 23382 query5 5742502 486 486 query6 497 156 152 152 query7 6509296 297 296 query8 302 210 202 202 query9 8857244224122412 query10 496 256 263 256 query11 18035 15079 15150 15079 query12 162 100 102 100 query13 1592389 374 374 query14 11764 678467316731 query15 243 185 174 174 query16 7608529 495 495 query17 1154583 573 573 query18 2093311 303 303 query19 291 165 151 151 query20 127 123 116 116 query21 217 114 110 110 query22 4396432342504250 query23 34075 33632 33007 33007 query24 5491295428862886 query25 543 408 394 394 query26 687 160 155 155 query27 1816267 274 267 query28 3790204320362036 query29 716 418 409 409 query30 252 152 148 148 query31 902 752 753 752 query32 82 52 54 52 query33 506 283 296 283 query34 852 451 456 451 query35 841 715 725 715 query36 1042927 899 899 query37 134 82 79 79 query38 3974386639043866 query39 1465137513891375 query40 207 116 118 116 query41 48 48 47 47 query42 121 94 96 94 query43 503 458 461 458 query44 1054729 741 729 query45 198 162 164 162 query46 1090776 741 741 query47 1830173317691733 query48 366 286 294 286 query49 766 423 420 420 query50 795 396 410 396 query51 6742670165636563 query52 105 90 91 90 query53 258 186 182 182 query54 572 455 555 455 query55 73 73 71 71 query56 251 242 249 242 query57 1129103310321032 query58 216 240 242 240 query59 2902274828062748 query60 307 270 258 258 query61 94 96 95 95 query62 751 639 670 639 query63 214 179 179 179 query64 3314172217121712 query65 3210314131553141 query66 700 330 329 329 query67 15389 14774 14844 14774 query68 8658554 566 554 query69 606 389 268 268 query70 1504118011591159 query71 536 276 269 269 query72 6520224019361936 query73 2579314 316 314 query74 9549877287908772 query75 4181266427082664 query76 47001015973 973 query77 727 315 359 315 query78 9632914389448944 query79 10347 524 522 522 query80 1510512 486 486 query81 582 225 223 223 query82 658 138 137 137 query83 224 142 146 142 query84 268 72 79 72 query85 1031264 297 264 query86 333 290 281 281 query87 4379412442134124 query88 4558230923182309 query89 506 295 285 285 query90 2298193 189 189 query91 120 93 94 93 query92 63 49 49 49 query93 5621545 539 539 query94 1064287 285 285 query95 352 254 256 254 query96 616 272 269 269 query97 3203306131023061 query98 220 208 196 196 query99 1484126312631263 Total cold run time: 336659 ms Total hot run time: 189073 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [I] [Bug] RoutineLoad重复消费 [doris]
DavidSH92 commented on issue #38551: URL: https://github.com/apache/doris/issues/38551#issuecomment-2288134603 Perhaps you need to upgrade your version. Doris 2.0 and above already support this feature，when you pause your routine task, the offset already consumed will be recorded, and when you restart it, it will be consumed from offsert -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nullif) revert nullif signature change by #38497 [doris]
doris-robot commented on PR #39342: URL: https://github.com/apache/doris/pull/39342#issuecomment-2288135184 TPC-DS: Total hot run time: 189502 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b75b3faa64ede7caa3bbcddb6b68a688d85a3e97, data reload: false query1 1244892 847 847 query2 6422190718521852 query3 10724 402738253825 query4 59268 24143 23271 23271 query5 5815531 518 518 query6 521 177 177 177 query7 6459295 287 287 query8 291 198 193 193 query9 9031242424012401 query10 506 266 264 264 query11 18111 15023 15236 15023 query12 173 100 101 100 query13 1562365 362 362 query14 14756 714776977147 query15 263 172 168 168 query16 7657479 488 479 query17 1148572 564 564 query18 1985295 297 295 query19 279 161 158 158 query20 126 134 113 113 query21 210 105 107 105 query22 4605430642754275 query23 34292 33460 33085 33085 query24 5572287529472875 query25 524 387 381 381 query26 688 153 156 153 query27 1809272 266 266 query28 3897204620322032 query29 681 402 412 402 query30 242 145 148 145 query31 926 760 722 722 query32 83 53 56 53 query33 439 271 284 271 query34 851 460 477 460 query35 829 719 719 719 query36 1083964 921 921 query37 138 80 80 80 query38 3897393437983798 query39 1453138213891382 query40 198 122 124 122 query41 49 46 43 43 query42 121 97 106 97 query43 516 460 486 460 query44 1115743 762 743 query45 195 163 167 163 query46 1105757 737 737 query47 1864179117711771 query48 373 339 295 295 query49 761 410 420 410 query50 798 400 404 400 query51 6717677566526652 query52 104 92 90 90 query53 261 184 182 182 query54 573 465 439 439 query55 75 73 75 73 query56 283 246 254 246 query57 1146107410861074 query58 233 212 223 212 query59 2860263426642634 query60 302 271 277 271 query61 98 97 127 97 query62 732 655 645 645 query63 212 186 182 182 query64 3323170017101700 query65 3210317331413141 query66 669 319 324 319 query67 15152 14884 14737 14737 query68 4500528 547 528 query69 427 265 283 265 query70 1235119111441144 query71 417 273 269 269 query72 6410226520462046 query73 751 318 322 318 query74 9336868986718671 query75 3366270826832683 query76 1965110910521052 query77 542 318 318 318 query78 9724900089988998 query79 2482526 523 523 query80 1239497 491 491 query81 551 229 228 228 query82 982 133 137 133 query83 257 145 148 145 query84 268 76 73 73 query85 1107287 275 275 query86 444 317 310 310 query87 4483418142254181 query88 4335228024182280 query89 405 295 291 291 query90 1987196 198 196 query91 133 109 107 107 query92 64 53 56 53 query93 3081539 528 528 query94 856 313 387 313 query95 344 264 257 257 query96 596 269 268 268 query97 3222300629952995 query98 221 206 203 203 query99 1574127712561256 Total cold run time: 318909 ms Total hot run time: 189502 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
github-actions[bot] commented on PR #39341: URL: https://github.com/apache/doris/pull/39341#issuecomment-2288135488 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
github-actions[bot] commented on PR #39341: URL: https://github.com/apache/doris/pull/39341#issuecomment-2288135561 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) do not run test_index_lowercase_fault_injection in cloud mode [doris]
github-actions[bot] commented on PR #39235: URL: https://github.com/apache/doris/pull/39235#issuecomment-2288136941 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](schema-change) Rebuild distribution info after columns as hash key are modified [doris]
TangSiyang2001 commented on PR #39319: URL: https://github.com/apache/doris/pull/39319#issuecomment-2288136119 run cloud_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) do not run test_index_lowercase_fault_injection in cloud mode [doris]
github-actions[bot] commented on PR #39235: URL: https://github.com/apache/doris/pull/39235#issuecomment-2288136882 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](thirdparty) upgrade arrow to 17.0.0 [doris]
doris-robot commented on PR #38572: URL: https://github.com/apache/doris/pull/38572#issuecomment-2288139463 ClickBench: Total hot run time: 30.08 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a66223089b8600a77ba9495bf7f5457c916adda8, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.230.050.04 query4 1.680.060.07 query5 0.490.470.47 query6 1.130.720.72 query7 0.020.020.02 query8 0.060.040.04 query9 0.540.500.49 query10 0.530.530.54 query11 0.150.110.11 query12 0.150.130.12 query13 0.600.600.58 query14 0.770.760.77 query15 0.850.810.82 query16 0.370.360.39 query17 1.051.061.03 query18 0.220.220.22 query19 1.851.751.74 query20 0.020.010.00 query21 15.40 0.750.64 query22 4.078.181.21 query23 18.26 1.371.30 query24 2.100.230.22 query25 0.160.080.09 query26 0.310.210.20 query27 0.450.230.23 query28 13.29 1.031.00 query29 12.66 3.403.36 query30 0.250.060.04 query31 2.910.400.39 query32 3.240.470.47 query33 2.852.982.96 query34 17.09 4.324.33 query35 4.404.424.40 query36 0.650.480.49 query37 0.190.160.16 query38 0.160.160.15 query39 0.040.030.04 query40 0.150.130.13 query41 0.090.050.05 query42 0.060.060.04 query43 0.040.040.04 Total cold run time: 109.66 s Total hot run time: 30.08 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](cache) File cache async init [doris]
suxiaogang223 commented on PR #39036: URL: https://github.com/apache/doris/pull/39036#issuecomment-2288143962 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) reuse connection in load_stream_stub [doris]
liaoxin01 merged PR #39231: URL: https://github.com/apache/doris/pull/39231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](catalog) set timeout for split fetch [doris]
morningman opened a new pull request, #39346: URL: https://github.com/apache/doris/pull/39346 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function) avoid calculating sqrt of negative in agg function CORR [doris]
doris-robot commented on PR #39324: URL: https://github.com/apache/doris/pull/39324#issuecomment-2288145196 TPC-H: Total hot run time: 37311 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8890c61865bd276564e4998f6b5a57f969b99f21, data reload: false -- Round 1 -- q1 17603 434343074307 q2 2020178 180 178 q3 11885 126211021102 q4 10534 738 757 738 q5 7778279627892789 q6 220 138 139 138 q7 946 593 606 593 q8 9539204420862044 q9 8681652165146514 q10 7021211422562114 q11 445 244 236 236 q12 392 219 214 214 q13 18941 298629502950 q14 295 245 238 238 q15 514 481 467 467 q16 499 393 388 388 q17 961 648 697 648 q18 7265671267546712 q19 5979972 956 956 q20 685 333 347 333 q21 3901263327892633 q22 1126101910211019 Total cold run time: 117230 ms Total hot run time: 37311 ms - Round 2, with runtime_filter_mode=off - q1 4448431742394239 q2 360 264 258 258 q3 2847265525932593 q4 1924166516661665 q5 5694572356545654 q6 231 135 133 133 q7 2158172617401726 q8 3306344634593446 q9 8757874288228742 q10 3602321832723218 q11 606 523 527 523 q12 777 612 588 588 q13 17140 319131583158 q14 332 287 303 287 q15 532 493 505 493 q16 492 463 448 448 q17 1834151615361516 q18 8100796878037803 q19 1792165315701570 q20 191118831883 q21 8845538352725272 q22 1123102310221022 Total cold run time: 77122 ms Total hot run time: 56237 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](catalog) set timeout for split fetch [doris]
doris-robot commented on PR #39346: URL: https://github.com/apache/doris/pull/39346#issuecomment-2288145163 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (73787b68b4d -> a99812c4855)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 73787b68b4d [feature](restore) Support clean_tables/clean_partitions properties for restore job (#39028) add a99812c4855 [improve](move-memtable) reuse connection in load_stream_stub (#39231) No new revisions were added by this update. Summary of changes: be/src/runtime/exec_env.h| 4 be/src/runtime/exec_env_init.cpp | 6 +- be/src/util/brpc_client_cache.cpp| 22 +++--- be/src/util/brpc_client_cache.h | 24 be/src/util/doris_metrics.h | 1 + be/src/vec/sink/load_stream_stub.cpp | 3 +-- be/src/vec/sink/writer/vtablet_writer_v2.cpp | 2 +- 7 files changed, 47 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Compatible with ojdbc6 by adding version check [doris]
doris-robot commented on PR #39341: URL: https://github.com/apache/doris/pull/39341#issuecomment-2288146049 ClickBench: Total hot run time: 30.04 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 27139502490b745ea24b2f8bd31671dafd61b6a6, data reload: false query1 0.040.040.04 query2 0.080.040.04 query3 0.220.050.05 query4 1.680.070.07 query5 0.510.490.47 query6 1.130.720.73 query7 0.010.010.02 query8 0.050.040.04 query9 0.540.480.49 query10 0.540.550.53 query11 0.150.120.12 query12 0.150.110.12 query13 0.620.600.59 query14 0.780.780.78 query15 0.870.820.82 query16 0.360.350.38 query17 1.070.990.99 query18 0.220.220.23 query19 1.821.771.68 query20 0.010.010.01 query21 15.40 0.740.65 query22 4.588.201.19 query23 18.28 1.321.32 query24 2.120.230.21 query25 0.150.090.07 query26 0.290.220.21 query27 0.460.230.23 query28 13.36 1.011.00 query29 12.61 3.423.36 query30 0.230.060.04 query31 2.880.400.40 query32 3.260.490.49 query33 2.972.982.96 query34 16.89 4.354.44 query35 4.414.434.42 query36 0.660.460.47 query37 0.190.160.15 query38 0.160.140.15 query39 0.040.040.03 query40 0.150.130.12 query41 0.090.050.04 query42 0.060.040.05 query43 0.050.040.04 Total cold run time: 110.14 s Total hot run time: 30.04 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function)timediff with now function causes a error signature (… [doris]
doris-robot commented on PR #39347: URL: https://github.com/apache/doris/pull/39347#issuecomment-2288146340 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](function)timediff with now function causes a error signature (… [doris]
Mryange opened a new pull request, #39347: URL: https://github.com/apache/doris/pull/39347 …#39322) https://github.com/apache/doris/pull/39322 ``` mysql [(none)]>select round(timediff(now(),'2024-08-15')/60/60,2); ERROR 1105 (HY000): errCode = 2, detailMessage = argument 1 requires datetimev2 type, however 'now()' is of datetime type ``` The reason is that the function parameter types were modified in expectedInputTypes, which led to no match being found. The code here is from a long time ago. Because the precision of datetimev2 could not be deduced in the past, a separate implementation was made here. This code can be safely deleted. ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function)timediff with now function causes a error signature (… [doris]
Mryange commented on PR #39347: URL: https://github.com/apache/doris/pull/39347#issuecomment-2288146547 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nullif) revert nullif signature change by #38497 [doris]
doris-robot commented on PR #39342: URL: https://github.com/apache/doris/pull/39342#issuecomment-2288148225 ClickBench: Total hot run time: 30.67 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b75b3faa64ede7caa3bbcddb6b68a688d85a3e97, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.230.050.04 query4 1.670.070.07 query5 0.500.490.47 query6 1.120.720.72 query7 0.020.010.01 query8 0.050.040.04 query9 0.550.480.49 query10 0.540.530.54 query11 0.170.120.12 query12 0.160.120.12 query13 0.600.590.59 query14 0.760.810.78 query15 0.870.820.81 query16 0.370.360.36 query17 1.011.011.00 query18 0.230.220.22 query19 1.821.701.68 query20 0.020.010.03 query21 15.39 0.750.65 query22 4.038.111.81 query23 18.22 1.351.26 query24 2.130.230.23 query25 0.160.070.07 query26 0.300.210.21 query27 0.460.230.23 query28 13.23 1.020.99 query29 12.59 3.413.40 query30 0.240.060.05 query31 2.880.400.39 query32 3.290.500.48 query33 2.972.992.93 query34 17.13 4.374.52 query35 4.494.444.45 query36 0.640.490.47 query37 0.180.150.16 query38 0.160.150.16 query39 0.050.030.04 query40 0.150.110.13 query41 0.090.040.05 query42 0.060.050.04 query43 0.040.040.04 Total cold run time: 109.7 s Total hot run time: 30.67 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](move-memtable) add more fault injection cases for writer v2 [doris]
github-actions[bot] commented on PR #39169: URL: https://github.com/apache/doris/pull/39169#issuecomment-2288152928 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](doris-future) Support REGR_SXX_SXY_SYY aggregation functions [doris]
zclllyybb commented on code in PR #39187:
URL: https://github.com/apache/doris/pull/39187#discussion_r1716515897
##
be/src/vec/aggregate_functions/aggregate_function_regr_sxx_.h:
##
@@ -0,0 +1,270 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include
+#include
+#include
+#include
+
+#include "olap/olap_common.h"
+#include "runtime/decimalv2_value.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/columns/column.h"
+#include "vec/columns/column_nullable.h"
+#include "vec/common/assert_cast.h"
+#include "vec/core/field.h"
+#include "vec/core/types.h"
+#include "vec/data_types/data_type_decimal.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/io/io_helper.h"
+namespace doris {
+namespace vectorized {
+class Arena;
+class BufferReadable;
+class BufferWritable;
+template
+class ColumnDecimal;
+template
+class ColumnVector;
+} // namespace vectorized
+} // namespace doris
+
+namespace doris::vectorized {
+
+template
+struct AggregateFunctionRegrSxxData {
+UInt64 count = 0;
+double square_of_sum_x {};
+double sum_x {};
+void write(BufferWritable& buf) const {
+write_binary(square_of_sum_x, buf);
+write_binary(sum_x, buf);
+write_binary(count, buf);
+}
+
+void read(BufferReadable& buf) {
+read_binary(square_of_sum_x, buf);
+read_binary(sum_x, buf);
+read_binary(count, buf);
+}
+
+void reset() {
+square_of_sum_x = {};
+sum_x = {};
+count = 0;
+}
+
+double get_regr_sxx_result() const { return square_of_sum_x - (sum_x *
sum_x) / count; }
+
+void merge(const AggregateFunctionRegrSxxData& rhs) {
+if (rhs.count == 0) {
+return;
+}
+square_of_sum_x += rhs.square_of_sum_x;
+sum_x += rhs.sum_x;
+count += rhs.count;
+}
+
+void add(const IColumn* column_y, const IColumn* column_x, size_t row_num)
{
+#ifdef __clang__
+#pragma clang fp reassociate(on)
+#endif
+const auto& sources_x = assert_cast&>(*column_x);
Review Comment:
in frequently called function `add`, add `TypeCheckOnRelease::DISABLE` to
`assert_cast`'s template arguments. see more in
https://github.com/apache/doris/pull/39030/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](external)add check of engine and catalog types [doris]
doris-robot commented on PR #39343: URL: https://github.com/apache/doris/pull/39343#issuecomment-2288154141 TPC-H: Total hot run time: 37284 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5d2ff84bc5b458990c985b1d6ee979bc05814511, data reload: false -- Round 1 -- q1 17626 432542684268 q2 2004175 177 175 q3 11824 996 1114996 q4 10501 745 710 710 q5 7783277527502750 q6 225 139 135 135 q7 950 582 584 582 q8 9513202019971997 q9 8331649865406498 q10 6992221221732173 q11 447 243 243 243 q12 393 218 213 213 q13 17766 298429892984 q14 278 241 225 225 q15 530 479 487 479 q16 498 393 382 382 q17 962 646 726 646 q18 7319681967356735 q19 7726935 1106935 q20 676 326 316 316 q21 3785284729112847 q22 1085995 1000995 Total cold run time: 117214 ms Total hot run time: 37284 ms - Round 2, with runtime_filter_mode=off - q1 4460419742294197 q2 390 280 271 271 q3 2826257425542554 q4 1917165016721650 q5 5669569655935593 q6 223 134 131 131 q7 2095176717161716 q8 3223350634113411 q9 8768874587508745 q10 3520328232283228 q11 588 498 508 498 q12 843 600 586 586 q13 16960 309431273094 q14 315 283 286 283 q15 546 488 473 473 q16 501 422 436 422 q17 1813155915091509 q18 8107812278237823 q19 1754156915981569 q20 2074186618911866 q21 5472529553245295 q22 1112102110161016 Total cold run time: 73176 ms Total hot run time: 55930 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function)timediff with now function causes a error signature (… [doris]
doris-robot commented on PR #39349: URL: https://github.com/apache/doris/pull/39349#issuecomment-2288154356 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](move-memtable) add more fault injection cases for writer v2 [doris]
github-actions[bot] commented on PR #39169: URL: https://github.com/apache/doris/pull/39169#issuecomment-2288153033 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] There are 5 FE nodes in version 3.0 data. The master can query the external data. If you use non-master query, it will prompt that the field does not exist. [doris]
mpv945 opened a new issue, #39348: URL: https://github.com/apache/doris/issues/39348 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version Doris 3.0.0 ### What's Wrong? Five FE nodes, three FOLLOWER, and two OBSERVER, are loaded through nginx. After creating the data table, it can be queried on the master node, but on the non-master node it will prompt "Unknown column '_id' in 'doris.search_std_' "。 ### What You Expected? I expect to deploy multi-node FE in the new version. After passing the nginx load, I can access and query the data through the nginx load. Rather than entering the master node alone to perform operations! ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](function)timediff with now function causes a error signature (… [doris]
Mryange opened a new pull request, #39349: URL: https://github.com/apache/doris/pull/39349 …#39322) https://github.com/apache/doris/pull/39322 ## Proposed changes ``` mysql [(none)]>select round(timediff(now(),'2024-08-15')/60/60,2); ERROR 1105 (HY000): errCode = 2, detailMessage = argument 1 requires datetimev2 type, however 'now()' is of datetime type ``` The reason is that the function parameter types were modified in expectedInputTypes, which led to no match being found. The code here is from a long time ago. Because the precision of datetimev2 could not be deduced in the past, a separate implementation was made here. This code can be safely deleted. ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
