Re: [PR] [fix](ut) repair segcompaction ut [doris]
zhannngchen commented on PR #38165: URL: https://github.com/apache/doris/pull/38165#issuecomment-2240972000 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ut) repair segcompaction ut [doris]
github-actions[bot] commented on PR #38165: URL: https://github.com/apache/doris/pull/38165#issuecomment-2240975762 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improment]Query queued by be memory [doris]
wangbo merged PR #37559: URL: https://github.com/apache/doris/pull/37559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Improment]Query queued by be memory (#37559)
This is an automated email from the ASF dual-hosted git repository.
wangbo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 45cb1b041ce [Improment]Query queued by be memory (#37559)
45cb1b041ce is described below
commit 45cb1b041ce4671e058a1d09872e0097584f1b29
Author: wangbo
AuthorDate: Sat Jul 20 15:39:19 2024 +0800
[Improment]Query queued by be memory (#37559)
## Proposed changes
Add a mechanism of back pressure based on memory usage.
when any BE's mem usage bigger than ```query_queue_by_be_used_memory```,
query could queue in FE.
---
be/src/service/backend_service.cpp | 10 ++
be/src/service/backend_service.h | 3 +
.../main/java/org/apache/doris/common/Config.java | 11 ++
.../main/java/org/apache/doris/catalog/Env.java| 9 ++
.../main/java/org/apache/doris/qe/Coordinator.java | 20 ++-
.../java/org/apache/doris/qe/QeProcessorImpl.java | 7 +-
.../apache/doris/resource/AdmissionControl.java| 156 +
.../doris/resource/workloadgroup/QueryQueue.java | 90 +++-
.../doris/resource/workloadgroup/QueueToken.java | 36 +++--
.../resource/workloadgroup/WorkloadGroup.java | 6 +-
.../doris/tablefunction/MetadataGenerator.java | 11 +-
.../org/apache/doris/common/GenericPoolTest.java | 7 +
.../apache/doris/utframe/MockedBackendFactory.java | 7 +
gensrc/thrift/BackendService.thrift| 14 ++
.../workload_manager_p0/test_curd_wlg.groovy | 2 +-
15 files changed, 325 insertions(+), 64 deletions(-)
diff --git a/be/src/service/backend_service.cpp
b/be/src/service/backend_service.cpp
index 1d85a2bca69..4effc225110 100644
--- a/be/src/service/backend_service.cpp
+++ b/be/src/service/backend_service.cpp
@@ -1297,4 +1297,14 @@ void
BaseBackendService::get_realtime_exec_status(TGetRealtimeExecStatusResponse
response.__set_report_exec_status_params(*report_exec_status_params);
}
+void BaseBackendService::get_be_resource(TGetBeResourceResult& result,
+ const TGetBeResourceRequest& request)
{
+int64_t mem_usage = PerfCounters::get_vm_rss();
+int64_t mem_limit = MemInfo::mem_limit();
+TGlobalResourceUsage global_resource_usage;
+global_resource_usage.__set_mem_limit(mem_limit);
+global_resource_usage.__set_mem_usage(mem_usage);
+result.__set_global_resource_usage(global_resource_usage);
+}
+
} // namespace doris
diff --git a/be/src/service/backend_service.h b/be/src/service/backend_service.h
index 4a04f16e853..0ada1bf5393 100644
--- a/be/src/service/backend_service.h
+++ b/be/src/service/backend_service.h
@@ -138,6 +138,9 @@ public:
void get_realtime_exec_status(TGetRealtimeExecStatusResponse& response,
const TGetRealtimeExecStatusRequest&
request) override;
+void get_be_resource(TGetBeResourceResult& result,
+ const TGetBeResourceRequest& request) override;
+
// begin cloud backend functions
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 39e8e4f1c17..7af0da4f9d6 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -1767,6 +1767,17 @@ public class Config extends ConfigBase {
@ConfField(mutable = true, varType = VariableAnnotation.EXPERIMENTAL)
public static boolean enable_cpu_hard_limit = false;
+@ConfField(mutable = true, description = {
+"当BE内存用量大于该值时，查询会进入排队逻辑，默认值为-1，代表该值不生效。取值范围0~1的小数",
+"When be memory usage bigger than this value, query could queue, "
++ "default value is -1, means this value not work. Decimal
value range from 0 to 1"})
+public static double query_queue_by_be_used_memory = -1;
+
+@ConfField(mutable = true, description = {"基于内存反压场景FE定时拉取BE内存用量的时间间隔",
+"In the scenario of memory backpressure, "
++ "the time interval for obtaining BE memory usage at
regular intervals"})
+public static long get_be_resource_usage_interval_ms = 1;
+
@ConfField(mutable = false, masterOnly = true)
public static int backend_rpc_timeout_ms = 6; // 1 min
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index c999024d359..49157b19ffb 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -242,6 +242,7 @@ import org.apache.doris.qe.JournalObservable;
import
Re: [PR] [Enhancement](group commit)Optimize be select for group commit [doris]
dataroaring merged PR #35558: URL: https://github.com/apache/doris/pull/35558 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Enhancement](group commit)Optimize be select for group commit (#35558)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 99d5bcc2bba [Enhancement](group commit)Optimize be select for group
commit (#35558)
99d5bcc2bba is described below
commit 99d5bcc2bbacc2bc02e64ee08573fb31cb873a61
Author: abmdocrt
AuthorDate: Sat Jul 20 15:46:49 2024 +0800
[Enhancement](group commit)Optimize be select for group commit (#35558)
1. Streamload and insert into, if batched and sent to the master FE,
should use a consistent BE strategy (previously, insert into reused the
first selected BE, while streamload used round robin). First, a map
records a fixed be id for a certain table. The first
time a table is imported, a BE is randomly selected, and this table id
and be id are recorded in the map permanently. Subsequently, all data
imported into this table will select the BE corresponding to the table
id recorded in the map. This ensures that batching is maximized to a
single BE.
To address the issue of excessive load on a single BE, a variable
similar to a bvar window is used to monitor the total data volume sent
to a specific BE for a specific table during the batch interval (default
10 seconds). A second map is used to track
this. If a new import finds that its corresponding BE's window variable
is less than a certain value (e.g., 1G), the new import continues to be
sent to the corresponding BE according to map1. If it exceeds this
value, the new import is sent to another BE with the smallest window
variable value, and map1 is updated. If every BE exceeds this value, the
one with the smallest value is still chosen. This helps to alleviate
excessive pressure on a single BE.
2. For streamload, if batched and sent to a BE, it will batch directly
on this BE and will commit the transaction at the end of the import. At
this point, a request is sent to the FE, which records the size of this
import and adds it to the window variable.
3. Streamload sent to observer FE, as well as insert into sent to
observer FE, follow the logic in 1 by RPC, passing the table id to the
master FE to obtain the selected be id.
---
be/src/runtime/group_commit_mgr.cpp| 7 +
.../cloud/planner/CloudGroupCommitPlanner.java | 43 +
.../doris/common/util/SlidingWindowCounter.java| 73
.../org/apache/doris/httpv2/rest/LoadAction.java | 42 +++--
.../org/apache/doris/load/GroupCommitManager.java | 196 +
.../insert/OlapGroupCommitInsertExecutor.java | 13 ++
.../apache/doris/planner/GroupCommitPlanner.java | 29 +--
.../main/java/org/apache/doris/qe/Coordinator.java | 25 ++-
.../java/org/apache/doris/qe/MasterOpExecutor.java | 49 ++
.../apache/doris/service/FrontendServiceImpl.java | 31
gensrc/thrift/FrontendService.thrift | 15 ++
11 files changed, 446 insertions(+), 77 deletions(-)
diff --git a/be/src/runtime/group_commit_mgr.cpp
b/be/src/runtime/group_commit_mgr.cpp
index 5f989da023b..30885fa1ac9 100644
--- a/be/src/runtime/group_commit_mgr.cpp
+++ b/be/src/runtime/group_commit_mgr.cpp
@@ -442,6 +442,13 @@ Status GroupCommitTable::_finish_group_commit_load(int64_t
db_id, int64_t table_
request.__set_db_id(db_id);
request.__set_table_id(table_id);
request.__set_txnId(txn_id);
+request.__set_groupCommit(true);
+request.__set_receiveBytes(state->num_bytes_load_total());
+if (_exec_env->master_info()->__isset.backend_id) {
+request.__set_backendId(_exec_env->master_info()->backend_id);
+} else {
+LOG(WARNING) << "_exec_env->master_info not set backend_id";
+}
if (state) {
request.__set_commitInfos(state->tablet_commit_infos());
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/cloud/planner/CloudGroupCommitPlanner.java
b/fe/fe-core/src/main/java/org/apache/doris/cloud/planner/CloudGroupCommitPlanner.java
index 6978ec65e72..782f78e6bc4 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/cloud/planner/CloudGroupCommitPlanner.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/cloud/planner/CloudGroupCommitPlanner.java
@@ -20,24 +20,18 @@ package org.apache.doris.cloud.planner;
import org.apache.doris.catalog.Database;
import org.apache.doris.catalog.Env;
import org.apache.doris.catalog.OlapTable;
-import org.apache.doris.cloud.system.CloudSystemInfoService;
import org.apache.doris.common.DdlException;
-import org.apache.doris.common.ErrorCode;
-import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.LoadException;
import org.apache.doris.common.UserException;
import org.apache.doris.planner.GroupCommitPlanner;
import org.apache.doris.qe
Re: [PR] [fix](ut) repair segcompaction ut [doris]
doris-robot commented on PR #38165: URL: https://github.com/apache/doris/pull/38165#issuecomment-2241009006 TPC-H: Total hot run time: 39622 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 03eb24e2f48b7f6edb4d71c71f4b3f869c986999, data reload: false -- Round 1 -- q1 17633 425642184218 q2 2013194 186 186 q3 10438 115611031103 q4 10177 785 771 771 q5 7568267427172674 q6 216 135 139 135 q7 958 596 589 589 q8 9217206020432043 q9 8853652365356523 q10 8897375737663757 q11 449 237 237 237 q12 481 224 216 216 q13 18844 295830232958 q14 279 224 241 224 q15 524 484 482 482 q16 493 388 390 388 q17 956 625 680 625 q18 7971738773597359 q19 6668142814071407 q20 675 320 331 320 q21 4854313031443130 q22 344 290 277 277 Total cold run time: 118508 ms Total hot run time: 39622 ms - Round 2, with runtime_filter_mode=off - q1 4455422341914191 q2 368 265 264 264 q3 2986286329572863 q4 2011168817701688 q5 5613549454645464 q6 224 143 130 130 q7 2237187418071807 q8 3231339233733373 q9 8703877287918772 q10 4077379937953795 q11 598 503 532 503 q12 798 621 641 621 q13 15845 322131563156 q14 321 291 298 291 q15 516 493 494 493 q16 504 449 454 449 q17 1828151014461446 q18 8035776577207720 q19 1765166416861664 q20 2476188918581858 q21 8270490146884688 q22 584 510 532 510 Total cold run time: 75445 ms Total hot run time: 55746 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ut) repair segcompaction ut [doris]
doris-robot commented on PR #38165: URL: https://github.com/apache/doris/pull/38165#issuecomment-2241023077 TPC-DS: Total hot run time: 174086 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 03eb24e2f48b7f6edb4d71c71f4b3f869c986999, data reload: false query1 911 388 380 380 query2 6429189917571757 query3 6641210 218 210 query4 28427 17546 17596 17546 query5 3621472 485 472 query6 275 221 187 187 query7 4575288 292 288 query8 245 200 195 195 query9 8528245424312431 query10 456 280 290 280 query11 11839 10078 99979997 query12 120 87 85 85 query13 1646373 382 373 query14 10322 784571947194 query15 223 168 172 168 query16 7707452 462 452 query17 1587540 536 536 query18 1955287 283 283 query19 202 148 149 148 query20 93 84 85 84 query21 210 131 128 128 query22 4418414640274027 query23 34218 33637 33685 33637 query24 11060 292829612928 query25 588 382 414 382 query26 1026158 155 155 query27 2259282 277 277 query28 6696205720802057 query29 940 664 674 664 query30 271 158 161 158 query31 976 764 752 752 query32 94 53 55 53 query33 757 342 339 339 query34 982 501 523 501 query35 868 775 772 772 query36 11661007978 978 query37 139 85 83 83 query38 2943293428352835 query39 918 900 856 856 query40 224 119 117 117 query41 44 43 44 43 query42 114 102 93 93 query43 491 459 475 459 query44 1223730 734 730 query45 191 159 162 159 query46 1095729 747 729 query47 1863176417621762 query48 374 299 289 289 query49 839 407 413 407 query50 776 393 400 393 query51 7024690268086808 query52 104 98 92 92 query53 364 295 293 293 query54 906 478 449 449 query55 74 73 77 73 query56 291 273 268 268 query57 1154104210691042 query58 255 247 269 247 query59 2744274524662466 query60 304 287 288 287 query61 97 96 95 95 query62 820 658 655 655 query63 316 288 287 287 query64 9682226516841684 query65 3227314931323132 query66 819 329 332 329 query67 15389 15026 14974 14974 query68 4642554 536 536 query69 641 449 358 358 query70 1177108210571057 query71 422 280 280 280 query72 7268549254755475 query73 774 324 321 321 query74 6050566756905667 query75 3505271427112711 query76 2772960 928 928 query77 633 320 310 310 query78 10544 923290989098 query79 4482539 531 531 query80 1227487 488 487 query81 602 233 238 233 query82 767 144 146 144 query83 209 171 171 171 query84 274 94 88 88 query85 1342317 301 301 query86 403 323 315 315 query87 3313312130663066 query88 4755238123692369 query89 509 394 385 385 query90 1823193 193 193 query91 131 103 102 102 query92 62 48 52 48 query93 5118510 511 510 query94 954 292 299 292 query95 412 323 377 323 query96 603 271 269 269 query97 3212302330363023 query98 223 208 204 204 query99 1599128113071281 Total cold run time: 287583 ms Total hot run time: 174086 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Bug](function) fix mod function cause core dump [doris]
yiguolei merged PR #37999: URL: https://github.com/apache/doris/pull/37999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Bug](function) fix mod function cause core dump (#37999)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 287f5ca2de7 [Bug](function) fix mod function cause core dump (#37999)
287f5ca2de7 is described below
commit 287f5ca2de77a4b0a7c526b11d2d4892912c1ef6
Author: zhangstar333 <87313068+zhangstar...@users.noreply.github.com>
AuthorDate: Sat Jul 20 16:00:09 2024 +0800
[Bug](function) fix mod function cause core dump (#37999)
## Proposed changes
as (-9223372036854775808 % -1) will cause coredump directly by fpe,
so check this case and throw exception.
---
be/src/vec/functions/modulo.cpp| 18
.../sql_functions/math_functions/test_mod.out | 6 +++
.../sql_functions/math_functions/test_mod.groovy | 49 ++
3 files changed, 73 insertions(+)
diff --git a/be/src/vec/functions/modulo.cpp b/be/src/vec/functions/modulo.cpp
index 66376b66019..7a2dfc004ef 100644
--- a/be/src/vec/functions/modulo.cpp
+++ b/be/src/vec/functions/modulo.cpp
@@ -33,6 +33,20 @@
namespace doris::vectorized {
+template
+inline void throw_if_division_leads_to_FPE(A a, B b) {
+// http://avva.livejournal.com/2548306.html
+// (-9223372036854775808 % -1) will cause coredump directly, so check this
case to throw exception, or maybe could return 0 as result
+if constexpr (std::is_signed_v && std::is_signed_v) {
+if (b == -1 && a == std::numeric_limits::min()) {
+throw Exception(ErrorCode::INVALID_ARGUMENT,
+"Division of minimal signed number by minus one is
an undefined "
+"behavior, {} % {}. ",
+a, b);
+}
+}
+}
+
template
struct ModuloImpl {
using ResultType = typename NumberTraits::ResultOfModulo::Type;
@@ -51,6 +65,7 @@ struct ModuloImpl {
if constexpr (std::is_floating_point_v) {
c[i] = std::fmod((double)a[i], (double)b);
} else {
+throw_if_division_leads_to_FPE(a[i], b);
c[i] = a[i] % b;
}
}
@@ -65,6 +80,7 @@ struct ModuloImpl {
if constexpr (std::is_floating_point_v) {
return std::fmod((double)a, (double)b);
} else {
+throw_if_division_leads_to_FPE(a, b);
return a % b;
}
}
@@ -94,6 +110,7 @@ struct PModuloImpl {
if constexpr (std::is_floating_point_v) {
c[i] = std::fmod(std::fmod((double)a[i], (double)b) +
(double)b, double(b));
} else {
+throw_if_division_leads_to_FPE(a[i], b);
c[i] = (a[i] % b + b) % b;
}
}
@@ -108,6 +125,7 @@ struct PModuloImpl {
if constexpr (std::is_floating_point_v) {
return std::fmod(std::fmod((double)a, (double)b) + (double)b,
(double)b);
} else {
+throw_if_division_leads_to_FPE(a, b);
return (a % b + b) % b;
}
}
diff --git
a/regression-test/data/query_p0/sql_functions/math_functions/test_mod.out
b/regression-test/data/query_p0/sql_functions/math_functions/test_mod.out
new file mode 100644
index 000..6e3ce6219de
--- /dev/null
+++ b/regression-test/data/query_p0/sql_functions/math_functions/test_mod.out
@@ -0,0 +1,6 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !sql --
+-21474836484 -1
+1 2 3
+5 -9223372036854775808-1
+
diff --git
a/regression-test/suites/query_p0/sql_functions/math_functions/test_mod.groovy
b/regression-test/suites/query_p0/sql_functions/math_functions/test_mod.groovy
new file mode 100644
index 000..027983ff749
--- /dev/null
+++
b/regression-test/suites/query_p0/sql_functions/math_functions/test_mod.groovy
@@ -0,0 +1,49 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
Re: [PR] [fix](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth [doris]
yiguolei commented on code in PR #38168:
URL: https://github.com/apache/doris/pull/38168#discussion_r1685308773
##
be/src/runtime/workload_group/workload_group.h:
##
@@ -186,7 +167,7 @@ class WorkloadGroup : public
std::enable_shared_from_this {
// new query can not submit
// waiting running query to be cancelled or finish
bool _is_shutdown = false;
-std::unordered_map> _query_ctxs;
Review Comment:
这个不要改，我们需要在wg 中保存query ctx的weak ptr。
因为我们总是需要两条路径，通过queryctx 找到 wg；从wg 找到所有的query。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ut) repair segcompaction ut [doris]
doris-robot commented on PR #38165: URL: https://github.com/apache/doris/pull/38165#issuecomment-2241027661 ClickBench: Total hot run time: 30.59 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 03eb24e2f48b7f6edb4d71c71f4b3f869c986999, data reload: false query1 0.040.040.04 query2 0.080.040.04 query3 0.230.050.04 query4 1.680.070.07 query5 0.480.480.50 query6 1.140.720.71 query7 0.020.020.01 query8 0.040.040.04 query9 0.560.490.48 query10 0.520.560.55 query11 0.160.110.11 query12 0.150.120.12 query13 0.610.590.59 query14 0.770.810.76 query15 0.860.810.80 query16 0.380.340.37 query17 0.981.041.03 query18 0.220.220.22 query19 1.911.741.72 query20 0.010.010.02 query21 15.68 0.770.66 query22 4.867.101.83 query23 18.32 1.341.34 query24 2.110.230.22 query25 0.150.100.09 query26 0.290.210.21 query27 0.460.230.23 query28 13.39 1.010.99 query29 12.60 3.283.26 query30 0.260.060.06 query31 2.860.390.39 query32 3.270.470.47 query33 2.942.952.91 query34 17.15 4.384.36 query35 4.384.384.42 query36 0.660.460.47 query37 0.190.150.15 query38 0.150.150.15 query39 0.040.040.03 query40 0.150.130.12 query41 0.100.040.04 query42 0.060.050.05 query43 0.040.030.04 Total cold run time: 110.95 s Total hot run time: 30.59 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth [doris]
doris-robot commented on PR #38168: URL: https://github.com/apache/doris/pull/38168#issuecomment-2241033695 TPC-H: Total hot run time: 40058 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit bdd0a8d16d08c563841b74a3ecdb41188a65b3b8, data reload: false -- Round 1 -- q1 17619 440842604260 q2 2012187 186 186 q3 10456 122411091109 q4 10182 787 865 787 q5 7534271926922692 q6 224 139 138 138 q7 959 601 596 596 q8 9215211720842084 q9 8843657965356535 q10 8758380837623762 q11 465 241 237 237 q12 416 234 228 228 q13 19077 296829622962 q14 291 236 245 236 q15 544 475 478 475 q16 510 390 386 386 q17 993 695 709 695 q18 8159751575327515 q19 7175135213171317 q20 687 333 334 333 q21 4811324532453245 q22 337 285 280 280 Total cold run time: 119267 ms Total hot run time: 40058 ms - Round 2, with runtime_filter_mode=off - q1 4374421642024202 q2 360 265 256 256 q3 3021285829802858 q4 1975171017231710 q5 5603557254785478 q6 230 134 134 134 q7 2243184618691846 q8 3284348734373437 q9 8826893288618861 q10 4105377038813770 q11 603 494 489 489 q12 801 642 631 631 q13 17193 316431853164 q14 306 294 287 287 q15 531 477 468 468 q16 507 434 434 434 q17 1821153914971497 q18 8186794378007800 q19 1818170914601460 q20 3431188218761876 q21 10975 485346924692 q22 573 474 507 474 Total cold run time: 80766 ms Total hot run time: 55824 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth [doris]
doris-robot commented on PR #38168: URL: https://github.com/apache/doris/pull/38168#issuecomment-2241042149 TPC-DS: Total hot run time: 172975 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit bdd0a8d16d08c563841b74a3ecdb41188a65b3b8, data reload: false query1 911 382 367 367 query2 6454183018351830 query3 6648203 216 203 query4 28576 17447 17221 17221 query5 3580483 464 464 query6 279 173 167 167 query7 4583297 292 292 query8 250 196 195 195 query9 8562245924382438 query10 426 265 269 265 query11 10521 1 10050 1 query12 113 83 86 83 query13 1645364 363 363 query14 10046 775375577557 query15 210 162 164 162 query16 7759506 479 479 query17 1531532 537 532 query18 1943276 272 272 query19 189 148 149 148 query20 87 80 78 78 query21 204 128 121 121 query22 4404408740304030 query23 34002 33771 33481 33481 query24 8587291828682868 query25 597 391 395 391 query26 708 163 153 153 query27 2223290 276 276 query28 5418208220722072 query29 885 616 628 616 query30 255 154 158 154 query31 962 732 760 732 query32 94 52 54 52 query33 608 327 321 321 query34 914 511 490 490 query35 895 752 777 752 query36 1126985 959 959 query37 144 83 81 81 query38 2984284727902790 query39 910 811 803 803 query40 203 118 113 113 query41 45 48 42 42 query42 114 98 100 98 query43 512 470 476 470 query44 1105730 734 730 query45 195 156 156 156 query46 1091700 721 700 query47 1842182117831783 query48 377 290 292 290 query49 834 422 403 403 query50 769 384 395 384 query51 6915676967226722 query52 99 96 92 92 query53 354 293 285 285 query54 652 443 445 443 query55 76 77 77 77 query56 325 320 259 259 query57 1105105810781058 query58 250 259 262 259 query59 2973257525412541 query60 299 276 278 276 query61 94 96 95 95 query62 801 633 652 633 query63 324 285 290 285 query64 9107220217361736 query65 3144306630723066 query66 781 337 340 337 query67 15422 14993 14995 14993 query68 4500555 552 552 query69 479 338 361 338 query70 1158104311031043 query71 380 283 275 275 query72 7100521058215210 query73 758 329 326 326 query74 6145571256655665 query75 3377270826782678 query76 2125915 903 903 query77 431 313 311 311 query78 9768893688648864 query79 4075537 542 537 query80 3022491 490 490 query81 583 235 223 223 query82 1053134 131 131 query83 309 178 176 176 query84 286 87 97 87 query85 2110478 297 297 query86 487 306 288 288 query87 3279309431283094 query88 4328237523592359 query89 502 376 375 375 query90 1817193 195 193 query91 130 97 97 97 query92 61 47 49 47 query93 5720514 510 510 query94 1265275 289 275 query95 463 308 312 308 query96 613 269 268 268 query97 3197301430053005 query98 227 195 191 191 query99 1573127512831275 Total cold run time: 281362 ms Total hot run time: 172975 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth [doris]
doris-robot commented on PR #38168: URL: https://github.com/apache/doris/pull/38168#issuecomment-2241043404 ClickBench: Total hot run time: 30.68 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit bdd0a8d16d08c563841b74a3ecdb41188a65b3b8, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.220.040.05 query4 1.680.070.08 query5 0.500.470.49 query6 1.130.730.72 query7 0.020.020.02 query8 0.050.040.05 query9 0.550.500.50 query10 0.540.540.54 query11 0.150.110.12 query12 0.150.120.13 query13 0.610.580.58 query14 0.770.780.76 query15 0.870.810.84 query16 0.360.360.35 query17 0.970.950.97 query18 0.220.220.22 query19 1.841.761.67 query20 0.010.010.01 query21 15.40 0.740.66 query22 4.746.212.22 query23 18.28 1.471.25 query24 2.070.230.22 query25 0.160.080.08 query26 0.300.200.21 query27 0.450.230.23 query28 13.26 1.020.99 query29 12.61 3.323.30 query30 0.250.060.06 query31 2.860.390.38 query32 3.310.480.47 query33 2.852.882.84 query34 17.05 4.324.31 query35 4.424.394.55 query36 0.650.460.47 query37 0.180.150.16 query38 0.150.140.15 query39 0.040.040.03 query40 0.140.120.12 query41 0.090.050.05 query42 0.050.050.05 query43 0.040.030.04 Total cold run time: 110.11 s Total hot run time: 30.68 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] 根据官网教程run-docker-cluster，构建2.1.4的fe镜像，启动报错standard_init_linux.go:211: exec user process caused "no such file or directory" [doris]
aidfsoft closed issue #37825: [Bug] 根据官网教程run-docker-cluster，构建2.1.4的fe镜像，启动报错standard_init_linux.go:211: exec user process caused "no such file or directory" URL: https://github.com/apache/doris/issues/37825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
Mryange commented on PR #38169: URL: https://github.com/apache/doris/pull/38169#issuecomment-2241057571 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
doris-robot commented on PR #38169: URL: https://github.com/apache/doris/pull/38169#issuecomment-2241057560 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [only test] [doris]
Mryange opened a new pull request, #38169: URL: https://github.com/apache/doris/pull/38169 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
github-actions[bot] commented on PR #38169: URL: https://github.com/apache/doris/pull/38169#issuecomment-2241059568 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](inverted index) String type inverted index match function completion [doris]
doris-robot commented on PR #38170: URL: https://github.com/apache/doris/pull/38170#issuecomment-2241068164 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](inverted index) String type inverted index match function completion [doris]
zzzxl1993 commented on PR #38170: URL: https://github.com/apache/doris/pull/38170#issuecomment-2241068185 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feature](inverted index) String type inverted index match function completion [doris]
zzzxl1993 opened a new pull request, #38170: URL: https://github.com/apache/doris/pull/38170 ## Proposed changes 1. Inverted index of string type supports match_phrase_prefix and match_regexp. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](inverted index) String type inverted index match function completion [doris]
github-actions[bot] commented on PR #38170: URL: https://github.com/apache/doris/pull/38170#issuecomment-2241070034 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
doris-robot commented on PR #38169: URL: https://github.com/apache/doris/pull/38169#issuecomment-2241077008 TPC-H: Total hot run time: 40102 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit cfcd81f7890c68ba97a3f8eead52ec5a17d184ec, data reload: false -- Round 1 -- q1 17618 452243444344 q2 2017192 185 185 q3 10457 117211381138 q4 10196 852 858 852 q5 7525271627212716 q6 219 138 140 138 q7 964 613 606 606 q8 9221209321302093 q9 8907662565446544 q10 8799376237543754 q11 458 248 240 240 q12 411 222 230 222 q13 17920 299929832983 q14 287 235 238 235 q15 516 494 488 488 q16 506 375 382 375 q17 978 858 665 665 q18 8109735973047304 q19 7110143814601438 q20 686 326 330 326 q21 4912317031843170 q22 356 296 286 286 Total cold run time: 118172 ms Total hot run time: 40102 ms - Round 2, with runtime_filter_mode=off - q1 4379429242794279 q2 372 283 267 267 q3 2984292629432926 q4 2050172417421724 q5 5607557755015501 q6 221 133 133 133 q7 2239188318521852 q8 3331345934513451 q9 8799886988958869 q10 4120367739533677 q11 587 491 515 491 q12 828 614 648 614 q13 16018 315331993153 q14 305 287 301 287 q15 541 482 491 482 q16 484 452 438 438 q17 1805154814991499 q18 8106810478647864 q19 1837160816731608 q20 2214185418831854 q21 5179490047334733 q22 618 524 492 492 Total cold run time: 72624 ms Total hot run time: 56194 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [only test] [doris]
doris-robot commented on PR #38169: URL: https://github.com/apache/doris/pull/38169#issuecomment-2241080022 TPC-DS: Total hot run time: 173550 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit cfcd81f7890c68ba97a3f8eead52ec5a17d184ec, data reload: false query1 904 374 368 368 query2 6434193917691769 query3 6651206 217 206 query4 28491 17346 17179 17179 query5 3716474 503 474 query6 299 179 156 156 query7 4580284 291 284 query8 237 195 189 189 query9 8738246524362436 query10 424 297 259 259 query11 11567 994099599940 query12 116 81 79 79 query13 1632360 361 360 query14 10121 754476367544 query15 223 167 169 167 query16 7756451 411 411 query17 1133546 518 518 query18 1955293 276 276 query19 193 147 150 147 query20 91 80 79 79 query21 209 128 126 126 query22 4358407339073907 query23 34091 33680 33752 33680 query24 11213 285329652853 query25 602 376 399 376 query26 710 155 156 155 query27 2260277 277 277 query28 5878210220822082 query29 902 681 681 681 query30 249 163 157 157 query31 1002752 758 752 query32 96 52 57 52 query33 669 326 319 319 query34 881 501 500 500 query35 898 770 763 763 query36 1130970 989 970 query37 140 77 85 77 query38 2919281127552755 query39 843 833 824 824 query40 199 119 121 119 query41 44 42 45 42 query42 115 99 95 95 query43 497 463 464 463 query44 1061732 741 732 query45 191 163 161 161 query46 1088722 737 722 query47 1869177417741774 query48 385 292 319 292 query49 821 413 409 409 query50 759 384 389 384 query51 6904681567826782 query52 107 88 99 88 query53 346 284 294 284 query54 867 452 436 436 query55 77 73 72 72 query56 279 262 285 262 query57 1144107510481048 query58 254 245 251 245 query59 2849271627592716 query60 305 271 280 271 query61 94 92 92 92 query62 774 620 666 620 query63 312 282 283 282 query64 9226228517321732 query65 3197312031243120 query66 738 338 346 338 query67 15636 15227 15012 15012 query68 4495548 546 546 query69 552 411 347 347 query70 1126106711661067 query71 386 281 281 281 query72 7076593455195519 query73 732 325 324 324 query74 6025571756195619 query75 3385267426902674 query76 2210962 974 962 query77 460 309 323 309 query78 9768921588828882 query79 6365538 534 534 query80 3105490 491 490 query81 594 231 226 226 query82 1535141 134 134 query83 326 180 172 172 query84 286 215 83 83 query85 1827305 335 305 query86 483 315 321 315 query87 3282309030883088 query88 5043237523702370 query89 497 382 378 378 query90 1813192 190 190 query91 131 98 101 98 query92 62 50 49 49 query93 5642522 508 508 query94 1202281 286 281 query95 410 319 314 314 query96 608 272 271 271 query97 3193304630313031 query98 218 207 205 205 query99 1583123813011238 Total cold run time: 288869 ms Total hot run time: 173550 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [PR] [only test] [doris]
doris-robot commented on PR #38169: URL: https://github.com/apache/doris/pull/38169#issuecomment-2241081526 ClickBench: Total hot run time: 30.76 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit cfcd81f7890c68ba97a3f8eead52ec5a17d184ec, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.220.040.05 query4 1.690.080.07 query5 0.510.470.48 query6 1.140.720.72 query7 0.020.010.01 query8 0.050.040.05 query9 0.560.500.49 query10 0.560.550.55 query11 0.150.110.11 query12 0.150.130.12 query13 0.610.590.59 query14 0.770.770.78 query15 0.860.820.82 query16 0.360.370.37 query17 0.980.990.97 query18 0.240.220.22 query19 1.761.801.71 query20 0.010.010.01 query21 15.40 0.720.65 query22 4.347.462.13 query23 18.33 1.331.28 query24 2.160.230.22 query25 0.150.080.08 query26 0.300.220.21 query27 0.450.230.23 query28 13.21 1.021.00 query29 12.57 3.333.29 query30 0.250.060.06 query31 2.880.390.40 query32 3.250.470.46 query33 2.912.922.88 query34 16.96 4.324.31 query35 4.404.384.39 query36 0.660.470.47 query37 0.200.150.15 query38 0.150.140.14 query39 0.040.040.03 query40 0.150.120.12 query41 0.090.050.05 query42 0.050.050.04 query43 0.040.040.04 Total cold run time: 109.7 s Total hot run time: 30.76 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ut) repair segcompaction ut [doris]
github-actions[bot] commented on PR #38165: URL: https://github.com/apache/doris/pull/38165#issuecomment-2241082898 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](ut) repair segcompaction ut [doris]
github-actions[bot] commented on PR #38165: URL: https://github.com/apache/doris/pull/38165#issuecomment-2241082913 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](inverted index) String type inverted index match function completion [doris]
doris-robot commented on PR #38170: URL: https://github.com/apache/doris/pull/38170#issuecomment-2241091253 TPC-H: Total hot run time: 40148 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 11f71ef17bfd66a96a840faff8b306723535dc0d, data reload: false -- Round 1 -- q1 17630 456442984298 q2 2019197 189 189 q3 10439 121711351135 q4 10183 855 837 837 q5 7558274927092709 q6 221 134 137 134 q7 955 608 612 608 q8 9224207421352074 q9 8957654965626549 q10 8855377838213778 q11 454 239 235 235 q12 402 222 232 222 q13 18622 298530052985 q14 276 231 236 231 q15 541 480 481 480 q16 504 381 385 381 q17 977 653 778 653 q18 8176751074737473 q19 6754141014111410 q20 698 314 327 314 q21 4854316632293166 q22 356 287 290 287 Total cold run time: 118655 ms Total hot run time: 40148 ms - Round 2, with runtime_filter_mode=off - q1 4356425942434243 q2 389 263 265 263 q3 3097286430162864 q4 1970175317431743 q5 5581559854215421 q6 221 129 137 129 q7 2278190318431843 q8 3303346534123412 q9 8792889488398839 q10 4116376439663764 q11 593 494 511 494 q12 810 618 618 618 q13 16291 317832333178 q14 347 299 279 279 q15 528 502 493 493 q16 519 443 439 439 q17 1830152515341525 q18 8137796178297829 q19 1724147816271478 q20 190818741874 q21 5096470848324708 q22 577 525 521 521 Total cold run time: 72777 ms Total hot run time: 55957 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) refactor stream load regression test case [doris]
github-actions[bot] commented on PR #38158: URL: https://github.com/apache/doris/pull/38158#issuecomment-2241092711 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) refactor stream load regression test case [doris]
github-actions[bot] commented on PR #38158: URL: https://github.com/apache/doris/pull/38158#issuecomment-2241092716 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](inverted index) String type inverted index match function completion [doris]
doris-robot commented on PR #38170: URL: https://github.com/apache/doris/pull/38170#issuecomment-2241095105 TPC-DS: Total hot run time: 174108 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 11f71ef17bfd66a96a840faff8b306723535dc0d, data reload: false query1 922 371 368 368 query2 6420178617871786 query3 6625205 218 205 query4 28396 17482 17315 17315 query5 3700497 478 478 query6 280 168 179 168 query7 4595283 282 282 query8 239 193 193 193 query9 8583240023862386 query10 441 293 277 277 query11 10580 10190 10112 10112 query12 118 84 86 84 query13 1661378 368 368 query14 10278 770278867702 query15 235 172 170 170 query16 7639496 452 452 query17 1580593 586 586 query18 1687272 275 272 query19 198 146 149 146 query20 91 82 82 82 query21 210 125 130 125 query22 4349396539323932 query23 34199 33671 33549 33549 query24 11023 293428952895 query25 653 377 392 377 query26 1138155 150 150 query27 2424274 274 274 query28 6727207120682068 query29 878 639 656 639 query30 265 155 159 155 query31 990 749 770 749 query32 90 56 55 55 query33 767 345 335 335 query34 906 497 506 497 query35 898 754 749 749 query36 1141960 1001960 query37 152 87 90 87 query38 3011294128282828 query39 924 850 856 850 query40 211 122 122 122 query41 47 44 43 43 query42 117 99 98 98 query43 506 474 465 465 query44 1198730 728 728 query45 194 160 159 159 query46 718 712 712 query47 1867175617681756 query48 377 295 291 291 query49 851 416 427 416 query50 794 394 400 394 query51 6878684067876787 query52 111 89 98 89 query53 359 291 297 291 query54 878 451 438 438 query55 76 74 75 74 query56 288 260 275 260 query57 1112101410371014 query58 258 246 256 246 query59 2728258727122587 query60 299 284 285 284 query61 109 94 128 94 query62 788 652 658 652 query63 328 288 284 284 query64 9401225616841684 query65 3187311631003100 query66 742 342 335 335 query67 15402 15068 14957 14957 query68 4483535 543 535 query69 563 430 369 369 query70 1137113611711136 query71 414 294 277 277 query72 7141564857075648 query73 763 323 331 323 query74 6287573256515651 query75 3552277726942694 query76 2457975 952 952 query77 468 307 316 307 query78 9831888210020 8882 query79 2048534 519 519 query80 2277487 494 487 query81 598 223 228 223 query82 606 142 142 142 query83 300 168 172 168 query84 272 83 85 83 query85 2004313 301 301 query86 481 309 332 309 query87 3264314131053105 query88 4119237423792374 query89 466 372 400 372 query90 1799200 200 200 query91 129 174 100 100 query92 63 54 52 52 query93 1710506 501 501 query94 1182297 295 295 query95 414 326 325 325 query96 599 276 276 276 query97 3210300930523009 query98 236 205 188 188 query99 1510123912711239 Total cold run time: 279613 ms Total hot run time: 174108 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](inverted index) String type inverted index match function completion [doris]
doris-robot commented on PR #38170: URL: https://github.com/apache/doris/pull/38170#issuecomment-2241096267 ClickBench: Total hot run time: 30.35 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 11f71ef17bfd66a96a840faff8b306723535dc0d, data reload: false query1 0.050.040.03 query2 0.080.040.04 query3 0.220.050.05 query4 1.680.080.07 query5 0.490.480.48 query6 1.150.740.73 query7 0.020.020.01 query8 0.050.040.04 query9 0.550.490.50 query10 0.550.540.55 query11 0.150.120.11 query12 0.150.130.12 query13 0.600.590.58 query14 0.760.800.77 query15 0.860.820.82 query16 0.380.350.36 query17 0.981.041.06 query18 0.230.210.21 query19 1.761.711.73 query20 0.020.010.01 query21 15.39 0.780.65 query22 3.898.471.51 query23 18.32 1.311.32 query24 2.110.220.23 query25 0.170.090.08 query26 0.300.210.21 query27 0.460.240.23 query28 13.26 1.020.99 query29 12.64 3.373.33 query30 0.260.060.05 query31 2.890.380.40 query32 3.280.480.47 query33 2.872.972.94 query34 17.10 4.314.39 query35 4.504.414.41 query36 0.650.470.49 query37 0.180.160.15 query38 0.160.150.15 query39 0.040.040.04 query40 0.150.120.12 query41 0.100.050.05 query42 0.060.040.05 query43 0.040.050.04 Total cold run time: 109.55 s Total hot run time: 30.35 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](ut) repair segcompaction ut (#38165)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new d1af5d5cd24 [fix](ut) repair segcompaction ut (#38165)
d1af5d5cd24 is described below
commit d1af5d5cd245add33f530ab880db09939cdb1009
Author: zhannngchen <48427519+zhannngc...@users.noreply.github.com>
AuthorDate: Sat Jul 20 19:43:47 2024 +0800
[fix](ut) repair segcompaction ut (#38165)
SegcompactionTest is disabled by #16068 due to some code clean up, it
needs to be repaired
-
Co-authored-by: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
---
be/test/CMakeLists.txt | 1 -
be/test/olap/segcompaction_test.cpp | 518 +---
2 files changed, 246 insertions(+), 273 deletions(-)
diff --git a/be/test/CMakeLists.txt b/be/test/CMakeLists.txt
index c119af01e88..5a37267a937 100644
--- a/be/test/CMakeLists.txt
+++ b/be/test/CMakeLists.txt
@@ -46,7 +46,6 @@ list(REMOVE_ITEM UT_FILES
${CMAKE_CURRENT_SOURCE_DIR}/olap/rowset/segment_v2/frame_of_reference_page_test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/olap/rowset/segment_v2/plain_page_test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/olap/rowset/segment_v2/rle_page_test.cpp
-${CMAKE_CURRENT_SOURCE_DIR}/olap/segcompaction_test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/runtime/decimal_value_test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/runtime/result_buffer_mgr_test.cpp
${CMAKE_CURRENT_SOURCE_DIR}/util/decompress_test.cpp
diff --git a/be/test/olap/segcompaction_test.cpp
b/be/test/olap/segcompaction_test.cpp
index f612694e7e3..468b24784af 100644
--- a/be/test/olap/segcompaction_test.cpp
+++ b/be/test/olap/segcompaction_test.cpp
@@ -23,7 +23,6 @@
#include
#include "common/config.h"
-#include "env/env_posix.h"
#include "gen_cpp/AgentService_types.h"
#include "gen_cpp/olap_file.pb.h"
#include "io/fs/local_file_system.h"
@@ -47,14 +46,12 @@ namespace doris {
using namespace ErrorCode;
static const uint32_t MAX_PATH_LEN = 1024;
-static std::unique_ptr l_engine;
+static StorageEngine* l_engine;
static const std::string lTestDir = "./data_test/data/segcompaction_test";
class SegCompactionTest : public testing::Test {
public:
-SegCompactionTest() : _data_dir(std::make_unique(lTestDir)) {
-_data_dir->update_capacity();
-}
+SegCompactionTest() = default;
void SetUp() {
config::enable_segcompaction = true;
@@ -76,20 +73,24 @@ public:
doris::EngineOptions options;
options.store_paths = paths;
-Status s = doris::StorageEngine::open(options, &l_engine);
+
+auto engine = std::make_unique(options);
+l_engine = engine.get();
+ExecEnv::GetInstance()->set_storage_engine(std::move(engine));
+
+Status s = l_engine->open();
EXPECT_TRUE(s.ok()) << s.to_string();
-ExecEnv* exec_env = doris::ExecEnv::GetInstance();
+_data_dir = std::make_unique(*l_engine, lTestDir);
+static_cast(_data_dir->update_capacity());
EXPECT_TRUE(io::global_local_filesystem()->create_directory(lTestDir).ok());
-l_engine->start_bg_threads();
+s = l_engine->start_bg_threads();
+EXPECT_TRUE(s.ok()) << s.to_string();
}
-void TearDown() {
-l_engine.reset();
-config::enable_segcompaction = false;
-}
+void TearDown() { config::enable_segcompaction = false; }
protected:
OlapReaderStatistics _stats;
@@ -122,7 +123,8 @@ protected:
}
// (k1 int, k2 varchar(20), k3 int) keys (k1, k2)
-void create_tablet_schema(TabletSchemaSPtr tablet_schema, KeysType
keystype) {
+void create_tablet_schema(TabletSchemaSPtr tablet_schema, KeysType
keystype,
+ int num_value_col = 1) {
TabletSchemaPB tablet_schema_pb;
tablet_schema_pb.set_keys_type(keystype);
tablet_schema_pb.set_num_short_key_columns(2);
@@ -152,15 +154,18 @@ protected:
column_2->set_is_nullable(true);
column_2->set_is_bf_column(false);
-ColumnPB* column_3 = tablet_schema_pb.add_column();
-column_3->set_unique_id(3);
-column_3->set_name("v1");
-column_3->set_type("INT");
-column_3->set_length(4);
-column_3->set_is_key(false);
-column_3->set_is_nullable(false);
-column_3->set_is_bf_column(false);
-column_3->set_aggregation("SUM");
+for (int i = 1; i <= num_value_col; i++) {
+ColumnPB* v_column = tablet_schema_pb.add_column();
+v_column->set_unique_id(2 + i);
+v_column->set_name(fmt::format("v{}", i));
+v_column->set_type("INT");
+v_column->set_length(4);
+v_column->set_is_key(false);
+v_column->set_is_nullable(false);
+v_column-
Re: [PR] [fix](ut) repair segcompaction ut [doris]
dataroaring merged PR #38165: URL: https://github.com/apache/doris/pull/38165 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compaction) fix mismatch between segment key and value column rows during compaction [doris]
github-actions[bot] commented on PR #37960: URL: https://github.com/apache/doris/pull/37960#issuecomment-2241099898 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement]Add regression case for group commit [doris]
Tech-Circle-48 closed pull request #38161: [enhancement]Add regression case for group commit URL: https://github.com/apache/doris/pull/38161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) fix be core when using stream load record in cloud mode [doris]
github-actions[bot] commented on PR #37903: URL: https://github.com/apache/doris/pull/37903#issuecomment-2241103088 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index) refactor stream load regression test case [doris]
xiaokang commented on PR #38158: URL: https://github.com/apache/doris/pull/38158#issuecomment-2241108190 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) fix be core when using stream load record in cloud mode [doris]
liaoxin01 merged PR #37903: URL: https://github.com/apache/doris/pull/37903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](cloud) fix be core when using stream load record in cloud mode (#37903)
This is an automated email from the ASF dual-hosted git repository.
liaoxin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new cbbc15e3347 [fix](cloud) fix be core when using stream load record in
cloud mode (#37903)
cbbc15e3347 is described below
commit cbbc15e33475c62d7b690999f2e973173cb01606
Author: hui lai <1353307...@qq.com>
AuthorDate: Sat Jul 20 20:59:41 2024 +0800
[fix](cloud) fix be core when using stream load record in cloud mode
(#37903)
---
be/src/cloud/cloud_backend_service.cpp | 8 +++
be/src/cloud/cloud_backend_service.h | 3 +
be/src/cloud/cloud_storage_engine.cpp | 4 ++
be/src/http/action/http_stream.cpp | 8 ++-
be/src/http/action/stream_load.cpp | 8 ++-
be/src/olap/storage_engine.cpp | 41 ++--
be/src/olap/storage_engine.h | 16 ++---
be/src/service/backend_service.cpp | 48 --
be/src/service/backend_service.h | 4 ++
.../org/apache/doris/load/StreamLoadRecordMgr.java | 28
.../pipeline/cloud_p0/conf/be_custom.conf | 2 +
regression-test/pipeline/p0/conf/be.conf | 1 +
.../stream_load/test_stream_load_record.groovy | 76 ++
13 files changed, 178 insertions(+), 69 deletions(-)
diff --git a/be/src/cloud/cloud_backend_service.cpp
b/be/src/cloud/cloud_backend_service.cpp
index f576b60045d..d91e9e416b8 100644
--- a/be/src/cloud/cloud_backend_service.cpp
+++ b/be/src/cloud/cloud_backend_service.cpp
@@ -29,6 +29,8 @@
#include "common/status.h"
#include "io/cache/block_file_cache_downloader.h"
#include "io/cache/block_file_cache_factory.h"
+#include "runtime/stream_load/stream_load_context.h"
+#include "runtime/stream_load/stream_load_recorder.h"
#include "util/brpc_client_cache.h" // BrpcClientCache
#include "util/thrift_server.h"
@@ -186,4 +188,10 @@ void
CloudBackendService::check_warm_up_cache_async(TCheckWarmUpCacheAsyncRespon
response.status = t_status;
}
+void CloudBackendService::get_stream_load_record(TStreamLoadRecordResult&
result,
+ int64_t
last_stream_record_time) {
+BaseBackendService::get_stream_load_record(result, last_stream_record_time,
+
_engine.get_stream_load_recorder());
+}
+
} // namespace doris
diff --git a/be/src/cloud/cloud_backend_service.h
b/be/src/cloud/cloud_backend_service.h
index 88f0099fe73..358cb4d1f0b 100644
--- a/be/src/cloud/cloud_backend_service.h
+++ b/be/src/cloud/cloud_backend_service.h
@@ -53,6 +53,9 @@ public:
void check_warm_up_cache_async(TCheckWarmUpCacheAsyncResponse& response,
const TCheckWarmUpCacheAsyncRequest&
request) override;
+void get_stream_load_record(TStreamLoadRecordResult& result,
+int64_t last_stream_record_time) override;
+
private:
CloudStorageEngine& _engine;
};
diff --git a/be/src/cloud/cloud_storage_engine.cpp
b/be/src/cloud/cloud_storage_engine.cpp
index b1b455d2007..de4bbac7b3e 100644
--- a/be/src/cloud/cloud_storage_engine.cpp
+++ b/be/src/cloud/cloud_storage_engine.cpp
@@ -196,6 +196,10 @@ Status CloudStorageEngine::open() {
_tablet_hotspot = std::make_unique();
+RETURN_NOT_OK_STATUS_WITH_WARN(
+
init_stream_load_recorder(ExecEnv::GetInstance()->store_paths()[0].path),
+"init StreamLoadRecorder failed");
+
return ThreadPoolBuilder("SyncLoadForTabletsThreadPool")
.set_max_threads(config::sync_load_for_tablets_thread)
.set_min_threads(config::sync_load_for_tablets_thread)
diff --git a/be/src/http/action/http_stream.cpp
b/be/src/http/action/http_stream.cpp
index a3439969e60..87cc2f694eb 100644
--- a/be/src/http/action/http_stream.cpp
+++ b/be/src/http/action/http_stream.cpp
@@ -30,6 +30,7 @@
#include
#include
+#include "cloud/cloud_storage_engine.h"
#include "cloud/config.h"
#include "common/config.h"
#include "common/consts.h"
@@ -119,7 +120,7 @@ void HttpStreamAction::handle(HttpRequest* req) {
// add new line at end
str = str + '\n';
HttpChannel::send_reply(req, str);
-if (config::enable_stream_load_record && !config::is_cloud_mode()) {
+if (config::enable_stream_load_record) {
str = ctx->prepare_stream_load_record(str);
_save_stream_load_record(ctx, str);
}
@@ -364,8 +365,9 @@ Status HttpStreamAction::process_put(HttpRequest* http_req,
void
HttpStreamAction::_save_stream_load_record(std::shared_ptr
ctx,
const std::string& str) {
-auto stream_load_recorder =
-
ExecEnv::GetInstance()->storage_engine().to_local().get_stream_load_recorder();
+std::shared_ptr stream_load_recor
svn commit: r70440 - in /dev/doris/2.1.5-rc02: ./ apache-doris-2.1.5-rc02-src.tar.gz apache-doris-2.1.5-rc02-src.tar.gz.asc apache-doris-2.1.5-rc02-src.tar.gz.sha512
Author: yiguolei Date: Sat Jul 20 15:10:58 2024 New Revision: 70440 Log: add 2.1.5-rc02 Added: dev/doris/2.1.5-rc02/ dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz (with props) dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz.asc (with props) dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz.sha512 Added: dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz == Binary file - no diff available. Propchange: dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz -- svn:mime-type = application/x-gzip Added: dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz.asc == Binary file - no diff available. Propchange: dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz.asc -- svn:mime-type = application/pgp-signature Added: dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz.sha512 == --- dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz.sha512 (added) +++ dev/doris/2.1.5-rc02/apache-doris-2.1.5-rc02-src.tar.gz.sha512 Sat Jul 20 15:10:58 2024 @@ -0,0 +1 @@ +2fe26638e21846c4581b495ecc3af2628796ab8334004b676db264cce1f96aba3dc84f047e1bb33e39b842752869138432df207bae1fc162bf98ad3a6e0762be apache-doris-2.1.5-rc02-src.tar.gz - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Enhancement] Add regression case for group commit [doris]
Tech-Circle-48 opened a new pull request, #38171: URL: https://github.com/apache/doris/pull/38171 ## Proposed changes Issue Number: close #33942 Add a regression test case to verify that group commit functionlity now support parital updates in Doris. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] Add regression case for group commit [doris]
doris-robot commented on PR #38171: URL: https://github.com/apache/doris/pull/38171#issuecomment-2241190083 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](common) refactor the Exception code [doris]
doris-robot commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241210092 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Refactor](common) refactor the Exception code [doris]
HappenLee opened a new pull request, #38172: URL: https://github.com/apache/doris/pull/38172 ## Proposed changes 1. Remove the useless code in Exception 2. Use fmt replace the streamstring for performance -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](common) refactor the Exception code [doris]
HappenLee commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241210244 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](common) refactor the Exception code [doris]
github-actions[bot] commented on code in PR #38172: URL: https://github.com/apache/doris/pull/38172#discussion_r1685504197 ## be/src/common/exception.h: ## @@ -19,8 +19,8 @@ #include Review Comment: warning: 'fmt/format.h' file not found [clang-diagnostic-error] ```cpp #include ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth [doris]
xinyiZzz commented on PR #38168: URL: https://github.com/apache/doris/pull/38168#issuecomment-2241225102 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Refactor refresh workload groups weighted memory ratio and record refresh interval memory growth [doris]
xinyiZzz commented on code in PR #38168:
URL: https://github.com/apache/doris/pull/38168#discussion_r1685504799
##
be/src/runtime/workload_group/workload_group.h:
##
@@ -186,7 +167,7 @@ class WorkloadGroup : public
std::enable_shared_from_this {
// new query can not submit
// waiting running query to be cancelled or finish
bool _is_shutdown = false;
-std::unordered_map> _query_ctxs;
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](common) refactor the Exception code [doris]
doris-robot commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241255656 TPC-H: Total hot run time: 40036 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d5eb4d3764f33534eee5f2874f54f5facb90096e, data reload: false -- Round 1 -- q1 17618 444642974297 q2 2014193 188 188 q3 10445 121111751175 q4 10183 809 743 743 q5 7659272126912691 q6 221 138 138 138 q7 967 602 615 602 q8 9221209321312093 q9 8814658265176517 q10 8643375037543750 q11 446 239 234 234 q12 392 223 224 223 q13 18231 300729652965 q14 277 244 246 244 q15 541 474 508 474 q16 504 387 378 378 q17 974 673 703 673 q18 8236758574517451 q19 8120138413811381 q20 653 323 321 321 q21 5018324632133213 q22 343 285 287 285 Total cold run time: 119520 ms Total hot run time: 40036 ms - Round 2, with runtime_filter_mode=off - q1 4328422642224222 q2 377 263 275 263 q3 3083297229302930 q4 1951168617681686 q5 5552555855405540 q6 224 131 137 131 q7 2258183218451832 q8 3314400134393439 q9 8793893788318831 q10 4097374038623740 q11 589 514 528 514 q12 830 631 667 631 q13 16347 318732173187 q14 332 281 296 281 q15 521 480 486 480 q16 502 433 438 433 q17 1819152815201520 q18 8085799078477847 q19 1763159215831583 q20 2144191318461846 q21 5167489148464846 q22 590 496 527 496 Total cold run time: 72666 ms Total hot run time: 56278 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](common) refactor the Exception code [doris]
doris-robot commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241258631 TPC-DS: Total hot run time: 174304 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d5eb4d3764f33534eee5f2874f54f5facb90096e, data reload: false query1 916 382 369 369 query2 6431194217761776 query3 6629209 216 209 query4 25712 17452 17256 17256 query5 3658462 462 462 query6 269 171 159 159 query7 4583290 278 278 query8 236 190 184 184 query9 8520240823932393 query10 425 300 280 280 query11 10654 10069 99989998 query12 118 81 89 81 query13 1623362 359 359 query14 9921768875217521 query15 212 161 166 161 query16 7729435 447 435 query17 1377573 547 547 query18 1974288 291 288 query19 202 157 167 157 query20 90 86 84 84 query21 202 131 125 125 query22 4386402140584021 query23 34279 33784 33771 33771 query24 11079 291129062906 query25 631 417 408 408 query26 730 155 154 154 query27 2308280 284 280 query28 6391203420582034 query29 928 666 640 640 query30 257 159 158 158 query31 984 771 759 759 query32 100 56 55 55 query33 767 354 355 354 query34 897 492 499 492 query35 872 751 757 751 query36 1135958 994 958 query37 149 90 82 82 query38 2951290528552855 query39 921 869 879 869 query40 216 128 124 124 query41 51 48 48 48 query42 116 103 100 100 query43 500 467 464 464 query44 1218729 730 729 query45 198 167 163 163 query46 1084709 703 703 query47 1850175717621757 query48 371 293 295 293 query49 872 425 417 417 query50 770 385 396 385 query51 6863685168746851 query52 107 89 93 89 query53 355 297 293 293 query54 894 456 450 450 query55 77 75 74 74 query56 309 365 270 270 query57 1137103310451033 query58 251 242 245 242 query59 2960267728412677 query60 302 266 273 266 query61 95 95 111 95 query62 792 642 658 642 query63 314 288 289 288 query64 9338223316671667 query65 3282311831183118 query66 798 323 339 323 query67 15477 14874 14847 14847 query68 8486552 553 552 query69 737 448 389 389 query70 1423109310981093 query71 530 282 282 282 query72 9122579257995792 query73 2228322 325 322 query74 6161566155715571 query75 4443271026542654 query76 5032966 915 915 query77 714 314 299 299 query78 9867911390229022 query79 7787539 531 531 query80 2106488 487 487 query81 583 221 233 221 query82 289 133 130 130 query83 271 164 165 164 query84 278 86 91 86 query85 867 311 303 303 query86 347 287 331 287 query87 3254309831293098 query88 4871251124582458 query89 527 393 396 393 query90 1885205 201 201 query91 132 101 104 101 query92 70 49 51 49 query93 5945515 497 497 query94 1190303 298 298 query95 408 322 326 322 query96 608 284 277 277 query97 3158301030293010 query98 217 199 197 197 query99 1503127012261226 Total cold run time: 296882 ms Total hot run time: 174304 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Refactor](common) refactor the Exception code [doris]
doris-robot commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241259940 ClickBench: Total hot run time: 30.19 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d5eb4d3764f33534eee5f2874f54f5facb90096e, data reload: false query1 0.040.040.03 query2 0.070.040.04 query3 0.220.060.05 query4 1.670.090.11 query5 0.500.490.50 query6 1.130.720.72 query7 0.020.010.02 query8 0.050.040.04 query9 0.550.480.49 query10 0.550.540.53 query11 0.150.110.11 query12 0.140.120.12 query13 0.590.590.58 query14 0.770.780.77 query15 0.860.820.80 query16 0.360.340.36 query17 1.030.991.00 query18 0.240.240.23 query19 1.831.681.71 query20 0.010.000.00 query21 15.39 0.770.64 query22 4.218.241.54 query23 18.29 1.361.36 query24 2.150.230.22 query25 0.150.080.09 query26 0.300.210.21 query27 0.460.230.23 query28 13.24 1.021.01 query29 12.62 3.243.26 query30 0.250.060.05 query31 2.940.410.39 query32 3.220.480.48 query33 2.922.842.90 query34 16.92 4.894.36 query35 4.394.464.40 query36 0.640.470.48 query37 0.190.150.15 query38 0.150.150.14 query39 0.040.040.03 query40 0.160.130.13 query41 0.100.050.04 query42 0.060.040.04 query43 0.040.040.04 Total cold run time: 109.61 s Total hot run time: 30.19 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](common) refactor the Exception code [doris]
HappenLee commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241442019 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] [streamLoader] There are null values in the imported data [doris]
liugddx opened a new issue, #38173: URL: https://github.com/apache/doris/issues/38173 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 2.0.3 ### What's Wrong? There are null values in the imported data ### What You Expected? - ### How to Reproduce? - ddl ``` create database e2e_sink; CREATE TABLE `doris_e2e_unique_table` ( `F_ID` bigint(20) NULL, `F_INT` int(11) NULL, `F_BIGINT` bigint(20) NULL, `F_TINYINT` tinyint(4) NULL, `F_SMALLINT` smallint(6) NULL, `F_DECIMAL` DECIMAL(18, 6) NULL, `F_LARGEINT` largeint(40) NULL, `F_BOOLEAN` boolean NULL, `F_DOUBLE` double NULL, `F_FLOAT` float NULL, `F_CHAR` char(1) NULL, `F_VARCHAR_11` varchar(11) NULL, `F_STRING` text NULL, `F_DATETIME_P` datetime(6) NULL, `F_DATETIME` datetime NULL, `F_DATE` date NULL, `MAP_VARCHAR_BOOLEAN` MAP NULL, `MAP_CHAR_TINYINT` MAP NULL, `MAP_STRING_SMALLINT` MAP NULL, `MAP_INT_INT` MAP NULL, `MAP_TINYINT_BIGINT` MAP NULL, `MAP_SMALLINT_LARGEINT` MAP NULL, `MAP_BIGINT_FLOAT` MAP NULL, `MAP_LARGEINT_DOUBLE` MAP NULL, `MAP_STRING_DECIMAL` MAP NULL, `MAP_DECIMAL_DATE` MAP NULL, `MAP_DATE_DATETIME` MAP NULL, `MAP_DATETIME_CHAR` MAP NULL, `MAP_CHAR_VARCHAR` MAP NULL, `MAP_VARCHAR_STRING` MAP NULL ) ENGINE=OLAP UNIQUE KEY(`F_ID`) COMMENT 'OLAP' DISTRIBUTED BY HASH(`F_ID`) BUCKETS 10 PROPERTIES ( "replication_allocation" = "tag.location.default: 1", "is_being_synced" = "false", "storage_format" = "V2", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false" ); ``` - data [data.json](https://github.com/user-attachments/files/16323004/data.json) - shell ``` curl --location-trusted -u root: \ -H "Expect:100-continue" \ -H "format:json" \ -H "strip_outer_array:true" \ -T data.json \ -XPUT http://localhost:8040/api/e2e_sink/doris_e2e_unique_table/_stream_load ``` ### Anything Else? - ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] The value of timestamp type data obtained through arrow is incorrect. [doris]
liugddx opened a new issue, #38174:
URL: https://github.com/apache/doris/issues/38174
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
master
### What's Wrong?
The value of timestamp type data obtained through arrow is incorrect.
https://github.com/user-attachments/assets/557ceca9-b6ad-44b5-a56c-253e33f52d2c";>
https://github.com/user-attachments/assets/ae00da53-fc5b-4499-a617-93330de3c90f";>
### What You Expected?
The actual value is 2024, but the extracted value is 1970.
### How to Reproduce?
- ddl
```
-- e2e_sink.doris_e2e_unique_table definition
CREATE TABLE `doris_e2e_unique_table` (
`F_ID` bigint(20) NULL,
`F_INT` int(11) NULL,
`F_BIGINT` bigint(20) NULL,
`F_TINYINT` tinyint(4) NULL,
`F_SMALLINT` smallint(6) NULL,
`F_DECIMAL` DECIMAL(18, 6) NULL,
`F_LARGEINT` largeint(40) NULL,
`F_BOOLEAN` boolean NULL,
`F_DOUBLE` double NULL,
`F_FLOAT` float NULL,
`F_CHAR` char(1) NULL,
`F_VARCHAR_11` varchar(11) NULL,
`F_STRING` text NULL,
`F_DATETIME_P` datetime(6) NULL,
`F_DATETIME` datetime NULL,
`F_DATE` date NULL,
`MAP_VARCHAR_BOOLEAN` MAP NULL,
`MAP_CHAR_TINYINT` MAP NULL,
`MAP_STRING_SMALLINT` MAP NULL,
`MAP_INT_INT` MAP NULL,
`MAP_TINYINT_BIGINT` MAP NULL,
`MAP_SMALLINT_LARGEINT` MAP NULL,
`MAP_BIGINT_FLOAT` MAP NULL,
`MAP_LARGEINT_DOUBLE` MAP NULL,
`MAP_STRING_DECIMAL` MAP NULL,
`MAP_DECIMAL_DATE` MAP NULL,
`MAP_DATE_DATETIME` MAP NULL,
`MAP_DATETIME_CHAR` MAP NULL,
`MAP_CHAR_VARCHAR` MAP NULL,
`MAP_VARCHAR_STRING` MAP NULL
) ENGINE=OLAP
UNIQUE KEY(`F_ID`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`F_ID`) BUCKETS 10
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"is_being_synced" = "false",
"storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false"
);
```
-data
```
INSERT INTO `doris_e2e_unique_table`
(F_ID, F_INT, F_BIGINT, F_TINYINT, F_SMALLINT, F_DECIMAL, F_LARGEINT,
F_BOOLEAN, F_DOUBLE, F_FLOAT, F_CHAR, F_VARCHAR_11, F_STRING, F_DATETIME_P,
F_DATETIME, F_DATE, MAP_VARCHAR_BOOLEAN, MAP_CHAR_TINYINT, MAP_STRING_SMALLINT,
MAP_INT_INT, MAP_TINYINT_BIGINT, MAP_SMALLINT_LARGEINT, MAP_BIGINT_FLOAT,
MAP_LARGEINT_DOUBLE, MAP_STRING_DECIMAL, MAP_DECIMAL_DATE, MAP_DATE_DATETIME,
MAP_DATETIME_CHAR, MAP_CHAR_VARCHAR, MAP_VARCHAR_STRING)
VALUES
(1, 10, 100, 1, 100, 123.456789, 9223372036854775807, TRUE, 3.14159,
2.71828, 'A', 'Hello World', 'Sample text for the string field.', '2024-07-19
12:00:00.123456', '2024-07-19 12:00:00', '2024-07-19',
{'key1': TRUE, 'key2': FALSE},
{'A': 1, 'B': 2},
{'text1': 1, 'text2': 2},
{1: 1, 2: 4},
{1: 100},
{100: 9223372036854775807},
{100: 3.14},
{9223372036854775807: 3.14},
{'value1': 123.45, 'value2': 678.90},
{123.45: '2024-07-19'},
{'2024-07-19': '2024-07-19 12:00:00'},
{'2024-07-19 12:00:00': 'Datetime example'},
{'key1': 'value1', 'key2': 'value2'},
{'keyA': 'Sample text', 'keyB': 'Another text'}
)
```
-code
```
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.liugddx;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.BigIntVector;
import org.apache.arrow.vector.BitVector;
import org.apache.arrow.vector.DateDayVector;
import org.apache.arrow.vector.DecimalVector;
import org.apache.arrow.vector.FieldVector;
import org.apache.arrow.vector.Float4Vector;
import org.apache.arrow.vector.Float8Vector;
import org.apache.arrow.vector.IntVector;
import org.apache.arrow.vector.SmallIntVector;
import org.apache.arrow.vector.TimeSecVector;
import org.apache.arrow.vector.TimeStampMicro
Re: [PR] [Refactor](common) refactor the Exception code [doris]
doris-robot commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241455243 TPC-H: Total hot run time: 39810 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ec290709790053809f1541bd6b885627ec6f2fcc, data reload: false -- Round 1 -- q1 17614 446342974297 q2 2023201 196 196 q3 10426 123011231123 q4 10188 718 719 718 q5 7535267726772677 q6 230 137 138 137 q7 949 611 597 597 q8 9218206521092065 q9 8877661565916591 q10 8896377038133770 q11 482 235 229 229 q12 562 232 220 220 q13 17760 300129532953 q14 287 228 243 228 q15 520 474 490 474 q16 489 374 374 374 q17 965 619 709 619 q18 7989739274007392 q19 7768134912921292 q20 687 325 335 325 q21 5230324433183244 q22 338 289 290 289 Total cold run time: 119033 ms Total hot run time: 39810 ms - Round 2, with runtime_filter_mode=off - q1 4405429642474247 q2 365 253 252 252 q3 3045293128892889 q4 1997170816911691 q5 5646559155375537 q6 223 141 136 136 q7 2246189318751875 q8 3253339434183394 q9 8730888788648864 q10 4104385638193819 q11 596 493 511 493 q12 816 656 681 656 q13 17349 313531783135 q14 326 288 311 288 q15 520 516 495 495 q16 488 436 429 429 q17 1808150815081508 q18 8139791378107810 q19 1784162516731625 q20 2936187718371837 q21 5228485449844854 q22 589 523 508 508 Total cold run time: 74593 ms Total hot run time: 56342 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](common) refactor the Exception code [doris]
doris-robot commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241458468 TPC-DS: Total hot run time: 173106 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit ec290709790053809f1541bd6b885627ec6f2fcc, data reload: false query1 909 372 359 359 query2 6483188018351835 query3 6650213 217 213 query4 28333 17728 17300 17300 query5 3686475 471 471 query6 279 194 179 179 query7 4604289 291 289 query8 251 198 196 196 query9 8604245224192419 query10 446 302 271 271 query11 11948 10078 10183 10078 query12 118 83 86 83 query13 1654370 367 367 query14 10169 701577287015 query15 226 167 167 167 query16 7722475 494 475 query17 1540547 514 514 query18 1917273 274 273 query19 191 150 144 144 query20 93 83 80 80 query21 209 128 123 123 query22 4434421139513951 query23 34142 33889 33783 33783 query24 11151 291429442914 query25 624 398 437 398 query26 1119153 153 153 query27 2406290 283 283 query28 7328203820372037 query29 889 642 624 624 query30 261 159 147 147 query31 964 766 792 766 query32 93 52 56 52 query33 741 342 347 342 query34 914 484 504 484 query35 847 780 743 743 query36 1169990 1001990 query37 137 81 80 80 query38 2961285327792779 query39 873 835 808 808 query40 202 119 120 119 query41 45 44 44 44 query42 114 99 96 96 query43 512 434 451 434 query44 1172704 713 704 query45 197 160 162 160 query46 1120727 728 727 query47 1815175417911754 query48 364 283 284 283 query49 850 409 415 409 query50 803 408 386 386 query51 6996685168116811 query52 106 96 96 96 query53 357 280 280 280 query54 857 446 441 441 query55 76 71 78 71 query56 290 259 267 259 query57 1121102710721027 query58 258 243 296 243 query59 2882255525912555 query60 314 283 290 283 query61 127 100 97 97 query62 784 637 658 637 query63 328 287 286 286 query64 9422225516761676 query65 3183311231293112 query66 715 324 327 324 query67 15696 15114 15019 15019 query68 4474531 534 531 query69 550 410 371 371 query70 1153105310841053 query71 406 332 273 273 query72 6819532755295327 query73 741 326 322 322 query74 6052571956585658 query75 3371274926512651 query76 2578895 881 881 query77 443 295 305 295 query78 10291 896790008967 query79 2573503 511 503 query80 2062480 485 480 query81 581 227 220 220 query82 875 138 132 132 query83 316 183 169 169 query84 269 94 83 83 query85 1784317 298 298 query86 464 323 311 311 query87 3321306431363064 query88 4129243424442434 query89 485 386 387 386 query90 1766192 190 190 query91 127 119 103 103 query92 61 50 54 50 query93 2604489 487 487 query94 1210289 287 287 query95 406 318 319 318 query96 606 277 283 277 query97 3225303130333031 query98 225 198 191 191 query99 1651125312451245 Total cold run time: 283409 ms Total hot run time: 173106 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Refactor](common) refactor the Exception code [doris]
doris-robot commented on PR #38172: URL: https://github.com/apache/doris/pull/38172#issuecomment-2241459779 ClickBench: Total hot run time: 30.62 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit ec290709790053809f1541bd6b885627ec6f2fcc, data reload: false query1 0.040.030.04 query2 0.090.040.04 query3 0.220.050.05 query4 1.680.090.08 query5 0.490.480.49 query6 1.120.720.72 query7 0.020.010.02 query8 0.050.040.04 query9 0.560.500.50 query10 0.550.550.54 query11 0.150.110.12 query12 0.140.120.12 query13 0.600.580.58 query14 0.750.790.78 query15 0.860.820.82 query16 0.380.360.38 query17 0.961.010.97 query18 0.230.220.22 query19 1.761.771.76 query20 0.010.000.00 query21 15.40 0.770.65 query22 4.197.741.86 query23 18.27 1.391.24 query24 2.080.230.22 query25 0.150.090.08 query26 0.290.210.21 query27 0.450.220.23 query28 13.33 1.041.01 query29 12.63 3.293.24 query30 0.260.050.05 query31 2.860.410.39 query32 3.250.470.48 query33 2.842.932.92 query34 17.15 4.374.35 query35 4.504.454.50 query36 0.630.490.47 query37 0.190.150.16 query38 0.150.140.14 query39 0.040.030.03 query40 0.140.120.13 query41 0.090.050.05 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 109.66 s Total hot run time: 30.62 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
