Re: [PR] branch-2.1: [feat](thirdparty) Add aws dentity-management and sts library for be and ry #49565 [doris]
doris-robot commented on PR #49684: URL: https://github.com/apache/doris/pull/49684#issuecomment-2781217310 # BE UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/4216c8e3b9f180cbf66eaec501a2c02f245bed68_4216c8e3b9f180cbf66eaec501a2c02f245bed68/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/4216c8e3b9f180cbf66eaec501a2c02f245bed68_4216c8e3b9f180cbf66eaec501a2c02f245bed68/report/index.html) | Category | Coverage | |---|| | Function Coverage | 38.18% (10061/26350) | | Line Coverage | 29.20% (82884/283873) | | Region Coverage | 27.87% (42620/152918) | | Branch Coverage | 24.49% (21597/88196) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](thriftconnection) change default timeout when user not set [doris]
github-actions[bot] commented on PR #49691: URL: https://github.com/apache/doris/pull/49691#issuecomment-2781223548 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](thriftconnection) change default timeout when user not set [doris]
github-actions[bot] commented on PR #49691: URL: https://github.com/apache/doris/pull/49691#issuecomment-2781223534 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (97c39b0f951 -> 9a0bf9e2617)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 97c39b0f951 Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 new 9a0bf9e2617 Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (97c39b0f951) \ N -- N -- N refs/heads/asf-site (9a0bf9e2617) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](paimon)Enhance the observability of split and JNI in profile [doris]
zhiqiang- commented on code in PR #49688:
URL: https://github.com/apache/doris/pull/49688#discussion_r2030053767
##
be/src/vec/exec/jni_connector.cpp:
##
@@ -82,12 +87,15 @@ Status JniConnector::open(RuntimeState* state,
RuntimeProfile* profile) {
batch_size = _state->batch_size();
}
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
-SCOPED_TIMER(_open_scanner_time);
+MonotonicStopWatch _watch;
+_watch.start();
_scanner_params.emplace("time_zone", _state->timezone());
RETURN_IF_ERROR(_init_jni_scanner(env, batch_size));
// Call org.apache.doris.common.jni.JniScanner#open
env->CallVoidMethod(_jni_scanner_obj, _jni_scanner_open);
RETURN_ERROR_IF_EXC(env);
+_watch.stop();
Review Comment:
what if `open` returned before this line
##
be/src/vec/exec/jni_connector.cpp:
##
@@ -113,11 +121,16 @@ Status JniConnector::get_next_block(Block* block, size_t*
read_rows, bool* eof)
JNIEnv* env = nullptr;
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
long meta_address = 0;
-{
-SCOPED_TIMER(_java_scan_time);
-meta_address = env->CallLongMethod(_jni_scanner_obj,
_jni_scanner_get_next_batch);
-}
+
+MonotonicStopWatch _watch;
+_watch.start();
+meta_address = env->CallLongMethod(_jni_scanner_obj,
_jni_scanner_get_next_batch);
RETURN_ERROR_IF_EXC(env);
+_watch.stop();
Review Comment:
what if returned at line 128
##
fe/fe-core/src/main/java/org/apache/doris/common/profile/Profile.java:
##
@@ -295,6 +295,7 @@ public synchronized void updateSummary(Map
summaryInfo, boolean
}
return;
}
+summaryProfile.queryFinished();
Review Comment:
why adding this line?
##
be/src/util/runtime_profile.h:
##
@@ -278,6 +278,39 @@ class RuntimeProfile {
DerivedCounterFunction _counter_fn;
};
+using ConditionCounterFunction = std::function;
+
+class ConditionCounter : public Counter {
Review Comment:
add comment to explain the usage
##
fe/fe-core/src/main/java/org/apache/doris/common/profile/SummaryProfile.java:
##
@@ -179,7 +185,12 @@ public class SummaryProfile {
TRACE_ID,
TRANSACTION_COMMIT_TIME,
SYSTEM_MESSAGE,
-EXECUTED_BY_FRONTEND
+EXECUTED_BY_FRONTEND,
+NEREIDS_BE_FOLD_CONST_TIME,
Review Comment:
why need this
##
be/src/vec/exec/jni_connector.cpp:
##
@@ -113,11 +121,16 @@ Status JniConnector::get_next_block(Block* block, size_t*
read_rows, bool* eof)
JNIEnv* env = nullptr;
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
long meta_address = 0;
-{
-SCOPED_TIMER(_java_scan_time);
-meta_address = env->CallLongMethod(_jni_scanner_obj,
_jni_scanner_get_next_batch);
-}
+
+MonotonicStopWatch _watch;
+_watch.start();
+meta_address = env->CallLongMethod(_jni_scanner_obj,
_jni_scanner_get_next_batch);
RETURN_ERROR_IF_EXC(env);
+_watch.stop();
+_java_scan_watcher += _watch.elapsed_time();
+LOG(INFO) << "mmc "
Review Comment:
Should not be a INFO log
##
be/src/vec/exec/jni_connector.cpp:
##
@@ -70,10 +70,15 @@ namespace doris::vectorized {
Status JniConnector::open(RuntimeState* state, RuntimeProfile* profile) {
_state = state;
_profile = profile;
-ADD_TIMER(_profile, _connector_name.c_str());
-_open_scanner_time = ADD_CHILD_TIMER(_profile, "OpenScannerTime",
_connector_name.c_str());
-_java_scan_time = ADD_CHILD_TIMER(_profile, "JavaScanTime",
_connector_name.c_str());
-_fill_block_time = ADD_CHILD_TIMER(_profile, "FillBlockTime",
_connector_name.c_str());
+ADD_TIMER_WITH_LEVEL(_profile, _connector_name.c_str(), 1);
Review Comment:
do not change counter level
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](runtime-filter) add lock for RuntimeFilterConsumer::acquire_expr/signal to avoid mult… [doris]
BiteThet commented on PR #49739: URL: https://github.com/apache/doris/pull/49739#issuecomment-2781267348 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](profile) Refactor of RuntimeFilter profile [doris]
doris-robot commented on PR #49777: URL: https://github.com/apache/doris/pull/49777#issuecomment-2781269050 TeamCity cloud ut coverage result: Function Coverage: 83.07% (1089/1311) Line Coverage: 66.13% (18178/27488) Region Coverage: 65.56% (8954/13657) Branch Coverage: 55.41% (4825/8708) Coverage Report: http://coverage.selectdb-in.cc/coverage/c1ad005cd91bf500a8bd6452d8ca343775fb5eae_c1ad005cd91bf500a8bd6452d8ca343775fb5eae_cloud/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feat][test](nereids)(array) support array_reduce and regression test [doris]
cyer commented on PR #48994: URL: https://github.com/apache/doris/pull/48994#issuecomment-2781268917 removed duplicate code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](profile) Refactor of RuntimeFilter profile [doris]
zhiqiang- commented on PR #49777: URL: https://github.com/apache/doris/pull/49777#issuecomment-2781250796 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (9a0bf9e2617 -> 74bfacd9291)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 9a0bf9e2617 Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 new 74bfacd9291 Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (9a0bf9e2617) \ N -- N -- N refs/heads/asf-site (74bfacd9291) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-49565-branch-2.1 updated (74b15bae794 -> 4216c8e3b9f)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch auto-pick-49565-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git discard 74b15bae794 [feat](thirdparty) Add aws dentity-management and sts library for be and ry (#49565) add badd6f6e18a [Fix](case) Fix test base compaction case (#49693) add e5fe002805e [Bug](materialized-view) check duplicate expr when create mv stmt not have groupby exprs (#49595) (#49696) add f2dac1f876a branch-2.1: [fix](test) fix unstable test infer_intersect_except #49000 (#49709) add a9939c09c15 branch-2.1: [improve](thrift) Config thrift_max_message_size for FE SIMPLE and TH… #49678 (#49725) add e898dbbba02 branch-2.1: [fix](mc)Fixed the issue that maxcompute catalog can only read part of the timestamp data #49600 (#49706) add 997db43dda0 branch-2.1: [improve](thrift) Config thrift_max_message_size for THREAD_POOL and … #49677 (#49724) add f7f230dd34c branch-2.1: [fix](nereids)canInferNotNullForMarkSlot method get wrong result if fold constant rule is disabled (#49695) add cf847b8c711 [fix](load) return DataQualityError when filtered rows exceeds limit (#47617) (#49289) add 145e393d3db branch-2.1: [fix](function) check return type is nullptr in FunctionBasePtr::build #49737 (#49761) add 0735c19cdda branch-2.1: [fix](paimon) Covert Paimon DeletionFile Path to StoragePath in fe #49645 (#49751) add 3d10db4786a branch-2.1 [opt](nereids) set column stats unkown by default when derive Not expressoin #48864 (#49742) add 2b084295777 branch-2.1: [fix](meta) do not check replica allocation when replay #49569 (#49604) add 672829096e0 [chore](enhancement) remove atomic load shared ptr usage (#49781) add 15662c06e66 branch-2.1: [fix](auth)Ignore replay edit log error of auth #49348 (#49787) add 3a282bd3072 branch-2.1:[fix](auth)Delete from should not check select_priv (#49794) add 342b55afc2b branch-2.1: [fix](binlog) get table with db lock if the table not exists #49566 (#49648) add 486e4586182 branch-2.1: [case](mtmv)add case for insert overwrite of mtmv force drop partition #48946 (#49800) add 6974a8fc4ab branch-2.1: [opt](nereids)do not apply CSE(Common Sub Expression) upon multiDataSink #33746 (#49797) add c0bc16d88f0 [fix](function) wrong result of arrays_overlap (#49403) (#49707) add 0ea87fba6f0 branch-2.1: [Bug][function] fix the string cast jsonb cause null map have not init value #49810 (#49817) add 4216c8e3b9f [feat](thirdparty) Add aws dentity-management and sts library for be and ry (#49565) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (74b15bae794) \ N -- N -- N refs/heads/auto-pick-49565-branch-2.1 (4216c8e3b9f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: be/src/pipeline/task_queue.cpp | 16 +-- be/src/pipeline/task_queue.h | 6 +- be/src/runtime/runtime_state.cpp | 8 +- be/src/runtime/runtime_state.h | 2 +- be/src/util/thrift_server.cpp | 27 +++- be/src/vec/exec/format/csv/csv_reader.cpp | 21 ++- be/src/vec/exec/format/json/new_json_reader.cpp| 57 be/src/vec/exec/scan/vfile_scanner.cpp | 12 +- .../vec/functions/array/function_array_cum_sum.cpp | 10 +- .../functions/array/function_array_difference.h| 8 +- .../vec/functions/array/function_array_element.h | 6 +- .../array/function_array_enumerate_uniq.cpp| 1 - .../vec/functions/array/function_arrays_overlap.h | 87 +--- be/src/vec/functions/function.h| 8 ++ be/src/vec/functions/function_cast.h | 12 +- be/src/vec/sink/group_commit_block_sink.cpp| 14 +- be/src/vec/sink/vrow_distribution.cpp | 18 +-- be/src/vec/sink/vtablet_block_convertor.cpp| 47 +++ be/src/vec/sink/vtablet_block_convertor.h | 13 +- be/src/vec/sink/vtablet_finder.cpp | 16 +-- be/src/vec/sink/vtablet_finder.h | 4 +- .../vec/function/function_arrays_overlap_test.cpp | 63 +++-- be/test/vec/function/function_test_util.cpp| 3 + .../vec/function/simple_function_factory_test.cpp | 98 + .../doris/maxcompute/MaxComputeColumnVa
Re: [PR] branch-2.1: [feat](thirdparty) Add aws dentity-management and sts library for be and ry #49565 [doris]
yiguolei commented on PR #49684: URL: https://github.com/apache/doris/pull/49684#issuecomment-2781207572 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (00f5d741123 -> 97c39b0f951)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 00f5d741123 Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 new 97c39b0f951 Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (00f5d741123) \ N -- N -- N refs/heads/asf-site (97c39b0f951) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](paimon)Enhance the observability of split and JNI in profile [doris]
doris-robot commented on PR #49688: URL: https://github.com/apache/doris/pull/49688#issuecomment-2781232647 TeamCity cloud ut coverage result: Function Coverage: 83.07% (1089/1311) Line Coverage: 66.08% (18165/27488) Region Coverage: 65.52% (8948/13657) Branch Coverage: 55.36% (4821/8708) Coverage Report: http://coverage.selectdb-in.cc/coverage/29a330ffef20ee4b032cc8e080557af6941a6bcb_29a330ffef20ee4b032cc8e080557af6941a6bcb_cloud/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](paimon)Enhance the observability of split and JNI in profile [doris]
wuwenchi commented on PR #49688: URL: https://github.com/apache/doris/pull/49688#issuecomment-2781226537 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-48326-branch-3.0 updated (4e24298b1e8 -> f3168c8dbd7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-48326-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 4e24298b1e8 branch-3.0: [fix](test) fix unstable test infer_intersect_except #49000 (#49217) add f3168c8dbd7 [chore](ci) rm unused file (#48326) No new revisions were added by this update. Summary of changes: .../pipeline/common/check-pr-if-need-run-build.sh | 200 - 1 file changed, 200 deletions(-) delete mode 100755 regression-test/pipeline/common/check-pr-if-need-run-build.sh - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support format_round scala function [doris]
doris-robot commented on PR #49084: URL: https://github.com/apache/doris/pull/49084#issuecomment-2774746335 TPC-DS: Total hot run time: 187056 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 939c1a7e7b8011612117ae82eeae933b315a424b, data reload: false query1 995 504 468 468 query2 6558196919341934 query3 6741235 213 213 query4 26492 23867 23456 23456 query5 4380663 486 486 query6 325 218 219 218 query7 4620506 285 285 query8 349 240 234 234 query9 8620258725802580 query10 460 313 263 263 query11 15741 15024 14926 14926 query12 170 109 108 108 query13 1648518 383 383 query14 9557613862566138 query15 205 199 170 170 query16 7259641 484 484 query17 1164737 576 576 query18 1958396 298 298 query19 186 183 155 155 query20 118 121 116 116 query21 215 120 113 113 query22 4131402642154026 query23 33785 33045 33227 33045 query24 8534242724202420 query25 582 473 425 425 query26 1286268 150 150 query27 2752502 337 337 query28 4352243123812381 query29 796 592 462 462 query30 294 219 193 193 query31 961 842 779 779 query32 84 67 69 67 query33 582 367 321 321 query34 817 856 541 541 query35 802 821 731 731 query36 964 1003913 913 query37 120 107 76 76 query38 4332421740684068 query39 1462138814191388 query40 214 124 109 109 query41 56 55 52 52 query42 124 105 110 105 query43 499 517 486 486 query44 1347805 803 803 query45 176 185 169 169 query46 856 1022637 637 query47 1746178517081708 query48 379 422 295 295 query49 803 523 423 423 query50 675 693 422 422 query51 4186423840974097 query52 111 107 111 107 query53 232 272 184 184 query54 588 577 538 538 query55 83 83 80 80 query56 301 305 289 289 query57 1139114610581058 query58 272 268 262 262 query59 2629275125892589 query60 329 329 318 318 query61 136 126 128 126 query62 793 734 659 659 query63 232 189 194 189 query64 45151016735 735 query65 4332423242314231 query66 1154418 313 313 query67 15849 15500 15351 15351 query68 7946898 524 524 query69 468 313 275 275 query70 1194110510921092 query71 455 320 297 297 query72 5562474348424743 query73 668 646 350 350 query74 9108889187358735 query75 3839326527072707 query76 36341213858 858 query77 796 393 293 293 query78 10035 995092509250 query79 2017863 580 580 query80 605 513 474 474 query81 491 263 226 226 query82 468 128 97 97 query83 272 261 245 245 query84 250 114 91 91 query85 801 365 309 309 query86 341 325 301 301 query87 4485442943734373 query88 3790228422942284 query89 397 317 284 284 query90 2018228 228 228 query91 144 149 114 114 query92 77 66 56 56 query93 1643981 580 580 query94 684 477 319 319 query95 380 300 292 292 query96 497 581 280 280 query97 3145325131033103 query98 234 216 211 211 query99 1435139912231223 Total cold run time: 276032 ms Total hot run time: 187056 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [chore](log) Fix typo when meta-service checks priority_networks [doris]
github-actions[bot] commented on PR #49704: URL: https://github.com/apache/doris/pull/49704#issuecomment-2765684223 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-49165-branch-3.0 updated (1404d32f294 -> 4228f2df3a7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-49165-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 1404d32f294 branch-3.0: [fix](group commit)Fix replay wal fail problem on agg state type #49081 (#49145) add 4228f2df3a7 [Fix](cloud-mow) Fix FE's wrong handling when low version MS don't set tablet states for `GetDeleteBitmapUpdateLockResponse` (#49165) No new revisions were added by this update. Summary of changes: .../org/apache/doris/cloud/transaction/CloudGlobalTransactionMgr.java | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](mtmv) optimize mtmv rewrite performance [doris]
doris-robot commented on PR #49514: URL: https://github.com/apache/doris/pull/49514#issuecomment-2774509928 TPC-H: Total hot run time: 34411 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit cda01c290ab91fa41aaf5e092f61dad38d14d978, data reload: false -- Round 1 -- q1 26261 509950595059 q2 2079275 189 189 q3 10500 1234693 693 q4 10240 1002544 544 q5 7705237723742374 q6 189 164 133 133 q7 920 729 604 604 q8 9326130811701170 q9 6831513951065106 q10 6815232418901890 q11 500 288 259 259 q12 347 355 217 217 q13 17764 378631593159 q14 238 219 215 215 q15 524 490 484 484 q16 659 639 582 582 q17 605 865 370 370 q18 8069724171647164 q19 1735983 561 561 q20 334 342 233 233 q21 4090338224522452 q22 1056998 953 953 Total cold run time: 116787 ms Total hot run time: 34411 ms - Round 2, with runtime_filter_mode=off - q1 5181506151285061 q2 242 337 229 229 q3 2140266222752275 q4 1423189414741474 q5 4502439044164390 q6 226 170 127 127 q7 2018198717861786 q8 2661255224592459 q9 7257713471497134 q10 2970314427782778 q11 587 517 490 490 q12 703 756 625 625 q13 3509397933553355 q14 284 299 271 271 q15 536 488 478 478 q16 668 703 645 645 q17 1163153714141414 q18 7866755373657365 q19 817 813 865 813 q20 1951202918681868 q21 5253494147164716 q22 1062106910351035 Total cold run time: 53019 ms Total hot run time: 50788 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (c505e473585 -> 36d86cf6a01)
This is an automated email from the ASF dual-hosted git repository. hellostephen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from c505e473585 [fix](kerberos) Back to use principal and keytab to login kerberos instead of using kerberos ticket cache. (#48655) add 36d86cf6a01 Revert "[refactor](pipeline) Refine scheduling logics (#49180)" (#49279) No new revisions were added by this update. Summary of changes: be/src/pipeline/pipeline.cpp | 2 +- be/src/pipeline/pipeline_fragment_context.cpp | 2 +- be/src/pipeline/pipeline_task.cpp | 121 +- be/src/pipeline/pipeline_task.h | 50 --- be/src/pipeline/task_scheduler.cpp| 24 ++--- be/test/pipeline/pipeline_test.cpp| 6 +- 6 files changed, 78 insertions(+), 127 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](log) Standardize S3 failure log formats to enable critical operation monitoring [doris]
doris-robot commented on PR #49813: URL: https://github.com/apache/doris/pull/49813#issuecomment-2780965287 # BE UT Coverage Report Increment line coverage `22.73% (5/22)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/c4a98da2eb00a34cbbdd7e875f5d86bb3ba911fb_c4a98da2eb00a34cbbdd7e875f5d86bb3ba911fb/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/c4a98da2eb00a34cbbdd7e875f5d86bb3ba911fb_c4a98da2eb00a34cbbdd7e875f5d86bb3ba911fb/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.23% (13998/26799) | | Line Coverage | 40.97% (120608/294375) | | Region Coverage | 39.68% (61339/154566) | | Branch Coverage | 34.31% (30628/89262) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](cloud-mow)Add some metrics aboout delete bitmap update lock [doris]
doris-robot commented on PR #47988: URL: https://github.com/apache/doris/pull/47988#issuecomment-2742178015 TPC-DS: Total hot run time: 194048 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit af8c71680fb78aa883e7907ae16ded497032c0ff, data reload: false query1 1371106010381038 query2 6247197718911891 query3 11053 444243904390 query4 54692 25660 23177 23177 query5 5234609 484 484 query6 363 206 188 188 query7 4958476 291 291 query8 323 279 243 243 query9 5897258825902588 query10 403 295 249 249 query11 15174 15346 14821 14821 query12 153 108 105 105 query13 1066496 386 386 query14 11675 673369646733 query15 202 204 176 176 query16 7066670 497 497 query17 1090743 605 605 query18 1541411 320 320 query19 199 196 170 170 query20 128 127 122 122 query21 215 124 104 104 query22 4749477347444744 query23 34031 33169 33478 33169 query24 5732244223922392 query25 530 469 395 395 query26 677 269 147 147 query27 1918498 331 331 query28 2758247024762470 query29 568 589 428 428 query30 273 227 197 197 query31 888 864 794 794 query32 74 62 62 62 query33 449 373 313 313 query34 756 863 497 497 query35 799 846 757 757 query36 946 1014912 912 query37 118 107 77 77 query38 4313424841134113 query39 1506147513931393 query40 213 115 104 104 query41 54 56 49 49 query42 122 107 100 100 query43 504 498 485 485 query44 1312826 807 807 query45 185 171 165 165 query46 846 1040644 644 query47 1846194118181818 query48 386 408 305 305 query49 679 536 439 439 query50 743 761 418 418 query51 4261432342974297 query52 100 103 93 93 query53 226 254 186 186 query54 490 516 428 428 query55 85 86 81 81 query56 277 286 257 257 query57 1168119711211121 query58 247 245 238 238 query59 2887293528182818 query60 319 289 268 268 query61 127 161 129 129 query62 742 738 681 681 query63 218 177 187 177 query64 14971050702 702 query65 453943934393 query66 704 420 302 302 query67 15849 15479 15455 15455 query68 7392813 504 504 query69 541 307 263 263 query70 1175106811021068 query71 499 284 274 274 query72 5926521554005215 query73 1432690 349 349 query74 9017881190148811 query75 3865323827262726 query76 41871185759 759 query77 665 372 282 282 query78 10130 10121 93939393 query79 2640796 559 559 query80 741 516 516 516 query81 475 266 219 219 query82 684 119 95 95 query83 178 178 155 155 query84 281 94 74 74 query85 781 349 315 315 query86 368 313 277 277 query87 4443443044014401 query88 3528225122372237 query89 403 304 275 275 query90 1795205 204 204 query91 157 142 110 110 query92 74 58 54 54 query93 21831053579 579 query94 671 425 291 291 query95 346 267 271 267 query96 478 564 278 278 query97 3321340433203320 query98 256 212 209 209 query99 1421140612401240 Total cold run time: 300814 ms Total hot run time: 194048 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Improve](audit) sql that fails to be syntax parsed is not audited [doris]
doris-robot commented on PR #48527: URL: https://github.com/apache/doris/pull/48527#issuecomment-2737096699 TPC-H: Total hot run time: 32352 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f408571f8e664bf58ad76fda6a27780ed391e3a9, data reload: false -- Round 1 -- q1 24343 510050215021 q2 2051317 185 185 q3 10364 1247693 693 q4 10230 997 536 536 q5 7559239523462346 q6 184 162 132 132 q7 938 725 604 604 q8 9307123310781078 q9 4928481047184718 q10 6806229318701870 q11 492 287 256 256 q12 346 349 218 218 q13 17769 368530403040 q14 234 232 208 208 q15 534 478 486 478 q16 612 623 583 583 q17 576 864 344 344 q18 6723654063836383 q19 1215957 544 544 q20 320 329 198 198 q21 2821212719611961 q22 10671008956 956 Total cold run time: 109419 ms Total hot run time: 32352 ms - Round 2, with runtime_filter_mode=off - q1 5112509651155096 q2 235 327 240 240 q3 2144269323082308 q4 1386178213631363 q5 4243414844324148 q6 217 169 132 132 q7 2013196117631763 q8 2685251525482515 q9 7174726471827182 q10 2960320327612761 q11 600 507 486 486 q12 686 794 606 606 q13 3514383833163316 q14 274 295 262 262 q15 513 476 487 476 q16 635 698 646 646 q17 1172158413591359 q18 7661767975997599 q19 782 799 873 799 q20 1990201318781878 q21 5411495247034703 q22 1043102910171017 Total cold run time: 52450 ms Total hot run time: 50655 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
doris.git: Error while running github feature from main:.asf.yaml
An error occurred while processing the github feature in .asf.yaml: while parsing a mapping in "doris.git/.asf.yaml::github", line 78, column 1: - Build Third Party Librar ... ^ (line: 78) unexpected key not in schema 'required_pull_request_reviews' in "doris.git/.asf.yaml::github", line 81, column 1: ^ (line: 81) --- With regards, ASF Infra. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](pipeline) Make pipeline task shared by task queue [doris]
doris-robot commented on PR #49391: URL: https://github.com/apache/doris/pull/49391#issuecomment-2751539072 TPC-H: Total hot run time: 34330 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a13e22050183a8d04986ad0fc8aec0332d74de00, data reload: false -- Round 1 -- q1 26616 506350255025 q2 2097297 162 162 q3 10413 1295680 680 q4 10215 1042541 541 q5 7566237124012371 q6 182 163 133 133 q7 920 778 618 618 q8 9320126711341134 q9 6907514551695145 q10 6876235019201920 q11 486 288 260 260 q12 349 351 214 214 q13 17772 365230513051 q14 229 228 227 227 q15 551 501 477 477 q16 628 642 589 589 q17 585 867 337 337 q18 7578730572607260 q19 2141975 559 559 q20 320 335 196 196 q21 3940260324372437 q22 11011019994 994 Total cold run time: 116792 ms Total hot run time: 34330 ms - Round 2, with runtime_filter_mode=off - q1 5252513951005100 q2 245 336 234 234 q3 2224267223362336 q4 1419189815031503 q5 4599446244734462 q6 221 185 133 133 q7 1981192817581758 q8 2629259626292596 q9 7356707973767079 q10 2999320027742774 q11 594 508 512 508 q12 708 782 651 651 q13 3519392533383338 q14 285 297 273 273 q15 529 478 469 469 q16 659 717 675 675 q17 1168153014071407 q18 7821758476067584 q19 795 836 934 836 q20 1915202519181918 q21 5430488249414882 q22 1131109810151015 Total cold run time: 53479 ms Total hot run time: 51531 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](beut) add pipeline SetOperator beut [doris]
doris-robot commented on PR #49152: URL: https://github.com/apache/doris/pull/49152#issuecomment-2732877194 TPC-DS: Total hot run time: 192324 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6bac414a9a5b1c525f554aa62f416f9cbe56ecd9, data reload: false query1 1409104010481040 query2 6329194119671941 query3 11023 436746264367 query4 25702 23766 23313 23313 query5 3961653 497 497 query6 315 209 206 206 query7 3986507 293 293 query8 307 249 243 243 query9 8511259825932593 query10 485 303 244 244 query11 15464 15131 15051 15051 query12 162 107 101 101 query13 1560538 389 389 query14 9542628366736283 query15 208 184 177 177 query16 7588610 457 457 query17 1355792 611 611 query18 1998424 334 334 query19 204 189 170 170 query20 131 123 125 123 query21 211 132 117 117 query22 4467441345874413 query23 34474 33436 33548 33436 query24 8034242124242421 query25 490 449 414 414 query26 1207273 156 156 query27 2514497 347 347 query28 3988250324432443 query29 714 564 428 428 query30 282 224 194 194 query31 957 881 769 769 query32 72 64 62 62 query33 563 361 307 307 query34 790 883 500 500 query35 822 868 763 763 query36 10011032903 903 query37 121 103 79 79 query38 4362415541734155 query39 1479143314121412 query40 209 125 112 112 query41 55 53 52 52 query42 121 104 106 104 query43 512 528 488 488 query44 1352807 786 786 query45 182 172 171 171 query46 916 1041647 647 query47 1852186017781778 query48 377 423 329 329 query49 761 506 423 423 query50 706 772 414 414 query51 4279432442844284 query52 107 105 103 103 query53 247 267 189 189 query54 515 491 418 418 query55 85 83 82 82 query56 265 269 277 269 query57 1197116711051105 query58 240 241 235 235 query59 2829284327122712 query60 294 298 260 260 query61 121 117 116 116 query62 820 754 689 689 query63 230 183 187 183 query64 39681068675 675 query65 4647444945144449 query66 1088404 296 296 query67 16129 15534 15550 15534 query68 9762875 503 503 query69 466 344 258 258 query70 1177107710761076 query71 483 294 279 279 query72 5105362637683626 query73 801 763 356 356 query74 9216923689048904 query75 4272316026822682 query76 37241184724 724 query77 991 370 278 278 query78 10016 10316 93619361 query79 2299831 655 655 query80 628 531 438 438 query81 493 255 224 224 query82 681 126 91 91 query83 171 170 151 151 query84 229 88 69 69 query85 761 360 302 302 query86 328 302 290 290 query87 4506440543334333 query88 3269228722712271 query89 410 315 282 282 query90 1936290 215 215 query91 163 137 107 107 query92 70 59 56 56 query93 11751057593 593 query94 651 424 300 300 query95 357 277 252 252 query96 490 562 280 280 query97 3319336732373237 query98 221 206 193 193 query99 1592142512831283 Total cold run time: 280654 ms Total hot run time: 192324 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] Set runtime filter [doris]
hello-stephen commented on PR #49573: URL: https://github.com/apache/doris/pull/49573#issuecomment-2760639521 # BE UT Coverage Report Increment line coverage `32.84% (22/67)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/62a33d07d30527c09fa44cb32aa531ffcb156542_62a33d07d30527c09fa44cb32aa531ffcb156542/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/62a33d07d30527c09fa44cb32aa531ffcb156542_62a33d07d30527c09fa44cb32aa531ffcb156542/report/index.html) | Category | Coverage | |---|| | Function Coverage | 50.90% (13627/26773) | | Line Coverage | 40.28% (118310/293742) | | Region Coverage | 38.95% (60093/154289) | | Branch Coverage | 33.87% (30231/89266) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](lzo) fix lzo decompression failed [doris]
hello-stephen commented on PR #49538: URL: https://github.com/apache/doris/pull/49538#issuecomment-2758699296 # BE UT Coverage Report Increment line coverage `0.00% (0/8)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/ba623a20a3bbe8f45248ccead5c8f18b77f49f6e_ba623a20a3bbe8f45248ccead5c8f18b77f49f6e/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/ba623a20a3bbe8f45248ccead5c8f18b77f49f6e_ba623a20a3bbe8f45248ccead5c8f18b77f49f6e/report/index.html) | Category | Coverage | |---|| | Function Coverage | 50.89% (13624/26769) | | Line Coverage | 40.26% (118245/293693) | | Region Coverage | 38.94% (60071/154258) | | Branch Coverage | 33.86% (30213/89242) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement showRowPolicyCommand in nereids [doris]
starocean999 merged PR #49123: URL: https://github.com/apache/doris/pull/49123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](pipeline) Refactor state transition in pipeline task [doris]
doris-robot commented on PR #49281: URL: https://github.com/apache/doris/pull/49281#issuecomment-2738974281 TPC-H: Total hot run time: 32266 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 556f59a35b9866103fae289439059c7ccbe18379, data reload: false -- Round 1 -- q1 24504 514650725072 q2 2038329 167 167 q3 10371 1253734 734 q4 10225 1250540 540 q5 7560238823452345 q6 182 161 132 132 q7 927 736 608 608 q8 9310129810161016 q9 5015480347184718 q10 6793232118991899 q11 477 273 251 251 q12 347 351 215 215 q13 17750 363530963096 q14 229 234 211 211 q15 530 480 484 480 q16 630 617 587 587 q17 584 864 346 346 q18 7162649162406240 q19 1208955 537 537 q20 326 334 203 203 q21 2745209318971897 q22 11021015972 972 Total cold run time: 110015 ms Total hot run time: 32266 ms - Round 2, with runtime_filter_mode=off - q1 5255509951935099 q2 240 322 224 224 q3 2115268322852285 q4 1405179714301430 q5 4250418645334186 q6 217 169 124 124 q7 2020192117741774 q8 2649262624952495 q9 7200711171227111 q10 2997321227432743 q11 589 510 484 484 q12 687 755 642 642 q13 3513382332973297 q14 275 298 291 291 q15 531 468 493 468 q16 649 676 648 648 q17 1138155913781378 q18 7836743573697369 q19 806 767 907 767 q20 1965207219111911 q21 5510496646244624 q22 1096101110081008 Total cold run time: 52943 ms Total hot run time: 50358 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](inverted index) Clear inverted index cache from file cache #49685 [doris]
hello-stephen commented on PR #49738: URL: https://github.com/apache/doris/pull/49738#issuecomment-2768840477 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (cc216b3574e -> ac01b368d09)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard cc216b3574e Automated deployment with doris branch @ 1847e9f039b67a226281dda3abb79012b2f125cb new ac01b368d09 Automated deployment with doris branch @ 1847e9f039b67a226281dda3abb79012b2f125cb This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (cc216b3574e) \ N -- N -- N refs/heads/asf-site (ac01b368d09) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-48747-branch-2.1 created (now 17ffd301fbc)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-48747-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git at 17ffd301fbc [fix](mtmv) Fix collecting mv candidates when dml controlled by enable_dml_materialized_view_rewrite switch #48374 (#49263) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) fix extract common factor [doris]
yujun777 commented on PR #49774: URL: https://github.com/apache/doris/pull/49774#issuecomment-2772493317 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](cloud-mow)Add some metrics aboout delete bitmap update lock [doris]
doris-robot commented on PR #47988: URL: https://github.com/apache/doris/pull/47988#issuecomment-2733068357 TPC-H: Total hot run time: 32440 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 87c352e4a86c81a175bfa0fb41c03c72f35fa447, data reload: false -- Round 1 -- q1 24570 510150565056 q2 2052317 189 189 q3 10361 1288692 692 q4 10229 1006528 528 q5 7564238723592359 q6 196 166 134 134 q7 938 750 603 603 q8 9301123211011101 q9 4904484047134713 q10 6803230818761876 q11 479 270 258 258 q12 337 354 213 213 q13 17798 366130613061 q14 229 227 207 207 q15 543 482 477 477 q16 616 619 586 586 q17 562 852 366 366 q18 6797648263426342 q19 1226950 543 543 q20 326 328 197 197 q21 2836212619641964 q22 10801015975 975 Total cold run time: 109747 ms Total hot run time: 32440 ms - Round 2, with runtime_filter_mode=off - q1 5146509551465095 q2 243 327 239 239 q3 2160268923482348 q4 1420179513651365 q5 4207414144634141 q6 214 171 127 127 q7 2041193317791779 q8 2666264526152615 q9 7117719471187118 q10 2993319127402740 q11 571 509 487 487 q12 661 761 608 608 q13 3510383532123212 q14 289 294 271 271 q15 519 465 471 465 q16 653 689 648 648 q17 1125158513481348 q18 7766760775087508 q19 817 834 845 834 q20 2009208318451845 q21 5321483947754775 q22 1048103710011001 Total cold run time: 52496 ms Total hot run time: 50569 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](statistics)Fix replace table doesn't remove table stat meta memory leak bug. (#49345) [doris]
yiguolei merged PR #49367: URL: https://github.com/apache/doris/pull/49367 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](runtime-filter) add lock for RuntimeFilterConsumer::acquire_expr/signal to avoid mult… [doris]
BiteThet commented on PR #49739: URL: https://github.com/apache/doris/pull/49739#issuecomment-2775680225 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function) wrong result of arrays_overlap (#49403) [doris]
Thearas commented on PR #49707: URL: https://github.com/apache/doris/pull/49707#issuecomment-2765720708 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parquet) Hive 1.x column name sush as _col0, read empty result [doris]
doris-robot commented on PR #48390: URL: https://github.com/apache/doris/pull/48390#issuecomment-2739006811 ClickBench: Total hot run time: 31.69 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 2395290e2d12f9aec93f9d5e6be0138f7108d83c, data reload: false query1 0.040.040.02 query2 0.120.100.11 query3 0.240.190.19 query4 1.600.210.19 query5 0.610.590.60 query6 1.180.720.70 query7 0.020.020.02 query8 0.050.030.03 query9 0.590.510.52 query10 0.570.590.57 query11 0.150.110.11 query12 0.150.110.11 query13 0.610.610.60 query14 2.692.812.80 query15 0.940.850.84 query16 0.410.380.38 query17 1.041.071.07 query18 0.210.190.20 query19 1.952.081.83 query20 0.010.020.01 query21 15.38 0.910.57 query22 0.781.190.72 query23 14.77 1.430.64 query24 6.811.081.39 query25 0.490.290.06 query26 0.590.170.13 query27 0.050.050.05 query28 9.620.940.44 query29 12.60 4.023.34 query30 0.250.090.06 query31 2.830.620.38 query32 3.220.550.46 query33 2.952.983.04 query34 15.67 5.144.57 query35 4.604.554.59 query36 0.650.490.49 query37 0.090.060.06 query38 0.050.040.03 query39 0.030.020.02 query40 0.170.130.13 query41 0.080.020.02 query42 0.030.020.02 query43 0.040.030.03 Total cold run time: 104.93 s Total hot run time: 31.69 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)use analyzed plan instead of rewritten plan in CreateMaterializedViewCommand [doris]
doris-robot commented on PR #49297: URL: https://github.com/apache/doris/pull/49297#issuecomment-2739136121 TPC-DS: Total hot run time: 192015 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c92f0b5105d0257de1b11f07a7af966011a3b1ca, data reload: false query1 1450108410901084 query2 6167196119111911 query3 11140 476046404640 query4 27622 2 23008 23008 query5 4845655 471 471 query6 287 191 168 168 query7 3981505 292 292 query8 281 232 228 228 query9 8524258625842584 query10 486 313 261 261 query11 16173 15149 14763 14763 query12 157 111 108 108 query13 1718529 397 397 query14 10057 629467606294 query15 212 186 183 183 query16 7617650 453 453 query17 1492757 597 597 query18 2039405 312 312 query19 192 191 175 175 query20 122 141 131 131 query21 201 118 106 106 query22 4597453645644536 query23 34218 33619 33364 33364 query24 7112244924342434 query25 510 480 384 384 query26 1204271 159 159 query27 2518464 372 372 query28 4680244124242424 query29 705 554 449 449 query30 280 233 201 201 query31 906 861 769 769 query32 70 64 61 61 query33 510 350 301 301 query34 817 874 516 516 query35 824 847 759 759 query36 996 1012903 903 query37 117 95 72 72 query38 4394425743464257 query39 1470144414531444 query40 210 112 103 103 query41 55 51 50 50 query42 124 102 104 102 query43 519 526 477 477 query44 1402822 819 819 query45 188 175 172 172 query46 849 1042646 646 query47 1825190218121812 query48 417 453 302 302 query49 793 510 445 445 query50 714 744 422 422 query51 4256432543074307 query52 114 107 93 93 query53 226 252 191 191 query54 490 502 408 408 query55 84 83 81 81 query56 271 251 257 251 query57 11291211 query58 243 241 237 237 query59 2725283728172817 query60 278 292 271 271 query61 118 124 143 124 query62 772 750 668 668 query63 231 197 198 197 query64 40891103779 779 query65 46034625 query66 972 396 302 302 query67 16165 15433 15601 15433 query68 8529887 502 502 query69 475 288 255 255 query70 1218110611081106 query71 491 292 256 256 query72 5760355437693554 query73 788 732 338 338 query74 9061922887608760 query75 3825315626972697 query76 37311181729 729 query77 791 365 273 273 query78 10087 10050 93739373 query79 3269832 587 587 query80 734 518 448 448 query81 512 253 214 214 query82 772 125 99 99 query83 167 165 151 151 query84 246 98 86 86 query85 765 347 296 296 query86 372 308 276 276 query87 4461450442984298 query88 3586227022742270 query89 414 315 280 280 query90 1834217 232 217 query91 140 135 110 110 query92 77 59 54 54 query93 22421055578 578 query94 665 399 295 295 query95 359 290 256 256 query96 485 572 278 278 query97 3339336632823282 query98 221 205 228 205 query99 1421140212571257 Total cold run time: 285390 ms Total hot run time: 192015 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Fix](GA)Remove extra blank lines to avoid parsing [doris]
github-actions[bot] commented on PR #49615: URL: https://github.com/apache/doris/pull/49615#issuecomment-2760088238 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) fix generate column need add cast in stream load [doris]
doris-robot commented on PR #49167: URL: https://github.com/apache/doris/pull/49167#issuecomment-2739917700 ClickBench: Total hot run time: 31.05 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 09e59c1147a96d370f9fa41e5c9404c2f6b3bc12, data reload: false query1 0.040.030.03 query2 0.130.100.11 query3 0.240.190.20 query4 1.590.200.20 query5 0.590.580.57 query6 1.180.720.71 query7 0.030.020.01 query8 0.040.030.03 query9 0.590.530.53 query10 0.580.590.57 query11 0.150.100.11 query12 0.140.110.12 query13 0.620.590.60 query14 2.692.702.79 query15 0.950.850.85 query16 0.380.380.37 query17 1.041.021.05 query18 0.210.200.19 query19 1.911.941.81 query20 0.010.010.02 query21 15.36 0.900.53 query22 0.761.220.67 query23 14.95 1.390.65 query24 6.641.780.62 query25 0.480.340.07 query26 0.660.160.14 query27 0.050.050.05 query28 8.920.880.45 query29 12.55 3.973.32 query30 0.250.100.06 query31 2.820.570.38 query32 3.240.540.48 query33 3.053.053.08 query34 15.81 5.144.57 query35 4.594.574.54 query36 0.670.480.48 query37 0.080.060.06 query38 0.050.040.03 query39 0.030.020.03 query40 0.170.140.12 query41 0.080.030.02 query42 0.040.030.02 query43 0.040.030.03 Total cold run time: 104.4 s Total hot run time: 31.05 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] check_overflow_for_number_cast [doris]
Mryange commented on PR #49270: URL: https://github.com/apache/doris/pull/49270#issuecomment-2740031412 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](exec) opt variant column in join op [doris]
doris-robot commented on PR #49020: URL: https://github.com/apache/doris/pull/49020#issuecomment-2731624827 ClickBench: Total hot run time: 31.75 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 55a6531c20f5aaa661c87c9d721e22cd42dcc8fa, data reload: false query1 0.040.040.04 query2 0.130.100.10 query3 0.250.200.19 query4 1.600.110.11 query5 0.570.550.54 query6 1.200.700.73 query7 0.020.020.01 query8 0.040.030.03 query9 0.580.530.51 query10 0.610.570.58 query11 0.160.110.10 query12 0.140.120.11 query13 0.630.600.61 query14 2.702.812.80 query15 0.920.860.85 query16 0.380.400.38 query17 1.021.041.06 query18 0.210.200.19 query19 2.111.891.77 query20 0.010.020.02 query21 15.36 0.910.54 query22 0.741.180.65 query23 14.94 1.430.65 query24 6.712.281.37 query25 0.540.180.19 query26 0.550.150.12 query27 0.040.050.05 query28 10.32 0.780.43 query29 12.55 4.063.42 query30 0.250.090.06 query31 2.820.570.38 query32 3.230.540.46 query33 2.992.983.07 query34 15.79 5.084.47 query35 4.524.544.53 query36 0.670.490.49 query37 0.080.060.06 query38 0.050.040.04 query39 0.020.020.02 query40 0.180.130.12 query41 0.070.020.03 query42 0.030.020.04 query43 0.030.030.03 Total cold run time: 105.8 s Total hot run time: 31.75 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0-pick: [Fix](partial update) use correct default value for missing columns in partial update (#49066) [doris]
doris-robot commented on PR #49221: URL: https://github.com/apache/doris/pull/49221#issuecomment-2733580233 ClickBench: Total hot run time: 31.91 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6537d988b9b7a0df41d21221c75daf8e7bf6a127, data reload: false query1 0.030.020.03 query2 0.060.030.03 query3 0.230.070.07 query4 1.630.100.10 query5 0.510.490.50 query6 1.140.720.73 query7 0.020.020.01 query8 0.040.020.03 query9 0.570.500.50 query10 0.550.550.56 query11 0.140.110.10 query12 0.140.110.11 query13 0.600.590.60 query14 2.732.842.75 query15 0.910.840.83 query16 0.380.380.37 query17 1.010.961.05 query18 0.240.230.23 query19 1.901.871.94 query20 0.010.010.01 query21 15.36 0.570.57 query22 2.332.211.50 query23 17.14 0.800.75 query24 2.931.021.76 query25 0.260.250.14 query26 0.450.140.14 query27 0.040.050.04 query28 9.900.550.53 query29 12.58 3.253.25 query30 0.240.070.07 query31 2.860.390.39 query32 3.220.460.45 query33 2.973.033.01 query34 17.04 4.514.44 query35 4.544.514.49 query36 0.680.480.47 query37 0.080.060.06 query38 0.050.040.04 query39 0.030.030.02 query40 0.170.130.13 query41 0.080.030.02 query42 0.030.020.02 query43 0.030.030.03 Total cold run time: 105.85 s Total hot run time: 31.91 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix]fix report query statistics to FE cores [doris]
doris-robot commented on PR #49711: URL: https://github.com/apache/doris/pull/49711#issuecomment-2768041714 ClickBench: Total hot run time: 32.99 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 85cb071a07fd4da916f7f0f600e7515068328b9d, data reload: false query1 0.030.030.03 query2 0.070.030.03 query3 0.230.070.07 query4 1.630.100.10 query5 0.510.490.51 query6 1.130.730.73 query7 0.020.010.01 query8 0.040.030.05 query9 0.570.510.51 query10 0.550.550.55 query11 0.150.090.10 query12 0.140.100.12 query13 0.610.620.60 query14 2.742.872.74 query15 0.890.840.83 query16 0.390.390.39 query17 1.091.061.05 query18 0.230.230.24 query19 1.991.902.03 query20 0.020.000.01 query21 15.37 0.600.59 query22 2.572.402.08 query23 17.00 1.200.88 query24 3.011.691.15 query25 0.250.130.09 query26 0.480.140.13 query27 0.050.040.05 query28 9.710.490.49 query29 12.58 3.333.31 query30 0.250.060.06 query31 2.860.390.39 query32 3.250.480.46 query33 2.983.043.07 query34 17.14 4.474.51 query35 4.564.584.54 query36 0.670.480.51 query37 0.090.060.06 query38 0.040.040.04 query39 0.030.030.03 query40 0.170.120.13 query41 0.090.020.03 query42 0.030.020.02 query43 0.030.020.02 Total cold run time: 106.24 s Total hot run time: 32.99 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) project child output to union output in correct order after eliminate empty relation [doris]
englefly commented on code in PR #49257:
URL: https://github.com/apache/doris/pull/49257#discussion_r2004696307
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/EliminateEmptyRelation.java:
##
@@ -109,15 +109,21 @@ public List buildRules() {
if (union.getConstantExprsList().isEmpty()) {
Plan child = nonEmptyChildren.get(0);
List unionOutput = union.getOutput();
-List childOutput = child.getOutput();
-List projects = Lists.newArrayList();
-for (int i = 0; i < unionOutput.size(); i++) {
-ExprId id = unionOutput.get(i).getExprId();
-Alias alias = new Alias(id, childOutput.get(i),
unionOutput.get(i).getName());
-projects.add(alias);
-}
+int childIdx =
union.children().indexOf(nonEmptyChildren.get(0));
+if (childIdx >= 0) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](cloud-mow)Add some metrics aboout delete bitmap update lock [doris]
hust-hhb commented on code in PR #47988: URL: https://github.com/apache/doris/pull/47988#discussion_r2006735110 ## be/src/runtime/routine_load/routine_load_task_executor.cpp: ## @@ -440,7 +443,10 @@ void RoutineLoadTaskExecutor::exec_task(std::shared_ptr ctx, consumer_pool->return_consumers(consumer_grp.get()); // commit txn +int64_t commit_and_publish_start_time = MonotonicNanos(); Review Comment: record commit and publish time usually use MonotonicNanos to make it more precise, like StreamLoadContext::commit_and_publish_txn_cost_nanos -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-49259-branch-2.1 deleted (was 17ffd301fbc)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-49259-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git was 17ffd301fbc [fix](mtmv) Fix collecting mv candidates when dml controlled by enable_dml_materialized_view_rewrite switch #48374 (#49263) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Fix](cloud-mow) Check partition's version to avoid wrongly update visible versions' delete bitmaps (#49710)
This is an automated email from the ASF dual-hosted git repository.
zhangchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new a2979e60e34 [Fix](cloud-mow) Check partition's version to avoid
wrongly update visible versions' delete bitmaps (#49710)
a2979e60e34 is described below
commit a2979e60e34c9be602cc8c0a214d6e61dce2f16c
Author: bobhan1
AuthorDate: Thu Apr 3 16:56:56 2025 +0800
[Fix](cloud-mow) Check partition's version to avoid wrongly update visible
versions' delete bitmaps (#49710)
### What problem does this PR solve?
considering the following problem:
1. Transaction X acquires the lock and attempts to publish with version
a. This task is sent to the BE. At this point, the tablet's maximum
version is a-1, and task (1) starts computation.
2. Transaction X fails on FE due to timeout and releases the lock.
3. Transaction Y acquires the lock, attempts to publish with version a,
and succeeds.
4. Transaction X retries and acquires the lock again, and attempts to
publish with version b.
5. Meanwhile, task (1) from Transaction X completes its computation on
BE and writes the generated delete bitmap to the MS with version a.
**Since Transaction X currently holds the lock, this write operation
succeeds, overwriting the delete bitmaps written of actual version a by
Transaction Y.**
6. Subsequent transactions on the tablet will use the pending delete
bitmap to delete the version a delete bitmap written by task (1) in the
MS.
The root cause is that when a load txn retries in publish phase, the
locks it gains are different, but they are the same in the current
implementation because they have the same lock_id and initiator.
This PR checks target partition's version when update delete bitmaps to
avoid this problem.
---
.../cloud/cloud_engine_calc_delete_bitmap_task.cpp | 6 +-
be/src/cloud/cloud_meta_mgr.cpp| 6 +-
be/src/cloud/cloud_meta_mgr.h | 2 +-
be/src/cloud/cloud_tablet.cpp | 16 +-
be/src/cloud/cloud_tablet.h| 7 +-
be/src/olap/base_tablet.cpp| 7 +-
be/src/olap/base_tablet.h | 3 +-
be/src/olap/tablet.cpp | 3 +-
be/src/olap/tablet.h | 4 +-
cloud/src/meta-service/meta_service.cpp| 114 +-
cloud/src/meta-service/meta_service_txn.cpp| 1 +
cloud/test/meta_service_test.cpp | 384 -
gensrc/proto/cloud.proto | 4 +
13 files changed, 533 insertions(+), 24 deletions(-)
diff --git a/be/src/cloud/cloud_engine_calc_delete_bitmap_task.cpp
b/be/src/cloud/cloud_engine_calc_delete_bitmap_task.cpp
index 06ebf249edb..53638abffc2 100644
--- a/be/src/cloud/cloud_engine_calc_delete_bitmap_task.cpp
+++ b/be/src/cloud/cloud_engine_calc_delete_bitmap_task.cpp
@@ -313,8 +313,10 @@ Status CloudTabletCalcDeleteBitmapTask::_handle_rowset(
// we still need to update delete bitmap KVs to MS when we skip to
calcalate delete bitmaps,
// because the pending delete bitmap KVs in MS we wrote before may
have been removed and replaced by other txns
int64_t lock_id = txn_info.is_txn_load ? txn_info.lock_id : -1;
-RETURN_IF_ERROR(
-tablet->save_delete_bitmap_to_ms(version, transaction_id,
delete_bitmap, lock_id));
+int64_t next_visible_version =
+txn_info.is_txn_load ? txn_info.next_visible_version : version;
+RETURN_IF_ERROR(tablet->save_delete_bitmap_to_ms(version,
transaction_id, delete_bitmap,
+ lock_id,
next_visible_version));
LOG(INFO) << "tablet=" << _tablet_id << ", " << txn_str
<< ", publish_status=SUCCEED, not need to re-calculate
delete_bitmaps.";
diff --git a/be/src/cloud/cloud_meta_mgr.cpp b/be/src/cloud/cloud_meta_mgr.cpp
index 2ab18b9ad9d..839f46ce1cd 100644
--- a/be/src/cloud/cloud_meta_mgr.cpp
+++ b/be/src/cloud/cloud_meta_mgr.cpp
@@ -1190,7 +1190,8 @@ Status CloudMetaMgr::update_tablet_schema(int64_t
tablet_id, const TabletSchema&
Status CloudMetaMgr::update_delete_bitmap(const CloudTablet& tablet, int64_t
lock_id,
int64_t initiator, DeleteBitmap*
delete_bitmap,
- int64_t txn_id, bool
is_explicit_txn) {
+ int64_t txn_id, bool is_explicit_txn,
+ int64_t next_visible_version) {
VLOG_DEBUG << "update_delete_bitmap , tablet_id: " << tablet.tablet_id();
UpdateDeleteBitmapRequest req;
UpdateDeleteBitmapResponse res;
@@ -1204,6 +12
(doris) branch master updated (58b12f69113 -> 902b95b1bfa)
This is an automated email from the ASF dual-hosted git repository. kirs pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 58b12f69113 [fix](mc)Fixed the issue that maxcompute catalog can only read part of the timestamp data (#49600) add 902b95b1bfa [Fix](Catalog) Close system resources when dropping catalog (#49621) No new revisions were added by this update. Summary of changes: .../org/apache/doris/datasource/CatalogIf.java | 2 +- .../org/apache/doris/datasource/CatalogMgr.java| 2 +- .../apache/doris/datasource/ExternalCatalog.java | 26 -- .../doris/datasource/hive/HMSExternalCatalog.java | 15 +++-- .../datasource/iceberg/IcebergExternalCatalog.java | 8 +++ .../datasource/iceberg/IcebergMetadataOps.java | 3 +++ .../doris/datasource/jdbc/JdbcExternalCatalog.java | 8 ++- .../mysql/privilege/AccessControllerManager.java | 8 ++- 8 files changed, 59 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](nereids) optimize small sql [doris]
924060929 commented on PR #43546: URL: https://github.com/apache/doris/pull/43546#issuecomment-2760387364 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
doris.git: Error while running github feature from auto-pick-48374-branch-2.1:.asf.yaml
An error occurred while processing the github feature in .asf.yaml: while parsing a mapping in "doris.git/.asf.yaml::github", line 78, column 1: - Build Third Party Librar ... ^ (line: 78) unexpected key not in schema 'required_pull_request_reviews' in "doris.git/.asf.yaml::github", line 81, column 1: ^ (line: 81) --- With regards, ASF Infra. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) project child output to union output in correct order after eliminate empty relation [doris]
englefly commented on code in PR #49257:
URL: https://github.com/apache/doris/pull/49257#discussion_r2004698359
##
regression-test/suites/empty_relation/eliminate_empty.groovy:
##
@@ -320,4 +320,177 @@ suite("eliminate_empty") {
sql """drop table if exists table_5_undef_partitions2_keys3"""
sql """drop table if exists table_10_undef_partitions2_keys3"""
}
+
+// union(A, empty) => A
+sql """
+set disable_nereids_rules='';
+drop table dwd_bill_fact_bill_standard_info;
+drop table ods_bill_fact_bill_jingdong_coupon;
+CREATE TABLE `dwd_bill_fact_bill_standard_info` (
+`tenant_id` bigint NULL,
+`event_day` date NULL,
+`platform` varchar(60) NULL,
+`trade_order_no` varchar(192) NULL,
+`bill_order_no` varchar(150) NULL,
+`item_type` varchar(256) NULL,
+`fa_type` varchar(384) NULL,
+`id` bigint NULL,
+`bill_name` varchar(150) NULL,
+`bill_platform` varchar(60) NULL,
+`sub_bill_platform` varchar(20) NULL COMMENT '平台子类型 ',
+`account_no` varchar(150) NULL,
+`shop_id` bigint NULL,
+`shop_name` varchar(150) NULL,
+`business_year` int NULL,
+`business_mouth` int NULL,
+`import_date` datetime NULL,
+`input_method` varchar(60) NULL,
+`import_code` bigint NULL,
+`import_name` varchar(60) NULL,
+`original_order_no` varchar(150) NULL,
+`refund_order_no` varchar(50) NULL COMMENT '退款单号 ',
+`bill_type` varchar(60) NULL,
+`fa_order_no` varchar(120) NULL,
+`sub_fa_type` text NULL,
+`time_of_expense` datetime NULL,
+`refund_time` datetime NULL COMMENT '退款时间 ',
+`fa_settlement_time` datetime NULL,
+`goods_name` varchar(765) NULL,
+`opt_pay_account` varchar(150) NULL,
+`account_name` varchar(150) NULL,
+`actual_amount` decimal(19,4) NULL,
+`income_out_direction` tinyint NULL,
+`balance` decimal(19,4) NULL,
+`amount_unit` varchar(30) NULL,
+`amount_field_name` varchar(128) NULL COMMENT '金额字段名称 ',
+`currency_id` int NULL,
+`currency` varchar(30) NULL,
+`business_channel` varchar(120) NULL,
+`document_type` varchar(30) NULL,
+`remark` varchar(765) NULL,
+`abstracts` varchar(765) NULL,
+`sub_order_no` varchar(150) NULL,
+`seq` tinyint NULL,
+`original_id` bigint NULL,
+`original_goods_id` bigint NULL,
+`plat_spec_id` bigint NULL COMMENT '平台规格ID ',
+`create_time` datetime NULL,
+`update_time` datetime NULL,
+`creator_id` bigint NULL,
+`updater_id` bigint NULL,
+`version_no` bigint NULL,
+`is_delete_doris` tinyint NULL COMMENT 'doris删除标记'
+) ENGINE=OLAP
+UNIQUE KEY(`tenant_id`, `event_day`, `platform`, `trade_order_no`,
`bill_order_no`, `item_type`, `fa_type`)
+DISTRIBUTED BY HASH(`platform`) BUCKETS 1
+properties("replication_num" = "1");
+
+CREATE TABLE `ods_bill_fact_bill_jingdong_coupon` (
+`tenant_id` bigint NOT NULL COMMENT '商户号',
+`event_day` date NOT NULL COMMENT '分区',
+`shop_id` bigint NOT NULL COMMENT '商店id',
+`bill_coupon_id` varchar(150) NOT NULL COMMENT '生成优惠券号',
+`coupon_name` varchar(765) NULL COMMENT '优惠券名称',
+`shop_name` varchar(765) NULL COMMENT '店铺名称',
+`effect_date` varchar(150) NULL COMMENT '作用日期',
+`coupon_type` varchar(150) NULL COMMENT '优惠券类型',
+`coupon_batch_code` varchar(150) NULL,
+`coupon_code` varchar(150) NULL COMMENT '优惠券码',
+`sku_id` varchar(150) NULL,
+`money` decimal(19,4) NULL COMMENT '优惠券金额',
+`contribute_party` varchar(150) NULL COMMENT '承担方',
+`occur_time` datetime NULL,
+`billing_time` datetime NULL,
+`fee_settlement_time` datetime NULL,
+`fa_settlement_time` datetime NULL COMMENT '业务日期',
+`settlement_status` varchar(150) NULL COMMENT '结算状态',
+`account_id` varchar(150) NULL COMMENT '账户id',
+`account_no` varchar(150) NULL COMMENT '账号',
+`trade_order_no` varchar(150) NULL COMMENT '商品编号',
+`create_time` datetime NULL COMMENT '创建时间',
+`update_time` datetime NULL COMMENT '修改时间',
+`creator_id` bigint NULL COMMENT '创建人',
+`updater_id` bigint NULL COMMENT '分区',
+`currency_id` int NULL COMMENT '分区',
+`version_no` bigint NULL COMMENT '分区',
+`is_delete_doris` tinyint NULL COMMENT 'doris删除标记',
+`summary_number` varchar(150) NULL
+) ENGINE=OLAP
+UNIQUE KEY(`tenant_id`, `event_day`, `shop_id`, `bill_coupon_id`)
+COMMENT 'OLAP'
+DISTRIBUTED BY HASH(`shop_id`) BUCKETS 1
+properties("replication_num" = "1");
+
+insert into
dwd_bill_fact_bill_standard_info(tenant_id,shop_id,event_day,actual_amount)
value
(doris) branch auto-pick-49669-branch-2.1 deleted (was ed8531b99ad)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-49669-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git was ed8531b99ad branch-2.1: [feat](regression) inject debug points need run in nonConcurrent or docker suites #49581 (#49618) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](cloud-mow) Check partition's version to avoid wrongly update visible versions' delete bitmaps [doris]
bobhan1 commented on PR #49710: URL: https://github.com/apache/doris/pull/49710#issuecomment-2771287508 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](terminate) add terminate interface [doris]
github-actions[bot] commented on PR #49638: URL: https://github.com/apache/doris/pull/49638#issuecomment-2768690161 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](schema-change) Nested types should only support enlarging varchar length with light schema change [doris]
doris-robot commented on PR #49452: URL: https://github.com/apache/doris/pull/49452#issuecomment-2753649698 TPC-DS: Total hot run time: 192312 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 3c9986c18eab3de04363424bc4ca386ed4edb66e, data reload: false query1 1390103610241024 query2 6285191918901890 query3 11128 470547064705 query4 25813 23889 22935 22935 query5 5077682 495 495 query6 306 205 192 192 query7 3982484 272 272 query8 311 255 238 238 query9 8534260525932593 query10 501 299 256 256 query11 15517 15137 15055 15055 query12 164 114 111 111 query13 1559531 395 395 query14 9221623560686068 query15 199 195 171 171 query16 7593614 492 492 query17 1160786 601 601 query18 2038441 336 336 query19 206 251 155 155 query20 130 122 112 112 query21 207 119 107 107 query22 4661451642654265 query23 34300 33585 33470 33470 query24 7890239724052397 query25 502 469 415 415 query26 1172269 143 143 query27 2810506 329 329 query28 4924245524192419 query29 688 570 447 447 query30 289 230 192 192 query31 887 849 804 804 query32 81 66 65 65 query33 529 411 306 306 query34 779 839 525 525 query35 842 878 788 788 query36 992 1032932 932 query37 124 103 77 77 query38 4119419043034190 query39 1518144614181418 query40 212 118 103 103 query41 53 55 52 52 query42 120 107 114 107 query43 505 501 487 487 query44 1321801 807 801 query45 184 178 168 168 query46 850 1051648 648 query47 1815189518011801 query48 387 422 304 304 query49 764 541 426 426 query50 706 752 417 417 query51 4272435643274327 query52 110 110 100 100 query53 229 259 186 186 query54 503 482 404 404 query55 81 86 83 83 query56 274 285 268 268 query57 1168121611171117 query58 256 241 278 241 query59 2610292127092709 query60 322 282 258 258 query61 140 137 135 135 query62 823 729 681 681 query63 222 179 182 179 query64 39991055702 702 query65 4452436743994367 query66 1126400 301 301 query67 16289 15961 15251 15251 query68 9641876 509 509 query69 475 295 248 248 query70 1181108410991084 query71 464 298 263 263 query72 5167480546814681 query73 706 587 343 343 query74 9012914487868786 query75 4299324427082708 query76 43801183753 753 query77 1005381 273 273 query78 995710328 92659265 query79 1653816 560 560 query80 754 507 443 443 query81 470 276 226 226 query82 430 123 94 94 query83 206 170 164 164 query84 285 98 70 70 query85 751 350 310 310 query86 336 280 302 280 query87 4456463242924292 query88 2842222722202220 query89 377 322 277 277 query90 1958204 207 204 query91 138 141 109 109 query92 73 59 55 55 query93 11881032588 588 query94 670 397 311 311 query95 359 269 329 269 query96 498 554 275 275 query97 3154323031653165 query98 217 208 208 208 query99 1457139612711271 Total cold run time: 280780 ms Total hot run time: 192312 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] branch-3.0: [fix](statistics)Fix replace table doesn't remove table stat meta memory leak bug. #49345 [doris]
hello-stephen commented on PR #49357: URL: https://github.com/apache/doris/pull/49357#issuecomment-2744935228 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (5a143e196bf -> 7f6f5af040f)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 5a143e196bf Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 new 7f6f5af040f Automated deployment with doris branch @ 0aae514566d8c54fc21945df2a2555fb7d65bd35 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (5a143e196bf) \ N -- N -- N refs/heads/asf-site (7f6f5af040f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [profile](rpc) add RpcInstanceDetails profile to debug rpc consumption problem #43284 [doris]
Mryange closed pull request #44978: [profile](rpc) add RpcInstanceDetails profile to debug rpc consumption problem #43284 URL: https://github.com/apache/doris/pull/44978 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](case) Fix cold data compaction fault injection case [doris]
Yukang-Lian commented on PR #49699: URL: https://github.com/apache/doris/pull/49699#issuecomment-2765535605 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
doris.git: Error while running github feature from branch-c108335-hive-sql:.asf.yaml
An error occurred while processing the github feature in .asf.yaml: while parsing a mapping in "doris.git/.asf.yaml::github", line 111, column 1: - Build Broker ^ (line: 111) unexpected key not in schema 'required_pull_request_reviews' in "doris.git/.asf.yaml::github", line 114, column 1: - COMPILE (DORIS_COMPILE) ^ (line: 114) --- With regards, ASF Infra. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: branch-2.1: [improve](thrift) Config thrift_max_message_size for FE SIMPLE and TH… #49678 (#49725)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new a9939c09c15 branch-2.1: [improve](thrift) Config
thrift_max_message_size for FE SIMPLE and TH… #49678 (#49725)
a9939c09c15 is described below
commit a9939c09c1584254a6d1bf887df45b21037aee9f
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Tue Apr 1 17:03:48 2025 +0800
branch-2.1: [improve](thrift) Config thrift_max_message_size for FE SIMPLE
and TH… #49678 (#49725)
Cherry-picked from #49678
Co-authored-by: walter
---
.../java/org/apache/doris/common/ThriftServer.java | 51 +-
1 file changed, 49 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
b/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
index f18dbb378a1..cdc4bba71d3 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
@@ -31,6 +31,7 @@ import org.apache.thrift.server.TSimpleServer;
import org.apache.thrift.server.TThreadPoolServer;
import org.apache.thrift.server.TThreadedSelectorServer;
import org.apache.thrift.transport.TNonblockingServerSocket;
+import org.apache.thrift.transport.TNonblockingSocket;
import org.apache.thrift.transport.TServerSocket;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransportException;
@@ -95,14 +96,42 @@ public class ThriftServer {
}
private void createSimpleServer() throws TTransportException {
-TServer.Args args = new TServer.Args(new
TServerSocket(port)).protocolFactory(
+TServerSocket.ServerSocketTransportArgs socketTransportArgs;
+
+if (FrontendOptions.isBindIPV6()) {
+socketTransportArgs = new TServerSocket.ServerSocketTransportArgs()
+.bindAddr(new InetSocketAddress("::0", port))
+.clientTimeout(Config.thrift_client_timeout_ms)
+.backlog(Config.thrift_backlog_num);
+} else {
+socketTransportArgs = new TServerSocket.ServerSocketTransportArgs()
+.bindAddr(new InetSocketAddress("0.0.0.0", port))
+.clientTimeout(Config.thrift_client_timeout_ms)
+.backlog(Config.thrift_backlog_num);
+}
+
+TServer.Args args = new TServer.Args(new
ImprovedTServerSocket(socketTransportArgs)).protocolFactory(
new TBinaryProtocol.Factory()).processor(processor);
server = new TSimpleServer(args);
}
private void createThreadedServer() throws TTransportException {
+TNonblockingServerSocket.NonblockingAbstractServerSocketArgs
socketTransportArgs;
+
+if (FrontendOptions.isBindIPV6()) {
+socketTransportArgs = new
TNonblockingServerSocket.NonblockingAbstractServerSocketArgs()
+.bindAddr(new InetSocketAddress("::0", port))
+.clientTimeout(Config.thrift_client_timeout_ms)
+.backlog(Config.thrift_backlog_num);
+} else {
+socketTransportArgs = new
TNonblockingServerSocket.NonblockingAbstractServerSocketArgs()
+.bindAddr(new InetSocketAddress("0.0.0.0", port))
+.clientTimeout(Config.thrift_client_timeout_ms)
+.backlog(Config.thrift_backlog_num);
+}
+
TThreadedSelectorServer.Args args = new TThreadedSelectorServer.Args(
-new TNonblockingServerSocket(port,
Config.thrift_client_timeout_ms))
+new ImprovedTNonblockingServerSocket(socketTransportArgs))
.protocolFactory(new TBinaryProtocol.Factory())
.processor(processor);
ThreadPoolExecutor threadPoolExecutor =
ThreadPoolManager.newDaemonCacheThreadPool(
@@ -214,4 +243,22 @@ public class ThriftServer {
return socket;
}
}
+
+static class ImprovedTNonblockingServerSocket extends
TNonblockingServerSocket {
+public
ImprovedTNonblockingServerSocket(NonblockingAbstractServerSocketArgs args)
throws TTransportException {
+super(args);
+}
+
+@Override
+public TNonblockingSocket accept() throws TTransportException {
+TNonblockingSocket socket = super.accept();
+
+TConfiguration cfg = socket.getConfiguration();
+cfg.setMaxMessageSize(Config.thrift_max_message_size);
+cfg.setMaxFrameSize(Config.thrift_max_frame_size);
+
+socket.updateKnownMessageSize(0); // Since we update the
configuration, reset consumed message size.
+return socket;
+}
+}
}
--
Re: [PR] [Enhancement](Compaction) Add cumulative compaction delay strategy [doris]
TangSiyang2001 commented on PR #48759: URL: https://github.com/apache/doris/pull/48759#issuecomment-2765704411 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](connector) Fixed the issue where the loading task got stuck when stream load ended unexpectedly [doris-spark-connector]
gnehil commented on code in PR #305:
URL:
https://github.com/apache/doris-spark-connector/pull/305#discussion_r2027089294
##
spark-doris-connector/spark-doris-connector-base/src/main/java/org/apache/doris/spark/client/write/AbstractStreamLoadProcessor.java:
##
@@ -132,57 +117,82 @@ public AbstractStreamLoadProcessor(DorisConfig config)
throws Exception {
// init stream load props
this.isTwoPhaseCommitEnabled =
config.getValue(DorisOptions.DORIS_SINK_ENABLE_2PC);
this.format = DataFormat.valueOf(properties.getOrDefault("format",
"csv").toUpperCase());
-this.isGzipCompressionEnabled =
properties.containsKey("compress_type") &&
"gzip".equals(properties.get("compress_type"));
+this.isGzipCompressionEnabled =
+properties.containsKey("compress_type") &&
"gzip".equals(properties.get("compress_type"));
if (properties.containsKey(GROUP_COMMIT)) {
String message = "";
-if (isTwoPhaseCommitEnabled) message = "group commit does not
support two-phase commit";
-if (properties.containsKey(PARTIAL_COLUMNS) &&
"true".equalsIgnoreCase(properties.get(PARTIAL_COLUMNS)))
+if (isTwoPhaseCommitEnabled) {
+message = "group commit does not support two-phase commit";
+}
+if (properties.containsKey(PARTIAL_COLUMNS) &&
"true".equalsIgnoreCase(properties.get(PARTIAL_COLUMNS))) {
message = "group commit does not support partial column
updates";
-if
(!VALID_GROUP_MODE.contains(properties.get(GROUP_COMMIT).toLowerCase()))
+}
+if
(!VALID_GROUP_MODE.contains(properties.get(GROUP_COMMIT).toLowerCase())) {
message = "Unsupported group commit mode: " +
properties.get(GROUP_COMMIT);
-if (!message.isEmpty()) throw new
IllegalArgumentException(message);
+}
+if (!message.isEmpty()) {
+throw new IllegalArgumentException(message);
+}
groupCommit = properties.get(GROUP_COMMIT).toLowerCase();
}
this.isPassThrough =
config.getValue(DorisOptions.DORIS_SINK_STREAMING_PASSTHROUGH);
}
public void load(R row) throws Exception {
+if (unexpectedException != null) {
+throw unexpectedException;
+}
Review Comment:
This check was removed because unexpected http request completion has
nothing to do with whether data was written or the buffer size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-49074-branch-2.1 created (now 7355cfe8b64)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-49074-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git at 7355cfe8b64 [Chore](runtime-filter) remove wrong check and set disable when SyncSizeClosure meet eof status (#49186) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](fe) optimize duplicate replicas properties checking [doris]
doris-robot commented on PR #49708: URL: https://github.com/apache/doris/pull/49708#issuecomment-2774282182 TPC-DS: Total hot run time: 193413 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit fbc892e97b0788cdfccd4121e4e956e91ec62076, data reload: false query1 1402108610791079 query2 6121192519041904 query3 11012 449544634463 query4 53672 25395 23413 23413 query5 5054519 462 462 query6 342 201 186 186 query7 4853492 282 282 query8 291 229 236 229 query9 5393260426082604 query10 459 329 265 265 query11 15278 15137 14882 14882 query12 164 110 102 102 query13 1052511 406 406 query14 10262 638964816389 query15 204 211 177 177 query16 7097646 511 511 query17 1094761 597 597 query18 1574428 335 335 query19 196 201 170 170 query20 134 128 128 128 query21 207 136 109 109 query22 4613463742544254 query23 33897 33080 33302 33080 query24 6583243524302430 query25 483 466 410 410 query26 736 276 154 154 query27 2386515 336 336 query28 3090245524412441 query29 609 555 440 440 query30 278 225 189 189 query31 876 865 790 790 query32 75 63 63 63 query33 463 359 319 319 query34 776 866 515 515 query35 808 853 779 779 query36 921 1017939 939 query37 123 103 77 77 query38 4177439243704370 query39 1466144114551441 query40 219 123 112 112 query41 54 62 54 54 query42 128 120 115 115 query43 503 513 484 484 query44 1373820 807 807 query45 186 177 188 177 query46 857 1055640 640 query47 1808190917811781 query48 395 431 324 324 query49 680 532 423 423 query50 672 694 404 404 query51 4255422541934193 query52 111 110 106 106 query53 234 262 180 180 query54 578 595 527 527 query55 84 84 83 83 query56 329 323 293 293 query57 1183120711241124 query58 269 277 261 261 query59 2943286228472847 query60 334 326 295 295 query61 131 138 170 138 query62 754 754 698 698 query63 242 192 195 192 query64 19471075699 699 query65 4404434043144314 query66 787 436 317 317 query67 16065 15530 15485 15485 query68 8476849 516 516 query69 537 300 268 268 query70 1221112210761076 query71 499 322 287 287 query72 5946477448764774 query73 1341647 354 354 query74 8891904986288628 query75 3844318327622762 query76 42331189781 781 query77 607 391 302 302 query78 994710001 93259325 query79 6050804 566 566 query80 687 522 449 449 query81 494 257 229 229 query82 701 134 101 101 query83 358 352 239 239 query84 299 110 95 95 query85 819 344 306 306 query86 373 298 304 298 query87 4421453643084308 query88 2901224322462243 query89 457 310 284 284 query90 1936218 222 218 query91 141 138 109 109 query92 72 62 55 55 query93 3532941 568 568 query94 718 407 323 323 query95 369 308 295 295 query96 498 559 270 270 query97 3201322031233123 query98 213 208 207 207 query99 1404139912611261 Total cold run time: 304932 ms Total hot run time: 193413 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [Fix](Compaction) Cumulative Points Not Incrementing After Delete Operation [doris]
github-actions[bot] commented on PR #47282: URL: https://github.com/apache/doris/pull/47282#issuecomment-2753371741 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [opt](nereids) set column stats unkown by default when derive Not expressoin #48864 [doris]
doris-robot commented on PR #49023: URL: https://github.com/apache/doris/pull/49023#issuecomment-2732097663 ClickBench: Total hot run time: 31.92 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 3d6b90a58dea0a3babd19c3de6badfbdefb07dd7, data reload: false query1 0.030.030.03 query2 0.060.030.03 query3 0.230.060.06 query4 1.630.100.10 query5 0.490.510.51 query6 1.140.720.72 query7 0.020.020.01 query8 0.040.030.03 query9 0.580.500.51 query10 0.550.560.55 query11 0.140.100.10 query12 0.130.120.11 query13 0.620.600.60 query14 2.862.892.77 query15 0.900.830.82 query16 0.390.380.37 query17 1.061.051.04 query18 0.240.210.20 query19 1.911.892.03 query20 0.010.010.02 query21 15.36 0.590.57 query22 2.532.781.46 query23 17.15 0.970.78 query24 3.271.121.04 query25 0.190.040.07 query26 0.600.150.14 query27 0.060.050.04 query28 10.06 0.540.44 query29 12.62 3.243.26 query30 0.250.060.06 query31 2.860.380.37 query32 3.260.460.46 query33 2.983.013.06 query34 16.91 4.514.49 query35 4.564.554.55 query36 0.680.480.50 query37 0.080.060.06 query38 0.040.030.03 query39 0.040.020.02 query40 0.160.130.12 query41 0.080.030.02 query42 0.040.020.02 query43 0.030.030.04 Total cold run time: 106.84 s Total hot run time: 31.92 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
doris.git: Error while running github feature from branch-2.1:.asf.yaml
An error occurred while processing the github feature in .asf.yaml: while parsing a mapping in "doris.git/.asf.yaml::github", line 78, column 1: - Build Third Party Librar ... ^ (line: 78) unexpected key not in schema 'required_pull_request_reviews' in "doris.git/.asf.yaml::github", line 81, column 1: ^ (line: 81) --- With regards, ASF Infra. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
doris.git: Error while running github feature from main:.asf.yaml
An error occurred while processing the github feature in .asf.yaml: while parsing a mapping in "doris.git/.asf.yaml::github", line 72, column 1: - BE UT (Doris BE UT) ^ (line: 72) unexpected key not in schema 'required_pull_request_reviews' in "doris.git/.asf.yaml::github", line 75, column 1: required_pull_request_reviews: ^ (line: 75) --- With regards, ASF Infra. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](iceberg)Optimize the batch mode conditions for Iceberg. [doris]
doris-robot commented on PR #49489: URL: https://github.com/apache/doris/pull/49489#issuecomment-2760051827 TPC-H: Total hot run time: 33916 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9dae4648216ba3bbc43fbf2598591dd18af3efcf, data reload: false -- Round 1 -- q1 25800 509955465099 q2 2082280 156 156 q3 10429 1232687 687 q4 10227 987 523 523 q5 7517240423102310 q6 183 158 135 135 q7 922 743 605 605 q8 9329128310721072 q9 6853525851495149 q10 6838231818701870 q11 478 280 258 258 q12 355 353 213 213 q13 17767 366230413041 q14 232 225 209 209 q15 524 485 485 485 q16 632 626 576 576 q17 559 849 338 338 q18 7487725571187118 q19 1769941 558 558 q20 306 327 192 192 q21 3830258923472347 q22 10781051975 975 Total cold run time: 115197 ms Total hot run time: 33916 ms - Round 2, with runtime_filter_mode=off - q1 5212522351565156 q2 243 325 242 242 q3 2263274323852385 q4 1424187014321432 q5 4527450045444500 q6 213 170 130 130 q7 1970190517481748 q8 2582262925362536 q9 7221727171467146 q10 3002319427392739 q11 598 523 486 486 q12 718 761 618 618 q13 3501394233233323 q14 291 292 268 268 q15 520 478 468 468 q16 630 697 625 625 q17 1140151613481348 q18 7949758675237523 q19 809 814 859 814 q20 1909199718471847 q21 5260487047654765 q22 1123108410181018 Total cold run time: 53105 ms Total hot run time: 51117 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](runtime-filter) add lock for RuntimeFilterConsumer::acquire_expr/signal to avoid mult… [doris]
github-actions[bot] commented on PR #49739: URL: https://github.com/apache/doris/pull/49739#issuecomment-2771140057 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0: [fix](cloud) fix corner case when warm up data larger than cache capacity #49050 [doris]
github-actions[bot] commented on PR #49675: URL: https://github.com/apache/doris/pull/49675#issuecomment-2765275010 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-46152-branch-3.0 deleted (was 03768332ada)
This is an automated email from the ASF dual-hosted git repository. kirs pushed a change to branch auto-pick-46152-branch-3.0 in repository https://gitbox.apache.org/repos/asf/doris.git was 03768332ada [fix](cross join) fix memory usage counter (#46152) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](be) split the usage of thread pool in internal service [doris]
github-actions[bot] commented on PR #49292: URL: https://github.com/apache/doris/pull/49292#issuecomment-2739958941 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement kill connection and query command in nereids [doris]
github-actions[bot] commented on PR #46882: URL: https://github.com/apache/doris/pull/46882#issuecomment-2771458428 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (0db47c86e51 -> d076d5b40d6)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 0db47c86e51 Automated deployment with doris branch @ f97c83201ec0b0fcc63672eeec89cefeaafa91a9 new d076d5b40d6 Automated deployment with doris branch @ f97c83201ec0b0fcc63672eeec89cefeaafa91a9 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (0db47c86e51) \ N -- N -- N refs/heads/asf-site (d076d5b40d6) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](backup) Save snapshot meta during replay [doris]
doris-robot commented on PR #49550: URL: https://github.com/apache/doris/pull/49550#issuecomment-2754604711 TPC-DS: Total hot run time: 192562 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c8974b054a03905919f2cdf0697e47b7955bb8a9, data reload: false query1 1420107410201020 query2 6238189919131899 query3 11095 472945484548 query4 25741 23829 22957 22957 query5 4416672 468 468 query6 298 192 186 186 query7 3986488 266 266 query8 294 229 228 228 query9 8503252525232523 query10 469 314 261 261 query11 15628 15025 14818 14818 query12 169 113 107 107 query13 1580527 410 410 query14 8943618162616181 query15 215 181 168 168 query16 7613629 515 515 query17 1212686 582 582 query18 2022419 334 334 query19 192 195 153 153 query20 128 116 114 114 query21 206 125 107 107 query22 4722470143034303 query23 34302 33861 33531 33531 query24 8550241023952395 query25 520 486 411 411 query26 1201270 140 140 query27 2939509 326 326 query28 4796241324182413 query29 689 566 457 457 query30 282 228 189 189 query31 920 856 810 810 query32 78 59 63 59 query33 563 386 324 324 query34 807 871 520 520 query35 858 915 749 749 query36 968 1006909 909 query37 116 104 81 81 query38 4237423842384238 query39 1503150714111411 query40 212 122 109 109 query41 55 87 52 52 query42 120 114 106 106 query43 495 503 470 470 query44 1329814 815 814 query45 184 174 165 165 query46 859 1043650 650 query47 1867191218291829 query48 397 414 315 315 query49 784 533 432 432 query50 731 758 425 425 query51 4318425542144214 query52 114 105 101 101 query53 224 260 174 174 query54 496 490 427 427 query55 80 79 80 79 query56 280 271 268 268 query57 1184121111401140 query58 261 237 240 237 query59 2834294227802780 query60 301 298 274 274 query61 133 134 126 126 query62 785 751 665 665 query63 226 187 184 184 query64 40391044720 720 query65 4491438343894383 query66 994 395 299 299 query67 16276 15645 15394 15394 query68 8201874 515 515 query69 480 301 259 259 query70 1145113811141114 query71 456 295 265 265 query72 5128462846134613 query73 649 576 346 346 query74 9039916389378937 query75 3971323327082708 query76 37641180757 757 query77 804 362 278 278 query78 979510077 92859285 query79 1222817 567 567 query80 602 517 447 447 query81 474 251 223 223 query82 191 124 93 93 query83 177 175 155 155 query84 232 93 73 73 query85 735 380 306 306 query86 327 301 282 282 query87 4374454143684368 query88 2804224522642245 query89 385 316 285 285 query90 2042211 208 208 query91 155 155 210 155 query92 72 62 55 55 query93 11441067578 578 query94 653 413 308 308 query95 357 264 268 264 query96 489 560 272 272 query97 3116326831993199 query98 238 209 198 198 query99 1439140912611261 Total cold run time: 277148 ms Total hot run time: 192562 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [BE](ut) add be ut about partition sort and analytic operator [doris]
github-actions[bot] commented on PR #49598: URL: https://github.com/apache/doris/pull/49598#issuecomment-2760441739 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [fix](Nereids) fold constant for string function process emoji character by mistake #49087 [doris]
github-actions[bot] commented on PR #49344: URL: https://github.com/apache/doris/pull/49344#issuecomment-2743793000 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)create alias function should fail when meet unsupported expr [doris]
doris-robot commented on PR #49698: URL: https://github.com/apache/doris/pull/49698#issuecomment-2766772630 TPC-DS: Total hot run time: 193084 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 3d53c88d1fe15b887299d052f26617e69565bba4, data reload: false query1 1425107910381038 query2 6332188719091887 query3 11134 462146454621 query4 25360 24047 23784 23784 query5 5377683 490 490 query6 347 206 186 186 query7 3979494 271 271 query8 305 246 223 223 query9 8515257725772577 query10 504 304 252 252 query11 15426 15229 14928 14928 query12 165 111 109 109 query13 1569522 396 396 query14 8858619161896189 query15 207 185 176 176 query16 7230639 489 489 query17 1185770 602 602 query18 2096421 339 339 query19 207 207 189 189 query20 133 123 129 123 query21 212 128 112 112 query22 4537466943374337 query23 34125 33563 33414 33414 query24 8691241623942394 query25 509 458 424 424 query26 799 273 146 146 query27 3183535 327 327 query28 4833243924312431 query29 661 577 452 452 query30 273 221 206 206 query31 883 844 776 776 query32 67 66 64 64 query33 554 360 317 317 query34 776 874 526 526 query35 842 842 803 803 query36 971 999 903 903 query37 131 104 79 79 query38 4139416440624062 query39 1503149714381438 query40 206 119 118 118 query41 56 53 54 53 query42 125 108 105 105 query43 508 508 496 496 query44 1313806 808 806 query45 182 175 167 167 query46 846 1034657 657 query47 1866192018011801 query48 376 425 321 321 query49 761 518 429 429 query50 687 783 427 427 query51 4243429642764276 query52 105 109 101 101 query53 226 254 186 186 query54 492 503 422 422 query55 90 82 84 82 query56 293 273 320 273 query57 1184121610931093 query58 256 259 242 242 query59 2613284726752675 query60 291 289 278 278 query61 135 134 132 132 query62 773 717 661 661 query63 224 179 186 179 query64 31491050697 697 query65 4413437743794377 query66 796 395 290 290 query67 16392 15486 15175 15175 query68 9606874 509 509 query69 548 309 257 257 query70 1227111010831083 query71 476 300 275 275 query72 5245470246884688 query73 712 559 349 349 query74 8944911487188718 query75 4547322627312731 query76 42171201769 769 query77 998 386 281 281 query78 996410157 92849284 query79 1632826 552 552 query80 657 528 475 475 query81 478 252 222 222 query82 371 126 103 103 query83 202 176 165 165 query84 280 100 80 80 query85 741 425 309 309 query86 328 316 282 282 query87 4456447444164416 query88 2793224822622248 query89 377 303 285 285 query90 1928209 215 209 query91 150 138 120 120 query92 69 60 61 60 query93 11731062582 582 query94 667 428 308 308 query95 340 281 262 262 query96 488 563 273 273 query97 3150320031113111 query98 240 209 203 203 query99 1421144312761276 Total cold run time: 279069 ms Total hot run time: 193084 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
(doris-website) branch asf-site updated (e9152cf32cb -> 0abf50de8c4)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard e9152cf32cb Automated deployment with doris branch @ 5a3477cb6511f4d41781a712f8e00dde7fa9a436 new 0abf50de8c4 Automated deployment with doris branch @ 5a3477cb6511f4d41781a712f8e00dde7fa9a436 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (e9152cf32cb) \ N -- N -- N refs/heads/asf-site (0abf50de8c4) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch asf-site updated (631893cf96c -> 8ba2d10dab3)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/doris-website.git discard 631893cf96c Automated deployment with doris branch @ 8f44f14392dbf303557426698fb03e666f69e68c new 8ba2d10dab3 Automated deployment with doris branch @ 8f44f14392dbf303557426698fb03e666f69e68c This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (631893cf96c) \ N -- N -- N refs/heads/asf-site (8ba2d10dab3) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/2.0/search-index.json | 2 +- docs/3.0/search-index.json | 2 +- docs/dev/search-index.json | 2 +- search-index.json| 2 +- zh-CN/docs/2.0/search-index.json | 2 +- zh-CN/docs/3.0/search-index.json | 2 +- zh-CN/docs/dev/search-index.json | 2 +- zh-CN/search-index.json | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
doris.git: Error while running github feature from auto-pick-49325-branch-3.0:.asf.yaml
An error occurred while processing the github feature in .asf.yaml: while parsing a mapping in "doris.git/.asf.yaml::github", line 72, column 1: - BE UT (Doris BE UT) ^ (line: 72) unexpected key not in schema 'required_pull_request_reviews' in "doris.git/.asf.yaml::github", line 75, column 1: required_pull_request_reviews: ^ (line: 75) --- With regards, ASF Infra. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](hive)Add a variable to control whether to use the cache. [doris]
github-actions[bot] commented on PR #49201: URL: https://github.com/apache/doris/pull/49201#issuecomment-2746804950 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Test](cbq) Record err and print at the end of check_before_quit test [doris]
dataroaring merged PR #49406: URL: https://github.com/apache/doris/pull/49406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
doris.git: Error while running github feature from branch-3.0:.asf.yaml
An error occurred while processing the github feature in .asf.yaml: while parsing a mapping in "doris.git/.asf.yaml::github", line 72, column 1: - BE UT (Doris BE UT) ^ (line: 72) unexpected key not in schema 'required_pull_request_reviews' in "doris.git/.asf.yaml::github", line 75, column 1: required_pull_request_reviews: ^ (line: 75) --- With regards, ASF Infra. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch auto-pick-49051-branch-2.1 created (now 54b3000de51)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch auto-pick-49051-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git at 54b3000de51 [fix](auth)create view check select_priv of table instead of column (#49268) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] (nereids)implement installPluginCommand in nereids [doris]
msridhar78 commented on PR #48589: URL: https://github.com/apache/doris/pull/48589#issuecomment-2752023091 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Nereids) support turn off ONLY_FULL_GROUP_BY sql mode [doris]
morrySnow commented on PR #49341: URL: https://github.com/apache/doris/pull/49341#issuecomment-2742278934 run buildal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](thrift) remove some unused query options [doris]
HappenLee merged PR #49254: URL: https://github.com/apache/doris/pull/49254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feat](nereids) support show columns command [doris]
xiedeyantu commented on PR #49591: URL: https://github.com/apache/doris/pull/49591#issuecomment-277553 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-2.1: [Opt](bvar) Add bvar for txn tablet map #49567 [doris]
dataroaring closed pull request #49627: branch-2.1: [Opt](bvar) Add bvar for txn tablet map #49567 URL: https://github.com/apache/doris/pull/49627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)Fix dlog1, trim, extract_url_parameter and parse_url FE constant calculate bug. (#49074) [doris]
Thearas commented on PR #49225: URL: https://github.com/apache/doris/pull/49225#issuecomment-2733466533 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] branch-3.0-pick: [Fix](cloud-mow) Check partition's version to avoid wrongly update visible versions' delete bitmaps (#49710) [doris]
doris-robot commented on PR #49796: URL: https://github.com/apache/doris/pull/49796#issuecomment-2775563742 TPC-H: Total hot run time: 40506 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 092c710fbb7a389bd68d3ea58291c0a603b3e1ee, data reload: false -- Round 1 -- q1 17639 714766626662 q2 2071173 181 173 q3 10651 110711931107 q4 10576 769 788 769 q5 7748297229112911 q6 227 138 142 138 q7 966 618 614 614 q8 9375200720772007 q9 6626642663956395 q10 6998225623362256 q11 472 261 264 261 q12 424 218 206 206 q13 17778 299729802980 q14 232 218 206 206 q15 501 456 468 456 q16 668 585 580 580 q17 1020618 573 573 q18 7371676566756675 q19 1398111411721114 q20 479 201 203 201 q21 4020324933473249 q22 1071973 990 973 Total cold run time: 108311 ms Total hot run time: 40506 ms - Round 2, with runtime_filter_mode=off - q1 6586660965696569 q2 334 241 237 237 q3 2905276429762764 q4 2023181118381811 q5 5729579657475747 q6 209 128 129 128 q7 2236182018211820 q8 3405361135913591 q9 8705896588698869 q10 3588353335413533 q11 604 498 491 491 q12 809 620 616 616 q13 9974325531893189 q14 307 270 270 270 q15 521 472 461 461 q16 674 655 640 640 q17 1862164116401640 q18 8378796876707670 q19 1668154616061546 q20 2120189318881888 q21 5438535453555354 q22 1016975 975 Total cold run time: 69186 ms Total hot run time: 59809 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improve](audit) sql that fails to be syntax parsed is not audited [doris]
dqz123 commented on PR #48527: URL: https://github.com/apache/doris/pull/48527#issuecomment-2738763636 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: branch-2.1: [fix](hudi) Fix Memory Leak in BitCaskDiskMap Due to Circular Reference #48955 (#49115)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new abc535a9e7e branch-2.1: [fix](hudi) Fix Memory Leak in BitCaskDiskMap
Due to Circular Reference #48955 (#49115)
abc535a9e7e is described below
commit abc535a9e7eb93edee104128a2a8fbbfbfb7ba1e
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Mar 21 22:44:35 2025 +0800
branch-2.1: [fix](hudi) Fix Memory Leak in BitCaskDiskMap Due to Circular
Reference #48955 (#49115)
Cherry-picked from #48955
Co-authored-by: Socrates
---
fe/check/checkstyle/suppressions.xml | 3 +
.../hudi/common/util/collection/DiskMap.java | 169 +
2 files changed, 172 insertions(+)
diff --git a/fe/check/checkstyle/suppressions.xml
b/fe/check/checkstyle/suppressions.xml
index 436b598b594..bcd034531c6 100644
--- a/fe/check/checkstyle/suppressions.xml
+++ b/fe/check/checkstyle/suppressions.xml
@@ -65,4 +65,7 @@ under the License.
+
+
+
diff --git
a/fe/fe-core/src/main/java/org/apache/hudi/common/util/collection/DiskMap.java
b/fe/fe-core/src/main/java/org/apache/hudi/common/util/collection/DiskMap.java
new file mode 100644
index 000..3069cda9591
--- /dev/null
+++
b/fe/fe-core/src/main/java/org/apache/hudi/common/util/collection/DiskMap.java
@@ -0,0 +1,169 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.util.collection;
+
+import org.apache.hudi.common.util.FileIOUtils;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.Serializable;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Stream;
+
+/* Copied From
+ *
https://github.com/apache/hudi/blob/release-0.15.0/hudi-common/src/main/java/org/apache/hudi/common/util/collection/DiskMap.java
+ * Doris Modification.
+ * Use Static cleaner class to avoid circular references in shutdown hooks
+ */
+
+/**
+ * This interface provides the map interface for storing records in disk after
+ * they
+ * spill over from memory. Used by {@link ExternalSpillableMap}.
+ *
+ * @param The generic type of the keys
+ * @param The generic type of the values
+ */
+public abstract class DiskMap
implements Map, Iterable {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DiskMap.class);
+ private static final String SUBFOLDER_PREFIX = "hudi";
+ private final File diskMapPathFile;
+ private transient Thread shutdownThread = null;
+
+ // Base path for the write file
+ protected final String diskMapPath;
+
+ public DiskMap(String basePath, String prefix) throws IOException {
+this.diskMapPath = String.format("%s/%s-%s-%s", basePath,
SUBFOLDER_PREFIX, prefix, UUID.randomUUID().toString());
+diskMapPathFile = new File(diskMapPath);
+FileIOUtils.deleteDirectory(diskMapPathFile);
+FileIOUtils.mkdir(diskMapPathFile);
+// Make sure the folder is deleted when JVM exits
+diskMapPathFile.deleteOnExit();
+addShutDownHook();
+ }
+
+ /**
+ * Register shutdown hook to force flush contents of the data written to
+ * FileOutputStream from OS page cache
+ * (typically 4 KB) to disk.
+ */
+ private void addShutDownHook() {
+// Register this disk map path with the static cleaner instead of using an
+// instance-specific hook
+DiskMapCleaner.registerForCleanup(diskMapPath);
+ }
+
+ /**
+ * @returns a stream of the values stored in the disk.
+ */
+ abstract Stream valueStream();
+
+ /**
+ * Number of bytes spilled to disk.
+ */
+ abstract long sizeOfFileOnDiskInBytes();
+
+ /**
+ * Close and cleanup the Map.
+ */
+ public void close() {
+cleanup(false);
+ }
+
+ /**
+ * Cleanup all resources, files and folders
+ * triggered by shutdownhook.
+ */
+ private void cleanup() {
+cleanup(true);
+ }
+
+ /**
+ * Cleanup all resources, files and fold
