Re: [PR] [Docs](Community)Use the standard ASF Code of Conduct template [doris]
github-actions[bot] commented on PR #29586: URL: https://github.com/apache/doris/pull/29586#issuecomment-1878271526 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Docs](Community)Use the standard ASF Code of Conduct template [doris]
github-actions[bot] commented on PR #29586: URL: https://github.com/apache/doris/pull/29586#issuecomment-1878271567 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regresion) fix 2.0 regression catch unsupported hive formatted exception [doris]
zy-kkk merged PR #29569: URL: https://github.com/apache/doris/pull/29569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](regresion) fix 2.0 regression catch unsupported hive formatted exception (#29569)
This is an automated email from the ASF dual-hosted git repository. zykkk pushed a commit to branch branch-2.0 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.0 by this push: new 0fcb7dd5037 [fix](regresion) fix 2.0 regression catch unsupported hive formatted exception (#29569) 0fcb7dd5037 is described below commit 0fcb7dd503707088775583edc3180a796f21eed1 Author: slothever <18522955+w...@users.noreply.github.com> AuthorDate: Fri Jan 5 16:01:11 2024 +0800 [fix](regresion) fix 2.0 regression catch unsupported hive formatted exception (#29569) --- .../suites/external_table_p2/hive/test_external_catalog_hive.groovy | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy b/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy index e15b833f06c..ef2b5a32e42 100644 --- a/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy +++ b/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy @@ -113,7 +113,9 @@ suite("test_external_catalog_hive", "p2") { try { sql """ select * from multi_catalog.unsupported_input_format_empty; """ } catch (Exception e) { -assertTrue(e.getMessage().contains("Unsupported hive input format: com.hadoop.mapred.DeprecatedLzoTextInputFormat")) +if (!e.getMessage().contains("Nonsupport hive metastore table")) { +assertTrue(e.getMessage().contains("Unsupported hive input format: com.hadoop.mapred.DeprecatedLzoTextInputFormat")) +} } // test remember last used database after switch / rename catalog - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) create double literal when create decimal literal failed [doris]
doris-robot commented on PR #28959: URL: https://github.com/apache/doris/pull/28959#issuecomment-1878274276 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit dba3fd83ad6b4c78173302af3d7eaa132f73ad54, data reload: false run tpch-sf100 query with default conf and session variables q1 5402513151715131 q2 391 172 150 150 q3 1477120711831183 q4 1085835 831 831 q5 3079297331022973 q6 226 133 134 133 q7 986 533 523 523 q8 2164229122562256 q9 6702665466636654 q10 3155309631263096 q11 342 225 215 215 q12 376 232 239 232 q13 4404364236383638 q14 246 214 219 214 q15 576 538 524 524 q16 442 390 418 390 q17 1040528 495 495 q18 7095671967006700 q19 1634149215301492 q20 588 331 332 331 q21 2923248824842484 q22 404 332 342 332 Total cold run time: 44737 ms Total hot run time: 39977 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 5116511850905090 q2 336 236 236 236 q3 3371334033253325 q4 2167202620402026 q5 5972591759125912 q6 223 128 126 126 q7 2394193419341934 q8 3573365836723658 q9 9124897790348977 q10 3898389739083897 q11 590 473 487 473 q12 792 640 649 640 q13 3881316931813169 q14 315 277 269 269 q15 583 532 523 523 q16 566 470 528 470 q17 2018180718291807 q18 8773841183978397 q19 1751166216911662 q20 2261198819761976 q21 5753537752995299 q22 595 471 499 471 Total cold run time: 64052 ms Total hot run time: 60337 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442592684 ## gensrc/thrift/FrontendService.thrift: ## @@ -470,6 +475,8 @@ struct TReportExecStatusParams { 23: optional list detailed_report 24: optional TQueryStatistics query_statistics + + 25: TReportWorkloadRuntimeStatusParams report_workload_runtime_stats Review Comment: TReportQueryRuntimeStatusParams -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feature](mtmv)mtmv support workload group [doris]
zddr opened a new pull request, #29595: URL: https://github.com/apache/doris/pull/29595 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](mtmv)mtmv support workload group [doris]
zddr commented on PR #29595: URL: https://github.com/apache/doris/pull/29595#issuecomment-1878277074 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Opt](parquet-reader) Opt `ColumnSelectVector::set_run_length_null_map()` in parquet reader. [doris]
kaka11chen opened a new pull request, #29596: URL: https://github.com/apache/doris/pull/29596 ## Proposed changes Backport from #29527 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] (opt)(match_phrase) Optimizing match_phrase with a new algorithm [doris]
zzzxl1993 commented on PR #29444: URL: https://github.com/apache/doris/pull/29444#issuecomment-1878277478 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](parquet-reader) Opt `ColumnSelectVector::set_run_length_null_map()` in parquet reader. [doris]
kaka11chen commented on PR #29596: URL: https://github.com/apache/doris/pull/29596#issuecomment-1878277507 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442594753 ## be/src/common/daemon.cpp: ## @@ -352,6 +354,65 @@ void Daemon::block_spill_gc_thread() { } } +void Daemon::report_be_workload_runtime_stats_thread() { Review Comment: 这里不要叫workload stats, 因为我们未来可能真的会有一个东东叫workload stats,比如我们可能按照workload group 级别向fe 汇报每个workload group的stats -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](dynamic partition) fix dynamic partition storage medium not working [doris]
doris-robot commented on PR #29490: URL: https://github.com/apache/doris/pull/29490#issuecomment-1878280701 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 0bcb3b278c5c8b3d6917a9e23153938d598bda78, data reload: false run tpch-sf100 query with default conf and session variables q1 5446517851775177 q2 418 156 161 156 q3 1472111511691115 q4 1075839 869 839 q5 3125315431813154 q6 221 133 139 133 q7 966 552 518 518 q8 2158221322242213 q9 6731665666946656 q10 3202313630703070 q11 368 217 228 217 q12 385 236 230 230 q13 4362362536853625 q14 249 218 209 209 q15 575 531 540 531 q16 454 395 417 395 q17 1050508 474 474 q18 7070676167346734 q19 1652152815431528 q20 560 331 337 331 q21 2857255224562456 q22 399 310 347 310 Total cold run time: 44795 ms Total hot run time: 40071 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 5204512950435043 q2 341 245 228 228 q3 3364333833133313 q4 2153202820182018 q5 5959593459135913 q6 223 126 132 126 q7 2380189719151897 q8 3577367436583658 q9 9080902590149014 q10 3885390039333900 q11 584 509 476 476 q12 798 649 632 632 q13 3897315531823155 q14 299 290 281 281 q15 594 539 524 524 q16 556 521 512 512 q17 2023179918151799 q18 8737832183948321 q19 1744169716671667 q20 2284197919761976 q21 5738533052855285 q22 549 465 463 463 Total cold run time: 63969 ms Total hot run time: 60201 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](group_commit) refector wal manager code [doris]
doris-robot commented on PR #29560: URL: https://github.com/apache/doris/pull/29560#issuecomment-1878280651 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit da2076f22c04e4a515a45ba87c11592670f9ead7, data reload: false -- Round 1 -- q1 17839 526351445144 q2 2017155 142 142 q3 10620 113411581134 q4 10231 773 819 773 q5 7842299329462946 q6 215 132 130 130 q7 915 511 541 511 q8 9258205520442044 q9 6868647964466446 q10 8324307830353035 q11 425 214 202 202 q12 384 229 227 227 q13 18000 362536373625 q14 240 215 213 213 q15 554 530 526 526 q16 472 407 396 396 q17 981 481 550 481 q18 7323682466796679 q19 1599139613351335 q20 699 330 322 322 q21 2822234324622343 q22 368 321 335 321 Total cold run time: 107996 ms Total hot run time: 38975 ms - Round 2, with runtime_filter_mode=off - q1 5180511651085108 q2 333 223 233 223 q3 3320329432653265 q4 2146203220312031 q5 5784580958275809 q6 216 124 123 123 q7 2316191619161916 q8 3383346934853469 q9 8816879187038703 q10 3801383138433831 q11 566 492 486 486 q12 806 646 624 624 q13 10507 320831903190 q14 294 322 283 283 q15 585 512 531 512 q16 593 504 502 502 q17 1947178017491749 q18 8664833083418330 q19 1631158515741574 q20 2204195719451945 q21 5693541653225322 q22 566 461 496 461 Total cold run time: 69351 ms Total hot run time: 59456 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442597338 ## be/src/common/daemon.cpp: ## @@ -352,6 +354,65 @@ void Daemon::block_spill_gc_thread() { } } +void Daemon::report_be_workload_runtime_stats_thread() { +while (!_stop_background_threads_latch.wait_for(std::chrono::milliseconds(1000))) { Review Comment: make this as a be conf -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](insert-into) record rows info in log for check [doris]
github-actions[bot] commented on PR #29581: URL: https://github.com/apache/doris/pull/29581#issuecomment-1878282277 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](insert-into) record rows info in log for check [doris]
github-actions[bot] commented on PR #29581: URL: https://github.com/apache/doris/pull/29581#issuecomment-1878282310 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](group commit) Add an option of enable relay wal on all regression test [doris]
doris-robot commented on PR #28757: URL: https://github.com/apache/doris/pull/28757#issuecomment-1878282379 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 0e49204b0dcf983784710fe9b58314312ae68ed8, data reload: false -- Round 1 -- q1 18183 543750935093 q2 2025158 141 141 q3 10609 113211261126 q4 10312 826 878 826 q5 7815299529102910 q6 217 132 133 132 q7 939 543 556 543 q8 9279201820372018 q9 6895640864256408 q10 8260299530722995 q11 426 229 214 214 q12 390 227 228 227 q13 18016 362736353627 q14 249 214 209 209 q15 558 529 517 517 q16 475 433 384 384 q17 974 465 503 465 q18 7373678467816781 q19 1586138513871385 q20 724 341 328 328 q21 2778232624062326 q22 365 321 320 320 Total cold run time: 108448 ms Total hot run time: 38975 ms - Round 2, with runtime_filter_mode=off - q1 5165512851145114 q2 345 229 251 229 q3 3314328733013287 q4 2149198520181985 q5 5853579857815781 q6 219 125 127 125 q7 2353187719051877 q8 3383345334683453 q9 8816881287698769 q10 3791383138523831 q11 580 483 483 483 q12 794 636 646 636 q13 9238322231823182 q14 300 262 269 262 q15 576 526 518 518 q16 556 504 500 500 q17 1948177517941775 q18 8705840582878287 q19 1625161015781578 q20 2208197719701970 q21 5591527852385238 q22 546 482 524 482 Total cold run time: 68055 ms Total hot run time: 59362 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](group commit) Fix wal mem back pressure fault injection case [doris]
Yukang-Lian commented on PR #29493: URL: https://github.com/apache/doris/pull/29493#issuecomment-1878282973 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Docs](Community)Use the standard ASF Code of Conduct template [doris]
dataroaring merged PR #29586: URL: https://github.com/apache/doris/pull/29586 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt](parquet-reader) Opt `ColumnSelectVector::set_run_length_null_map()` in parquet reader. [doris]
github-actions[bot] commented on PR #29596: URL: https://github.com/apache/doris/pull/29596#issuecomment-1878283767 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (0eee560f945 -> bf4e6787955)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 0eee560f945 [enhancement](Nereids): add test for push filter through operator (#27294) add bf4e6787955 [Docs](Community)Use the standard ASF Code of Conduct template (#29586) No new revisions were added by this update. Summary of changes: CODE_OF_CONDUCT.md | 97 +++--- 1 file changed, 4 insertions(+), 93 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Error while running github feature from .asf.yaml in doris!
An error occurred while running github feature in .asf.yaml!: You can only have a maximum of 10 external triage collaborators, please contact vp-in...@apache.org to request an exception. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442598601 ## be/src/common/daemon.cpp: ## @@ -352,6 +354,65 @@ void Daemon::block_spill_gc_thread() { } } +void Daemon::report_be_workload_runtime_stats_thread() { +while (!_stop_background_threads_latch.wait_for(std::chrono::milliseconds(1000))) { +std::map fe_id_query_stats; +// 1 get merged query statistics +{ +// add a code block here because we should release query_ctx_map quickly +std::unordered_map>& query_ctx_map = + ExecEnv::GetInstance()->fragment_mgr()->get_query_ctx_map(); + +for (const auto& [query_id, query_ctx_ptr] : query_ctx_map) { +std::unique_ptr query_stats = +query_ctx_ptr->get_merged_query_statistics(); +TNetworkAddress addr = query_ctx_ptr->coord_addr; +if (fe_id_query_stats.find(addr) == fe_id_query_stats.end()) { +TReportWorkloadRuntimeStatusParams treport_params; +treport_params.__set_backend_id(123); Review Comment: BackendOptions::tbackend struct TBackend { 1: required string host 2: required TPort be_port 3: required TPort http_port 4: optional TPort brpc_port 5: optional bool is_alive 6: optional i64 id } -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](nereids)need do type coercion after simplify comparasion predicate [doris]
starocean999 opened a new pull request, #29597: URL: https://github.com/apache/doris/pull/29597 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids)need do type coercion after simplify comparasion predicate [doris]
starocean999 commented on PR #29597: URL: https://github.com/apache/doris/pull/29597#issuecomment-1878285018 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group_commit) Fix group_commit ut [doris]
github-actions[bot] commented on PR #29587: URL: https://github.com/apache/doris/pull/29587#issuecomment-1878285806 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442599860 ## be/src/common/daemon.cpp: ## @@ -352,6 +354,65 @@ void Daemon::block_spill_gc_thread() { } } +void Daemon::report_be_workload_runtime_stats_thread() { +while (!_stop_background_threads_latch.wait_for(std::chrono::milliseconds(1000))) { +std::map fe_id_query_stats; +// 1 get merged query statistics +{ +// add a code block here because we should release query_ctx_map quickly +std::unordered_map>& query_ctx_map = + ExecEnv::GetInstance()->fragment_mgr()->get_query_ctx_map(); + +for (const auto& [query_id, query_ctx_ptr] : query_ctx_map) { +std::unique_ptr query_stats = +query_ctx_ptr->get_merged_query_statistics(); +TNetworkAddress addr = query_ctx_ptr->coord_addr; +if (fe_id_query_stats.find(addr) == fe_id_query_stats.end()) { +TReportWorkloadRuntimeStatusParams treport_params; +treport_params.__set_backend_id(123); +fe_id_query_stats[addr] = treport_params; +} + +TQueryStatistics tquery_stats; +tquery_stats.__set_scan_rows(query_stats->get_scan_rows()); Review Comment: add a method in QueryStatistics name toThrift(TQueryStatistics) to encapsulate this set_scan_rows logic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group_commit) Fix group_commit ut [doris]
github-actions[bot] commented on PR #29587: URL: https://github.com/apache/doris/pull/29587#issuecomment-1878285838 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](complextype)fix compare_at base function support nested types [doris]
eldenmoon commented on code in PR #29297: URL: https://github.com/apache/doris/pull/29297#discussion_r1442600036 ## be/src/vec/columns/column_struct.cpp: ## @@ -185,6 +185,27 @@ const char* ColumnStruct::deserialize_and_insert_from_arena(const char* pos) { return pos; } +int ColumnStruct::compare_at(size_t n, size_t m, const IColumn& rhs_, + int nan_direction_hint) const { +const ColumnStruct& rhs = assert_cast(rhs_); + +const size_t lhs_tuple_size = columns.size(); +const size_t rhs_tuple_size = rhs.tuple_size(); + +if (rhs_tuple_size < lhs_tuple_size) { Review Comment: is it correct to compare tuple size first? why not compare element first? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [optimize](cooldown)If cooldowned_rowsets is empty, there is no need to follow the logic of traversing the files [doris]
doris-robot commented on PR #27324: URL: https://github.com/apache/doris/pull/27324#issuecomment-1878287056 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Tpch sf100 test result on commit 133855f49ed7a00e2e4a030fdcc343491787a28e, data reload: false run tpch-sf100 query with default conf and session variables q1 5507512251535122 q2 392 167 163 163 q3 1456118011611161 q4 1097822 803 803 q5 3128314631203120 q6 224 134 137 134 q7 985 552 529 529 q8 2141220422182204 q9 6668667266586658 q10 3198309131263091 q11 356 217 225 217 q12 394 244 245 244 q13 4408368336453645 q14 255 221 215 215 q15 583 539 532 532 q16 473 400 414 400 q17 1051554 479 479 q18 7079696069346934 q19 1647155814841484 q20 558 353 343 343 q21 2888246025182460 q22 410 329 331 329 Total cold run time: 44898 ms Total hot run time: 40267 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 5115509950315031 q2 335 238 227 227 q3 3378331533193315 q4 2186204920402040 q5 5970591259305912 q6 225 127 133 127 q7 2408193119151915 q8 3556365336563653 q9 9041898389778977 q10 3857388339243883 q11 583 479 472 472 q12 818 645 644 644 q13 3895322631783178 q14 279 258 263 258 q15 575 527 533 527 q16 556 527 530 527 q17 2027182918211821 q18 8756826683478266 q19 1748172416881688 q20 2257198620791986 q21 5712543652855285 q22 574 502 493 493 Total cold run time: 63851 ms Total hot run time: 60225 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [log](cancel) User should see detail cancel message in terminal [doris]
zhiqiang- commented on PR #29589: URL: https://github.com/apache/doris/pull/29589#issuecomment-1878287442 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442601219 ## be/src/exec/exec_node.cpp: ## @@ -276,6 +260,10 @@ Status ExecNode::create_tree_helper(RuntimeState* state, ObjectPool* pool, // Step 1 Create current ExecNode according to current thrift plan node. ExecNode* cur_exec_node = nullptr; RETURN_IF_ERROR(create_node(state, pool, cur_plan_node, descs, &cur_exec_node)); +if (cur_exec_node != nullptr) { +state->get_query_ctx()->register_query_statistics( +cur_exec_node->get_query_stats_shared_ptr()); Review Comment: get_query_statistics not use get_query_stats_shared_ptr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442601439 ## be/src/exec/exec_node.h: ## @@ -239,6 +232,8 @@ class ExecNode { size_t children_count() const { return _children.size(); } +std::shared_ptr get_query_stats_shared_ptr() { return _query_statistics; } Review Comment: rename this method to get_query_statistics -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](clang-tidy): add bugprone linters [doris]
doris-robot commented on PR #29521: URL: https://github.com/apache/doris/pull/29521#issuecomment-1878288405 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 986988e5273203b89b1c0260f40cc326c26f002d, data reload: false -- Round 1 -- q1 18428 524651455145 q2 2027159 144 144 q3 10612 107511781075 q4 10196 781 828 781 q5 7831295629092909 q6 215 135 132 132 q7 917 526 542 526 q8 9271206920492049 q9 6897648464196419 q10 8253309930393039 q11 442 229 217 217 q12 402 237 241 237 q13 18023 360236913602 q14 240 220 217 217 q15 569 521 516 516 q16 461 403 399 399 q17 976 525 501 501 q18 7523672866776677 q19 1625141213921392 q20 724 358 328 328 q21 2825241224032403 q22 396 328 342 328 Total cold run time: 108853 ms Total hot run time: 39036 ms - Round 2, with runtime_filter_mode=off - q1 5190510451375104 q2 348 229 248 229 q3 3325329532563256 q4 2126202820372028 q5 5821576458025764 q6 217 126 128 126 q7 2330191719451917 q8 3411344934963449 q9 8884886588098809 q10 3800387838333833 q11 597 489 490 489 q12 816 650 643 643 q13 7698320531713171 q14 298 264 266 264 q15 579 532 520 520 q16 557 487 506 487 q17 1950178117551755 q18 8750839183798379 q19 1644161815871587 q20 2186197819701970 q21 5693534653205320 q22 568 516 530 516 Total cold run time: 66788 ms Total hot run time: 59616 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442601923 ## be/src/pipeline/pipeline_x/operator.h: ## @@ -107,6 +107,8 @@ class PipelineXLocalStateBase { // override in Scan MultiCastSink virtual RuntimeFilterDependency* filterdependency() { return nullptr; } +std::shared_ptr query_stats_shared_ptr() { return _query_statistics; } Review Comment: rename -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](group commit) Fix wal mem back pressure fault injection case [doris]
github-actions[bot] commented on PR #29493: URL: https://github.com/apache/doris/pull/29493#issuecomment-1878290245 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442603473 ## be/src/runtime/fragment_mgr.h: ## @@ -152,6 +152,11 @@ class FragmentMgr : public RestMonitorIface { return _query_ctx_map.size(); } +std::unordered_map>& get_query_ctx_map() { Review Comment: never return this map, because you add lock to return a reference,the map maybe modified concurrently. I think you could add a method named colloect_query_statistics in FragmentMgr to avoid this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](Nereids): add more test for eliminate inner join by fk [doris]
jackwener merged PR #29390: URL: https://github.com/apache/doris/pull/29390 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [test](Nereids): add more test for eliminate inner join by fk (#29390)
This is an automated email from the ASF dual-hosted git repository. jakevin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 77fbbf63ed4 [test](Nereids): add more test for eliminate inner join by fk (#29390) 77fbbf63ed4 is described below commit 77fbbf63ed4b5f0ed5f2bda88d228222d6925851 Author: 谢健 AuthorDate: Fri Jan 5 16:21:24 2024 +0800 [test](Nereids): add more test for eliminate inner join by fk (#29390) --- .../data/nereids_rules_p0/pkfk/eliminate_inner.out | 309 + .../nereids_rules_p0/pkfk/eliminate_inner.groovy | 113 2 files changed, 422 insertions(+) diff --git a/regression-test/data/nereids_rules_p0/pkfk/eliminate_inner.out b/regression-test/data/nereids_rules_p0/pkfk/eliminate_inner.out new file mode 100644 index 000..bc465978ccb --- /dev/null +++ b/regression-test/data/nereids_rules_p0/pkfk/eliminate_inner.out @@ -0,0 +1,309 @@ +-- This file is automatically generated. You should know what you did if you want to edit this +-- !name -- +simple_case + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() +PhysicalOlapScan[pkt] +PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +with_pk_col + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() +PhysicalOlapScan[pkt] +PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +with_pk_col + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() +PhysicalOlapScan[pkt] +hashJoin[INNER_JOIN] hashCondition=((fkt_not_null1.fk = fkt_not_null2.fk)) otherCondition=() +--PhysicalOlapScan[fkt_not_null] +--PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +with_pk_col + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() build RFs:RF1 fk->[pk] +filter((pkt.pk > 1)) +--PhysicalOlapScan[pkt] apply RFs: RF1 +hashJoin[INNER_JOIN] hashCondition=((fkt_not_null1.fk = fkt_not_null2.fk)) otherCondition=() build RFs:RF0 fk->[fk] +--filter((fkt_not_null1.fk > 1)) +PhysicalOlapScan[fkt_not_null] apply RFs: RF0 +--filter((fkt_not_null2.fk > 1)) +PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +with_pk_col + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() +PhysicalOlapScan[pkt] +hashAgg[LOCAL] +--PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +with_pk_col + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() +PhysicalOlapScan[pkt] +hashAgg[GLOBAL] +--hashAgg[LOCAL] +PhysicalUnion +--PhysicalOlapScan[fkt_not_null] +--PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +fk with window + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() +PhysicalOlapScan[pkt] +PhysicalWindow +--PhysicalQuickSort[LOCAL_SORT] +PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +fk with limit + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() build RFs:RF0 fk->[pk] +PhysicalOlapScan[pkt] apply RFs: RF0 +PhysicalLimit[GLOBAL] +--PhysicalLimit[LOCAL] +PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +pk with filter that same as fk + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() build RFs:RF0 fk->[pk] +filter((pkt.pk = 1)) +--PhysicalOlapScan[pkt] apply RFs: RF0 +filter((fkt_not_null.fk = 1)) +--PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +pk with filter that included same as fk + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() build RFs:RF0 fk->[pk] +filter((pkt.pk = 1)) +--PhysicalOlapScan[pkt] apply RFs: RF0 +filter((cast(f as DOUBLE) = 1) and (fkt_not_null.fk = 1)) +--PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +pk with filter that not same as fk + +-- !shape -- +PhysicalResultSink +--hashJoin[INNER_JOIN] hashCondition=((pkt.pk = fkt_not_null.fk)) otherCondition=() build RFs:RF0 fk->[pk] +filter((cast(p as DOUBLE) = 1) and (pkt.pk = 1)) +--PhysicalOlapScan[pkt] apply RFs: RF0 +filter((cast(f as DOUBLE) = 1) and (fkt_not_null.fk = 1)) +--PhysicalOlapScan[fkt_not_null] + +-- !res -- + +-- !name -- +simple_case + +-- !shape -- +PhysicalResultSink +--filter(( not fk IS NULL)) +PhysicalOlapScan[fkt] + +-- !res -- +1 John +1
Error while running github feature from .asf.yaml in doris!
An error occurred while running github feature in .asf.yaml!: You can only have a maximum of 10 external triage collaborators, please contact vp-in...@apache.org to request an exception. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](group_commit) refector wal manager code [doris]
doris-robot commented on PR #29560: URL: https://github.com/apache/doris/pull/29560#issuecomment-1878292394 TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools ``` TPC-DS sf100 test result on commit da2076f22c04e4a515a45ba87c11592670f9ead7, data reload: false run tpcds-sf100 query with default conf and session variables query1 931 340 333 333 query2 6407198018761876 query3 6655205 197 197 query4 26896 22455 22364 22364 query5 4678511 505 505 query6 283 185 184 184 query7 4584295 260 260 query8 239 204 201 201 query9 8258271426982698 query10 440 249 239 239 query11 16249 15619 15492 15492 query12 133 76 84 76 query13 1629322 332 322 query14 11390 711170197019 query15 212 189 192 189 query16 6357276 262 262 query17 1850501 486 486 query18 1905274 256 256 query19 183 138 128 128 query20 79 73 76 73 query21 184 93 95 93 query22 5067489747534753 query23 32000 31294 31095 31095 query24 12092 286427852785 query25 589 344 345 344 query26 1762142 143 142 query27 2896275 278 275 query28 7005194419401940 query29 2102383 382 382 query30 295 139 150 139 query31 975 778 764 764 query32 87 57 54 54 query33 732 271 278 271 query34 889 463 440 440 query35 857 773 740 740 query36 1413127012911270 query37 185 71 74 71 query38 3422334133033303 query39 1331127912801279 query40 299 86 89 86 query41 38 37 35 35 query42 90 82 80 80 query43 541 473 498 473 query44 1128719 725 719 query45 198 195 182 182 query46 1062654 660 654 query47 1699159316431593 query48 346 249 263 249 query49 1199334 317 317 query50 801 330 359 330 query51 5485529552925292 query52 96 82 91 82 query53 207 147 144 144 query54 1371561 586 561 query55 101 95 82 82 query56 195 194 188 188 query57 1018948 962 948 query58 236 203 201 201 query59 2910263826752638 query60 224 208 202 202 query61 91 90 91 90 query62 651 493 459 459 query63 160 146 146 146 query64 5929174017241724 query65 3322326132623261 query66 1360337 337 337 query67 16075 15433 15218 15218 query68 10817 539 524 524 query69 518 253 254 253 query70 1788149515891495 query71 501 206 215 206 query72 5649352935513529 query73 2157318 316 316 query74 6913644864716448 query75 4834233523052305 query76 6309115411341134 query77 667 257 288 257 query78 9129864786298629 query79 1045509 502 502 query80 553 365 360 360 query81 450 208 212 208 query82 203 95 92 92 query83 163 137 143 137 query84 247 56 55 55 query85 955 293 274 274 query86 382 389 394 389 query87 3546339433843384 query88 3253227222822272 query89 336 258 253 253 query90 1852200 196 196 query91 124 90 95 90 query92 57 52 53 52 query93 2067521 493 493 query94 814 182 185 182 query95 452 419 407 407 query96 632 321 328 321 query97 4251417241974172 query98 206 204 185 185 query99 1135861 839 839 Total cold run time: 292078 ms Total hot run time: 179550 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to
Re: [PR] [Fix](show data) Inverted index size not added to data size after index build [doris]
eldenmoon merged PR #29568: URL: https://github.com/apache/doris/pull/29568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (77fbbf63ed4 -> fa4721125e8)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 77fbbf63ed4 [test](Nereids): add more test for eliminate inner join by fk (#29390) add fa4721125e8 [Fix](show data) Inverted index size not added to data size after index build (#29568) No new revisions were added by this update. Summary of changes: be/src/olap/task/index_builder.cpp | 8 ++ .../suites/inverted_index_p0/test_show_data.groovy | 155 + 2 files changed, 163 insertions(+) create mode 100644 regression-test/suites/inverted_index_p0/test_show_data.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Error while running github feature from .asf.yaml in doris!
An error occurred while running github feature in .asf.yaml!: You can only have a maximum of 10 external triage collaborators, please contact vp-in...@apache.org to request an exception. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [log](cancel) User should see detail cancel message in terminal [doris]
github-actions[bot] commented on PR #29589: URL: https://github.com/apache/doris/pull/29589#issuecomment-1878293880 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](group_commit) refector wal manager code [doris]
doris-robot commented on PR #29560: URL: https://github.com/apache/doris/pull/29560#issuecomment-1878294531 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.18 seconds stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 27.5 seconds inserted 1000 Rows, about 363K ops/s storage size: 17183676533 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](group commit) Add an option of enable relay wal on all regression test [doris]
doris-robot commented on PR #28757: URL: https://github.com/apache/doris/pull/28757#issuecomment-1878294542 TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools ``` TPC-DS sf100 test result on commit 0e49204b0dcf983784710fe9b58314312ae68ed8, data reload: false run tpcds-sf100 query with default conf and session variables query1 926 340 332 332 query2 6417192620491926 query3 6651204 200 200 query4 27041 22336 22374 22336 query5 3879544 552 544 query6 265 178 173 173 query7 4580279 262 262 query8 221 201 200 200 query9 8217266726342634 query10 397 238 250 238 query11 16186 15486 15557 15486 query12 123 75 76 75 query13 1621330 326 326 query14 11471 707771697077 query15 216 188 187 187 query16 6111268 259 259 query17 1829495 471 471 query18 1805267 263 263 query19 173 136 138 136 query20 79 74 77 74 query21 186 97 94 94 query22 5009464047974640 query23 32117 31137 31250 31137 query24 12189 284828122812 query25 586 357 327 327 query26 1814144 145 144 query27 2921286 274 274 query28 6936193419481934 query29 2183377 367 367 query30 286 142 144 142 query31 998 759 777 759 query32 84 56 56 56 query33 719 252 243 243 query34 938 444 422 422 query35 877 755 782 755 query36 1324118112141181 query37 181 74 77 74 query38 3420329033073290 query39 1327129312841284 query40 305 86 87 86 query41 37 35 36 35 query42 96 82 87 82 query43 519 506 504 504 query44 1063703 733 703 query45 198 184 183 183 query46 1083636 632 632 query47 1731158516331585 query48 325 256 265 256 query49 1203300 322 300 query50 730 320 317 317 query51 5382526153055261 query52 101 82 85 82 query53 208 150 143 143 query54 1411551 571 551 query55 100 92 77 77 query56 187 184 183 183 query57 1009950 961 950 query58 217 204 196 196 query59 2799262425822582 query60 249 220 198 198 query61 85 86 81 81 query62 635 437 453 437 query63 165 139 146 139 query64 5978165717041657 query65 3346323732523237 query66 1399332 338 332 query67 15697 15451 15231 15231 query68 11315 526 536 526 query69 499 262 246 246 query70 1788155015231523 query71 492 225 227 225 query72 5533350135273501 query73 2298335 315 315 query74 6950652164726472 query75 4805231522902290 query76 6267113811061106 query77 654 239 277 239 query78 9097877685898589 query79 1036517 505 505 query80 556 345 351 345 query81 464 208 208 208 query82 211 96 94 94 query83 154 139 135 135 query84 243 53 53 53 query85 968 274 259 259 query86 392 373 412 373 query87 3508341333713371 query88 2805226122402240 query89 346 264 256 256 query90 1923195 191 191 query91 115 96 87 87 query92 54 49 51 49 query93 1985431 437 431 query94 873 182 181 181 query95 453 418 396 396 query96 625 322 322 322 query97 4290419841764176 query98 208 198 190 190 query99 1108858 812 812 Total cold run time: 290576 ms Total hot run time: 178860 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442606160 ## fe/fe-common/src/main/java/org/apache/doris/common/Config.java: ## @@ -2327,6 +2327,9 @@ public class Config extends ConfigBase { @ConfField(mutable = true, masterOnly = true) public static int workload_max_action_num_in_policy = 5; // mainly used to limit set session var action +@ConfField(mutable = true) +public static int workload_audit_log_interval_ms = 2000; // mainly used to limit set session var action Review Comment: From this name, get nothing useful, rename a better name -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](group_commit) refector wal manager code [doris]
doris-robot commented on PR #29560: URL: https://github.com/apache/doris/pull/29560#issuecomment-1878294592 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit da2076f22c04e4a515a45ba87c11592670f9ead7, data reload: false run tpch-sf100 query with default conf and session variables q1 5524517851405140 q2 384 202 157 157 q3 1478114811781148 q4 1083822 746 746 q5 3131312231133113 q6 222 133 133 133 q7 966 567 553 553 q8 2149226222232223 q9 6713664466386638 q10 3197308531013085 q11 347 215 212 212 q12 373 232 232 232 q13 4416364636523646 q14 246 226 228 226 q15 584 542 539 539 q16 460 390 402 390 q17 1045522 494 494 q18 7136670968176709 q19 1635154614441444 q20 591 349 342 342 q21 2874244725082447 q22 407 332 328 328 Total cold run time: 44961 ms Total hot run time: 39945 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 5201513951265126 q2 339 231 235 231 q3 3352335433373337 q4 2151210020492049 q5 5979592259115911 q6 224 129 129 129 q7 2379192119391921 q8 3567364536703645 q9 9060899389568956 q10 3858390739003900 q11 592 485 465 465 q12 799 629 628 628 q13 3921322931863186 q14 308 267 275 267 q15 589 535 530 530 q16 543 513 516 513 q17 2045183118501831 q18 8778833483498334 q19 1726169317081693 q20 2256198419831983 q21 5692538553205320 q22 556 511 516 511 Total cold run time: 63915 ms Total hot run time: 60466 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Pick [bugfix](thirdparty) Upgrade aws s3 sdk to prevent mem leak (#25106) [doris]
doris-robot commented on PR #25418: URL: https://github.com/apache/doris/pull/25418#issuecomment-1878295324 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 49.21 seconds stream load tsv: 583 seconds loaded 74807831229 Bytes, about 122 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s insert into select: 29.5 seconds inserted 1000 Rows, about 338K ops/s storage size: 17164208895 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip functions isipv4string and isipv6string [doris]
doris-robot commented on PR #28556: URL: https://github.com/apache/doris/pull/28556#issuecomment-1878296352 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 37b047590a160ff9f14ecb7536718581f9d621c7, data reload: false -- Round 1 -- q1 18650 516851805168 q2 2030154 144 144 q3 10614 110210371037 q4 10206 773 804 773 q5 7834297929862979 q6 212 134 137 134 q7 919 508 521 508 q8 9284204120492041 q9 6874645264046404 q10 8255308229972997 q11 436 223 205 205 q12 388 234 229 229 q13 18007 363235973597 q14 247 211 219 211 q15 559 536 523 523 q16 455 379 419 379 q17 971 492 556 492 q18 7572669366166616 q19 1609132413721324 q20 700 333 328 328 q21 2819235724382357 q22 377 334 305 305 Total cold run time: 109018 ms Total hot run time: 38751 ms - Round 2, with runtime_filter_mode=off - q1 5162504150735041 q2 349 238 243 238 q3 3305330732973297 q4 2128202620002000 q5 5804578557735773 q6 222 127 130 127 q7 2336189219071892 q8 3409344734953447 q9 8814879387838783 q10 3774383038253825 q11 580 465 469 465 q12 812 655 629 629 q13 8252318431603160 q14 293 271 282 271 q15 575 510 525 510 q16 568 522 501 501 q17 1946180217671767 q18 8699838983528352 q19 1623155316141553 q20 2172195019361936 q21 5681524752035203 q22 606 490 491 490 Total cold run time: 67110 ms Total hot run time: 59260 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](group commit) Add an option of enable relay wal on all regression test [doris]
doris-robot commented on PR #28757: URL: https://github.com/apache/doris/pull/28757#issuecomment-1878296777 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.55 seconds stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 27.5 seconds inserted 1000 Rows, about 363K ops/s storage size: 17183633544 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442607425 ## fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java: ## @@ -168,6 +168,14 @@ protected void auditAfterExec(String origStmt, StatementBase parsedStmt, AuditLogHelper.logAuditLog(ctx, origStmt, parsedStmt, statistics, printFuzzyVariables); } +protected void asyncAuditAfterExec(String origStmt, StatementBase parsedStmt, Review Comment: not add this method. There should be only one auditlog log exec method. Not use sync or async to distinguish them. It is very confused. For example, when to use sync when to use async??? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jni) add datetimev2 precision [doris]
AshinGau commented on PR #29528: URL: https://github.com/apache/doris/pull/29528#issuecomment-1878297237 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442608223 ## fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java: ## @@ -168,6 +168,14 @@ protected void auditAfterExec(String origStmt, StatementBase parsedStmt, AuditLogHelper.logAuditLog(ctx, origStmt, parsedStmt, statistics, printFuzzyVariables); } +protected void asyncAuditAfterExec(String origStmt, StatementBase parsedStmt, Review Comment: We should always use async!!! All audit log should wait 3s after the query finished and then flush to audit log file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statistics)Do not collect min max for agg table value columns while doing sample analyze [doris]
github-actions[bot] commented on PR #29483: URL: https://github.com/apache/doris/pull/29483#issuecomment-1878298178 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statistics)Do not collect min max for agg table value columns while doing sample analyze [doris]
github-actions[bot] commented on PR #29483: URL: https://github.com/apache/doris/pull/29483#issuecomment-1878298220 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](Nereids): make add constraint commad not forward [doris]
XieJiann opened a new pull request, #29598: URL: https://github.com/apache/doris/pull/29598 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
yiguolei commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442609448 ## gensrc/thrift/FrontendService.thrift: ## @@ -406,6 +406,11 @@ struct TQueryStatistics { 5: optional i64 max_peak_memory_bytes } +struct TReportWorkloadRuntimeStatusParams { +1: optional i64 backend_id +2: map query_stats_map Review Comment: query_statistics_map not use query_stats_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
wangbo commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442610238 ## be/src/common/daemon.cpp: ## @@ -352,6 +354,65 @@ void Daemon::block_spill_gc_thread() { } } +void Daemon::report_be_workload_runtime_stats_thread() { Review Comment: 这里改下没问题,后续汇报Workload group用独立的线程比较好 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](complextype)fix compare_at base function support nested types [doris]
amorynan commented on code in PR #29297: URL: https://github.com/apache/doris/pull/29297#discussion_r1442610158 ## be/src/vec/columns/column_struct.cpp: ## @@ -185,6 +185,27 @@ const char* ColumnStruct::deserialize_and_insert_from_arena(const char* pos) { return pos; } +int ColumnStruct::compare_at(size_t n, size_t m, const IColumn& rhs_, + int nan_direction_hint) const { +const ColumnStruct& rhs = assert_cast(rhs_); + +const size_t lhs_tuple_size = columns.size(); +const size_t rhs_tuple_size = rhs.tuple_size(); + +if (rhs_tuple_size < lhs_tuple_size) { Review Comment: I think it's correct, because struct is combination of multi columns, only tuple size is same, we can compare at elem column. other is self column and offset column. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](clang-tidy): add bugprone linters [doris]
doris-robot commented on PR #29521: URL: https://github.com/apache/doris/pull/29521#issuecomment-1878300509 TeamCity be ut coverage result: Function Coverage: 36.63% (8617/23522) Line Coverage: 28.67% (70030/244288) Region Coverage: 27.64% (36248/131121) Branch Coverage: 24.34% (18515/76078) Coverage Report: http://coverage.selectdb-in.cc/coverage/986988e5273203b89b1c0260f40cc326c26f002d_986988e5273203b89b1c0260f40cc326c26f002d/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
wangbo commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442610742 ## gensrc/thrift/FrontendService.thrift: ## @@ -470,6 +475,8 @@ struct TReportExecStatusParams { 23: optional list detailed_report 24: optional TQueryStatistics query_statistics + + 25: TReportWorkloadRuntimeStatusParams report_workload_runtime_stats Review Comment: 这个还是叫workload比较好,就是querystatistic和workload用同一个thrift接口,后续在这个类里加一个workload的status就行了 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](clang-tidy): add bugprone linters [doris]
doris-robot commented on PR #29521: URL: https://github.com/apache/doris/pull/29521#issuecomment-1878301390 TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools ``` TPC-DS sf100 test result on commit 986988e5273203b89b1c0260f40cc326c26f002d, data reload: false run tpcds-sf100 query with default conf and session variables query1 932 349 337 337 query2 6415195020061950 query3 6647210 209 209 query4 27092 22744 22423 22423 query5 3892567 500 500 query6 270 184 179 179 query7 4603274 266 266 query8 232 202 207 202 query9 8230263525702570 query10 395 265 257 257 query11 16351 15451 15580 15451 query12 134 77 77 77 query13 1670315 314 314 query14 11452 718571867185 query15 217 192 195 192 query16 6407278 266 266 query17 1856497 482 482 query18 1949266 258 258 query19 185 139 134 134 query20 80 74 75 74 query21 184 101 95 95 query22 5058480247164716 query23 32547 31154 31349 31154 query24 12156 282628382826 query25 595 346 333 333 query26 1731144 143 143 query27 2912277 280 277 query28 7057187918711871 query29 2067389 382 382 query30 289 142 149 142 query31 995 773 759 759 query32 89 57 56 56 query33 743 244 270 244 query34 922 460 450 450 query35 880 767 753 753 query36 1229125211881188 query37 185 63 70 63 query38 3355331632573257 query39 1349129712771277 query40 304 85 86 85 query41 38 35 33 33 query42 85 84 84 84 query43 530 502 508 502 query44 1091701 712 701 query45 202 189 184 184 query46 1077628 628 628 query47 1645154515341534 query48 360 252 262 252 query49 1229314 332 314 query50 774 358 328 328 query51 5387527352275227 query52 90 88 86 86 query53 215 151 150 150 query54 1345571 591 571 query55 98 85 77 77 query56 189 184 197 184 query57 1054958 965 958 query58 221 199 201 199 query59 2874268126702670 query60 233 217 225 217 query61 84 83 83 83 query62 657 466 483 466 query63 175 148 149 148 query64 5918173816781678 query65 3337326632723266 query66 1394332 352 332 query67 15492 15109 15041 15041 query68 13179 526 507 507 query69 542 245 249 245 query70 2080142915251429 query71 494 202 197 197 query72 5609350435183504 query73 3371318 316 316 query74 7131648764066406 query75 5368230922382238 query76 6309117811651165 query77 658 247 256 247 query78 9181862885148514 query79 1885502 520 502 query80 573 355 355 355 query81 472 207 206 206 query82 205 91 92 91 query83 184 138 137 137 query84 246 53 53 53 query85 968 273 264 264 query86 392 382 389 382 query87 3552334633713346 query88 3061227922532253 query89 352 262 266 262 query90 1986196 197 196 query91 119 88 106 88 query92 61 53 51 51 query93 1415444 425 425 query94 976 176 178 176 query95 465 422 399 399 query96 639 329 324 324 query97 4283419041944190 query98 206 199 193 193 query99 1094820 800 800 Total cold run time: 296506 ms Total hot run time: 178771 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to
Re: [PR] [Performance](exec) opt the exchange performance [doris]
HappenLee commented on PR #29579: URL: https://github.com/apache/doris/pull/29579#issuecomment-1878302103 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] (enhance)(S3) Change s3 metric from bvar adder to latency recorder [doris]
ByteYue commented on PR #28861: URL: https://github.com/apache/doris/pull/28861#issuecomment-1878301984 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](group commit) Add an option of enable relay wal on all regression test [doris]
doris-robot commented on PR #28757: URL: https://github.com/apache/doris/pull/28757#issuecomment-1878302023 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 0e49204b0dcf983784710fe9b58314312ae68ed8, data reload: false run tpch-sf100 query with default conf and session variables q1 5516517951895179 q2 388 158 159 158 q3 1447122611681168 q4 1083875 807 807 q5 3126313331173117 q6 217 134 132 132 q7 974 578 536 536 q8 2142228422292229 q9 6697667266746672 q10 3151315231683152 q11 349 216 211 211 q12 380 237 240 237 q13 4431367536243624 q14 242 222 222 222 q15 572 537 530 530 q16 491 408 391 391 q17 1048503 520 503 q18 7110670569236705 q19 1651153015861530 q20 543 361 325 325 q21 2920255724552455 q22 422 342 344 342 Total cold run time: 44900 ms Total hot run time: 40225 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 5134514050565056 q2 341 234 230 230 q3 3362333733233323 q4 2168203920382038 q5 5971593259325932 q6 228 127 127 127 q7 2405189319471893 q8 3572364736843647 q9 9160895989648959 q10 3885389438873887 q11 577 503 469 469 q12 811 633 626 626 q13 3868317932033179 q14 295 276 282 276 q15 584 525 526 525 q16 571 514 508 508 q17 2028181117951795 q18 8794832183028302 q19 1733166116691661 q20 2266197819861978 q21 5750542053995399 q22 580 533 518 518 Total cold run time: 64083 ms Total hot run time: 60328 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
wangbo commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442611854 ## be/src/exec/exec_node.cpp: ## @@ -276,6 +260,10 @@ Status ExecNode::create_tree_helper(RuntimeState* state, ObjectPool* pool, // Step 1 Create current ExecNode according to current thrift plan node. ExecNode* cur_exec_node = nullptr; RETURN_IF_ERROR(create_node(state, pool, cur_plan_node, descs, &cur_exec_node)); +if (cur_exec_node != nullptr) { +state->get_query_ctx()->register_query_statistics( +cur_exec_node->get_query_stats_shared_ptr()); Review Comment: 你是说命名还是不能用sharedptr,querystatistic的生命周期必须是要比query长的啊,所以只能用sharedptr存 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](clang-tidy): add bugprone linters [doris]
doris-robot commented on PR #29521: URL: https://github.com/apache/doris/pull/29521#issuecomment-1878304151 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.09 seconds stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 27.5 seconds inserted 1000 Rows, about 363K ops/s storage size: 17183853002 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](profile)Support report runtime workload statistics [doris]
wangbo commented on code in PR #29591: URL: https://github.com/apache/doris/pull/29591#discussion_r1442613406 ## fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java: ## @@ -168,6 +168,14 @@ protected void auditAfterExec(String origStmt, StatementBase parsedStmt, AuditLogHelper.logAuditLog(ctx, origStmt, parsedStmt, statistics, printFuzzyVariables); } +protected void asyncAuditAfterExec(String origStmt, StatementBase parsedStmt, Review Comment: 只有需要等待querystatistic的查询才需要异步,普通的其他查询走同步逻辑就行 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](group commit) Fix wal mem back pressure fault injection case [doris]
doris-robot commented on PR #29493: URL: https://github.com/apache/doris/pull/29493#issuecomment-1878305556 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 249fb4449586d9d54c5962ce75ad4284c75da911, data reload: false -- Round 1 -- q1 17923 529850885088 q2 2034155 147 147 q3 10607 113310961096 q4 10255 778 812 778 q5 7802293027892789 q6 220 133 132 132 q7 954 524 539 524 q8 9240204220282028 q9 6872643263876387 q10 8267301330223013 q11 432 234 201 201 q12 384 228 240 228 q13 18019 362736153615 q14 246 208 214 208 q15 567 530 514 514 q16 449 387 399 387 q17 953 458 484 458 q18 7389682266926692 q19 1584129913441299 q20 689 329 353 329 q21 2795233424412334 q22 379 310 333 310 Total cold run time: 108060 ms Total hot run time: 38557 ms - Round 2, with runtime_filter_mode=off - q1 5107514051365136 q2 340 228 236 228 q3 3278329132643264 q4 2116201820462018 q5 5801577559235775 q6 221 123 126 123 q7 2377188419521884 q8 3369344234403440 q9 8839875687578756 q10 3781384238483842 q11 573 472 500 472 q12 796 636 639 636 q13 8948320332013201 q14 292 269 280 269 q15 583 527 522 522 q16 543 490 512 490 q17 1963178417591759 q18 8709833082518251 q19 1632158615861586 q20 2214194819421942 q21 5500514851985148 q22 531 514 452 452 Total cold run time: 67513 ms Total hot run time: 59194 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](write)add metrics for flink load data [doris-flink-connector]
JNSimba merged PR #287: URL: https://github.com/apache/doris-flink-connector/pull/287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-flink-connector) branch master updated: [feature](write)add metrics for flink load data (#287)
This is an automated email from the ASF dual-hosted git repository. diwu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-flink-connector.git The following commit(s) were added to refs/heads/master by this push: new dc1f0ec [feature](write)add metrics for flink load data (#287) dc1f0ec is described below commit dc1f0ec3a8462c59e121b6e00d0ef165ed5f1b46 Author: Petrichor <31833513+vinle...@users.noreply.github.com> AuthorDate: Fri Jan 5 16:33:42 2024 +0800 [feature](write)add metrics for flink load data (#287) --- .../doris/flink/rest/models/RespContent.java | 70 - .../doris/flink/sink/writer/DorisStreamLoad.java | 6 +- .../doris/flink/sink/writer/DorisWriteMetrics.java | 348 + .../doris/flink/sink/writer/DorisWriter.java | 33 +- .../doris/flink/sink/writer/TestDorisWriter.java | 37 +++ 5 files changed, 475 insertions(+), 19 deletions(-) diff --git a/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/models/RespContent.java b/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/models/RespContent.java index 71edbfd..94a1dc4 100644 --- a/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/models/RespContent.java +++ b/flink-doris-connector/src/main/java/org/apache/doris/flink/rest/models/RespContent.java @@ -26,7 +26,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; public class RespContent { @JsonProperty(value = "TxnId") -private long txnId; +private Long txnId; @JsonProperty(value = "Label") private String label; @@ -44,42 +44,42 @@ public class RespContent { private String message; @JsonProperty(value = "NumberTotalRows") -private long numberTotalRows; +private Long numberTotalRows; @JsonProperty(value = "NumberLoadedRows") -private long numberLoadedRows; +private Long numberLoadedRows; @JsonProperty(value = "NumberFilteredRows") -private int numberFilteredRows; +private Integer numberFilteredRows; @JsonProperty(value = "NumberUnselectedRows") -private int numberUnselectedRows; +private Integer numberUnselectedRows; @JsonProperty(value = "LoadBytes") -private long loadBytes; +private Long loadBytes; @JsonProperty(value = "LoadTimeMs") -private int loadTimeMs; +private Integer loadTimeMs; @JsonProperty(value = "BeginTxnTimeMs") -private int beginTxnTimeMs; +private Integer beginTxnTimeMs; @JsonProperty(value = "StreamLoadPutTimeMs") -private int streamLoadPutTimeMs; +private Integer streamLoadPutTimeMs; @JsonProperty(value = "ReadDataTimeMs") -private int readDataTimeMs; +private Integer readDataTimeMs; @JsonProperty(value = "WriteDataTimeMs") -private int writeDataTimeMs; +private Integer writeDataTimeMs; @JsonProperty(value = "CommitAndPublishTimeMs") -private int commitAndPublishTimeMs; +private Integer commitAndPublishTimeMs; @JsonProperty(value = "ErrorURL") private String errorURL; -public long getTxnId() { +public Long getTxnId() { return txnId; } @@ -99,6 +99,50 @@ public class RespContent { return existingJobStatus; } +public Long getNumberTotalRows() { +return numberTotalRows; +} + +public Long getNumberLoadedRows() { +return numberLoadedRows; +} + +public Integer getNumberFilteredRows() { +return numberFilteredRows; +} + +public Integer getNumberUnselectedRows() { +return numberUnselectedRows; +} + +public Long getLoadBytes() { +return loadBytes; +} + +public Integer getLoadTimeMs() { +return loadTimeMs; +} + +public Integer getBeginTxnTimeMs() { +return beginTxnTimeMs; +} + +public Integer getStreamLoadPutTimeMs() { +return streamLoadPutTimeMs; +} + +public Integer getReadDataTimeMs() { +return readDataTimeMs; +} + +public Integer getWriteDataTimeMs() { +return writeDataTimeMs; +} + +public Integer getCommitAndPublishTimeMs() { +return commitAndPublishTimeMs; +} + @Override public String toString() { ObjectMapper mapper = new ObjectMapper(); diff --git a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/DorisStreamLoad.java b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/DorisStreamLoad.java index b095eb9..604eb5c 100644 --- a/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/DorisStreamLoad.java +++ b/flink-doris-connector/src/main/java/org/apache/doris/flink/sink/writer/DorisStreamLoad.java @@ -155,8 +155,8 @@ public class DorisStreamLoad implements Serializable { /** * try to discard pending transactions with labels beginning with labelSuffix. * - * @param labelSuffix - * @param chkID + * @param label
Re: [PR] [Performance](exec) opt the exchange performance [doris]
github-actions[bot] commented on PR #29579: URL: https://github.com/apache/doris/pull/29579#issuecomment-1878309220 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtbale) enable move memtable in routine load [doris]
HHoflittlefish777 commented on PR #28974: URL: https://github.com/apache/doris/pull/28974#issuecomment-1878309239 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](script) Add check_jvm_xmx for `start_fe.sh` [doris]
github-actions[bot] commented on PR #28989: URL: https://github.com/apache/doris/pull/28989#issuecomment-1878310240 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip functions isipv4string and isipv6string [doris]
doris-robot commented on PR #28556: URL: https://github.com/apache/doris/pull/28556#issuecomment-1878310286 TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools ``` TPC-DS sf100 test result on commit 37b047590a160ff9f14ecb7536718581f9d621c7, data reload: false run tpcds-sf100 query with default conf and session variables query1 935 352 341 341 query2 6415197118221822 query3 6648207 194 194 query4 26895 22516 22236 22236 query5 3906491 498 491 query6 266 183 177 177 query7 4579260 261 260 query8 222 215 197 197 query9 8238262525172517 query10 440 251 238 238 query11 16261 15440 15539 15440 query12 132 79 72 72 query13 1624326 314 314 query14 11592 695470246954 query15 216 194 190 190 query16 6271266 278 266 query17 1856492 476 476 query18 1897270 267 267 query19 176 130 133 130 query20 79 72 75 72 query21 186 97 92 92 query22 4938478844274427 query23 32005 31257 31254 31254 query24 12140 283327982798 query25 594 339 324 324 query26 1826141 138 138 query27 2908268 266 266 query28 7048186018501850 query29 2177376 367 367 query30 289 139 144 139 query31 1004778 767 767 query32 89 56 50 50 query33 736 245 259 245 query34 900 441 439 439 query35 878 795 754 754 query36 1275120311691169 query37 188 62 69 62 query38 3402328433113284 query39 1317127612761276 query40 308 87 85 85 query41 37 37 34 34 query42 94 85 81 81 query43 510 477 488 477 query44 1042686 691 686 query45 189 185 185 185 query46 1050624 603 603 query47 1584155215601552 query48 337 253 249 249 query49 1217308 317 308 query50 786 347 316 316 query51 5330526152405240 query52 86 84 80 80 query53 206 152 144 144 query54 1406556 540 540 query55 102 79 72 72 query56 197 194 188 188 query57 1011906 931 906 query58 223 202 196 196 query59 2796261125752575 query60 231 227 220 220 query61 88 84 89 84 query62 640 455 437 437 query63 166 143 142 142 query64 5834176217201720 query65 3318323032213221 query66 1384347 335 335 query67 15855 15401 15140 15140 query68 11967 504 515 504 query69 501 248 241 241 query70 1689148215491482 query71 492 228 215 215 query72 5601355335253525 query73 2673302 302 302 query74 7129646864846468 query75 4894229922662266 query76 6213109010941090 query77 659 250 258 250 query78 9127853484928492 query79 2373496 483 483 query80 689 341 333 333 query81 458 201 210 201 query82 206 90 92 90 query83 160 137 134 134 query84 247 52 50 50 query85 973 281 267 267 query86 433 377 390 377 query87 3584346033473347 query88 3448219121822182 query89 342 273 247 247 query90 1958192 183 183 query91 115 88 91 88 query92 58 55 49 49 query93 2498484 485 484 query94 799 176 172 172 query95 449 400 407 400 query96 618 309 312 309 query97 4256415341414141 query98 208 198 203 198 query99 1133839 826 826 Total cold run time: 294525 ms Total hot run time: 177567 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to
Re: [PR] [chore](clang-tidy): add bugprone linters [doris]
doris-robot commented on PR #29521: URL: https://github.com/apache/doris/pull/29521#issuecomment-1878310473 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 986988e5273203b89b1c0260f40cc326c26f002d, data reload: false run tpch-sf100 query with default conf and session variables q1 5475513651615136 q2 389 166 144 144 q3 1472115711561156 q4 1074779 788 779 q5 3152311330973097 q6 220 131 135 131 q7 972 549 536 536 q8 2155227622762276 q9 6701672066546654 q10 3197313431783134 q11 354 233 221 221 q12 370 235 225 225 q13 4432362236503622 q14 251 220 212 212 q15 567 550 529 529 q16 462 386 398 386 q17 1045495 511 495 q18 7106676376426763 q19 1644156414391439 q20 592 353 342 342 q21 2857252224612461 q22 405 324 345 324 Total cold run time: 44892 ms Total hot run time: 40062 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 5099508950385038 q2 336 237 235 235 q3 3349332733183318 q4 2145206920672067 q5 5943591959285919 q6 225 124 127 124 q7 2364189319561893 q8 3561366836663666 q9 9048898889858985 q10 3849390639013901 q11 569 483 481 481 q12 799 657 618 618 q13 3885317031843170 q14 302 289 267 267 q15 597 534 531 531 q16 562 501 513 501 q17 2039181918011801 q18 8726832983658329 q19 1742168816971688 q20 2275196919701969 q21 5763533053965330 q22 557 505 485 485 Total cold run time: 63735 ms Total hot run time: 60316 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](planner) Fix table sample not take effect [doris]
xinyiZzz commented on PR #29594: URL: https://github.com/apache/doris/pull/29594#issuecomment-1878310571 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] "where exists" with "GROUP BY" is ignored [doris]
LemonLiTree commented on issue #28081: URL: https://github.com/apache/doris/issues/28081#issuecomment-1878311973 https://github.com/apache/doris/pull/28713 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Change the connection pool parameters of Jdbc Catalog to be configurable [doris]
github-actions[bot] commented on PR #29195: URL: https://github.com/apache/doris/pull/29195#issuecomment-1878312058 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] "where exists" with "GROUP BY" is ignored [doris]
LemonLiTree closed issue #28081: [Bug] "where exists" with "GROUP BY" is ignored URL: https://github.com/apache/doris/issues/28081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip functions isipv4string and isipv6string [doris]
doris-robot commented on PR #28556: URL: https://github.com/apache/doris/pull/28556#issuecomment-1878312539 (From new machine)TeamCity pipeline, clickbench performance test result: the sum of best hot time: 47.24 seconds stream load tsv: 563 seconds loaded 74807831229 Bytes, about 126 MB/s stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s insert into select: 27.2 seconds inserted 1000 Rows, about 367K ops/s storage size: 17189416993 Bytes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](group_commit) refector wal manager code [doris]
doris-robot commented on PR #29560: URL: https://github.com/apache/doris/pull/29560#issuecomment-1878313495 TeamCity be ut coverage result: Function Coverage: 36.62% (8614/23521) Line Coverage: 28.66% (70027/244308) Region Coverage: 27.64% (36248/131133) Branch Coverage: 24.34% (18522/76090) Coverage Report: http://coverage.selectdb-in.cc/coverage/da2076f22c04e4a515a45ba87c11592670f9ead7_da2076f22c04e4a515a45ba87c11592670f9ead7/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip function ipv6_string_to_num(_or_default, _or_null), inet6_aton [doris]
github-actions[bot] commented on PR #28361: URL: https://github.com/apache/doris/pull/28361#issuecomment-1878313355 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip functions isipv4string and isipv6string [doris]
doris-robot commented on PR #28556: URL: https://github.com/apache/doris/pull/28556#issuecomment-1878313810 TeamCity be ut coverage result: Function Coverage: 36.63% (8622/23540) Line Coverage: 28.66% (70065/244428) Region Coverage: 27.65% (36275/131203) Branch Coverage: 24.34% (18527/76132) Coverage Report: http://coverage.selectdb-in.cc/coverage/37b047590a160ff9f14ecb7536718581f9d621c7_37b047590a160ff9f14ecb7536718581f9d621c7/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtbale) enable move memtable in routine load [doris]
github-actions[bot] commented on PR #28974: URL: https://github.com/apache/doris/pull/28974#issuecomment-1878314860 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](planner) Fix table sample not take effect [doris]
xinyiZzz commented on PR #29594: URL: https://github.com/apache/doris/pull/29594#issuecomment-1878315803 > add some test case please table sample not take effect because `selectedIndexId=-1` in `computeSampleTabletIds`, the previous run FE UT had special processing, so FE UT did not discover the BUG in time. ``` if (FeConstants.runningUnitTest && selectedIndexId == -1) { selectedIndexId = olapTable.getBaseIndexId(); } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Change the connection pool parameters of Jdbc Catalog to be configurable [doris]
zy-kkk commented on PR #29195: URL: https://github.com/apache/doris/pull/29195#issuecomment-1878316126 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) support ip functions isipv4string and isipv6string [doris]
doris-robot commented on PR #28556: URL: https://github.com/apache/doris/pull/28556#issuecomment-1878317017 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 37b047590a160ff9f14ecb7536718581f9d621c7, data reload: false run tpch-sf100 query with default conf and session variables q1 5490512451285124 q2 383 173 165 165 q3 1460119811641164 q4 1086889 810 810 q5 3122316031803160 q6 216 134 133 133 q7 977 562 523 523 q8 2165220722112207 q9 6715667866736673 q10 3179319331493149 q11 346 210 223 210 q12 380 229 229 229 q13 4397367536113611 q14 250 219 219 219 q15 573 537 525 525 q16 449 416 398 398 q17 1053492 490 490 q18 7133668868096688 q19 1647152214351435 q20 560 325 316 316 q21 2921251224632463 q22 393 330 320 320 Total cold run time: 44895 ms Total hot run time: 40012 ms run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off q1 5105507651055076 q2 347 239 235 235 q3 3366333533253325 q4 2158205620432043 q5 5948591359135913 q6 222 128 129 128 q7 2380194019011901 q8 3531366436693664 q9 9042899689638963 q10 3869390239303902 q11 586 479 473 473 q12 811 635 623 623 q13 3884320131843184 q14 296 271 270 270 q15 590 540 524 524 q16 562 517 502 502 q17 2017182217941794 q18 8776833584328335 q19 1744170617141706 q20 2262200219591959 q21 5815538153665366 q22 549 497 465 465 Total cold run time: 63860 ms Total hot run time: 60351 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](group_commit) refector wal manager code [doris]
hust-hhb commented on PR #29560: URL: https://github.com/apache/doris/pull/29560#issuecomment-1878317583 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](group commit) Fix wal mem back pressure fault injection case [doris]
doris-robot commented on PR #29493: URL: https://github.com/apache/doris/pull/29493#issuecomment-1878317920 TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools ``` TPC-DS sf100 test result on commit 249fb4449586d9d54c5962ce75ad4284c75da911, data reload: false run tpcds-sf100 query with default conf and session variables query1 913 349 334 334 query2 6417194120191941 query3 6653209 201 201 query4 26502 22428 22312 22312 query5 3907562 556 556 query6 261 181 172 172 query7 4592259 256 256 query8 224 202 198 198 query9 8294257825562556 query10 397 263 230 230 query11 16067 15582 15422 15422 query12 134 77 73 73 query13 1637326 311 311 query14 11813 721571977197 query15 220 182 185 182 query16 6286267 261 261 query17 1878489 481 481 query18 1930264 266 264 query19 195 135 136 135 query20 78 75 75 75 query21 182 100 94 94 query22 5227456145744561 query23 32075 31268 31184 31184 query24 11926 280827722772 query25 596 344 330 330 query26 1759138 137 137 query27 2884264 274 264 query28 6897187018691869 query29 1929397 385 385 query30 284 147 145 145 query31 984 765 765 765 query32 86 54 55 54 query33 731 279 261 261 query34 861 454 445 445 query35 887 787 791 787 query36 1310120311771177 query37 182 67 68 67 query38 3433326832193219 query39 1332129012581258 query40 300 83 84 83 query41 37 35 35 35 query42 85 91 91 91 query43 514 485 489 485 query44 1035698 705 698 query45 194 186 182 182 query46 1065659 635 635 query47 1625151715411517 query48 332 262 248 248 query49 1199310 300 300 query50 732 371 373 371 query51 5445530752825282 query52 101 85 89 85 query53 209 149 151 149 query54 1358553 557 553 query55 97 88 74 74 query56 176 194 185 185 query57 990 937 923 923 query58 223 194 197 194 query59 2699260625912591 query60 229 190 210 190 query61 85 92 81 81 query62 633 464 467 464 query63 166 142 150 142 query64 5645172216421642 query65 323932673239 query66 1346330 325 325 query67 15622 15581 15396 15396 query68 10719 516 505 505 query69 516 256 245 245 query70 1635145914211421 query71 498 213 222 213 query72 5529352135003500 query73 2148312 309 309 query74 6932645364426442 query75 4934230222432243 query76 6362109011151090 query77 659 254 283 254 query78 9126889086168616 query79 1412500 505 500 query80 613 339 344 339 query81 451 208 205 205 query82 201 86 92 86 query83 158 134 131 131 query84 248 52 52 52 query85 948 275 267 267 query86 374 385 357 357 query87 3548335433203320 query88 3473226622662266 query89 344 256 255 255 query90 1937195 195 195 query91 119 89 91 89 query92 63 51 54 51 query93 3616494 420 420 query94 844 180 177 177 query95 447 420 413 413 query96 652 318 311 311 query97 4261415041744150 query98 215 199 184 184 query99 1106844 823 823 Total cold run time: 291526 ms Total hot run time: 178520 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to
Re: [PR] (opt)(match_phrase) Optimizing match_phrase with a new algorithm [doris]
doris-robot commented on PR #29444: URL: https://github.com/apache/doris/pull/29444#issuecomment-1878317704 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 9bc57afbfd2554a2d22fbb8790c144c6e57f71bd, data reload: false -- Round 1 -- q1 18930 581851815181 q2 2033156 149 149 q3 10596 112311051105 q4 10204 803 844 803 q5 7796302229642964 q6 216 132 132 132 q7 922 519 548 519 q8 9257207120702070 q9 6887643464376434 q10 8281305830733058 q11 419 216 213 213 q12 398 237 236 236 q13 18016 364636573646 q14 240 216 207 207 q15 561 536 516 516 q16 462 388 393 388 q17 993 472 476 472 q18 7478684066876687 q19 1601140614241406 q20 729 351 320 320 q21 2876239624512396 q22 386 330 334 330 Total cold run time: 109281 ms Total hot run time: 39232 ms - Round 2, with runtime_filter_mode=off - q1 5201516651885166 q2 342 231 240 231 q3 3323331532923292 q4 2144204220312031 q5 5849583057985798 q6 216 123 124 123 q7 2338191919301919 q8 3419351835303518 q9 8849879287478747 q10 3811386538363836 q11 589 472 479 472 q12 793 632 641 632 q13 8313324232393239 q14 288 278 280 278 q15 575 517 519 517 q16 554 532 532 532 q17 1968178618141786 q18 8720848883238323 q19 1644157716051577 q20 2190197019591959 q21 5724538354915383 q22 543 520 498 498 Total cold run time: 67393 ms Total hot run time: 59857 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jni) add datetimev2 precision [doris]
doris-robot commented on PR #29528: URL: https://github.com/apache/doris/pull/29528#issuecomment-1878318708 TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools ``` Tpch sf100 test result on commit 35368ab92d8c4f0e60e3e1c81179bc97e2b7a46b, data reload: false -- Round 1 -- q1 17842 532250645064 q2 2016159 152 152 q3 10622 106810691068 q4 10509 810 775 775 q5 7816295228702870 q6 213 134 136 134 q7 919 505 551 505 q8 9295203020252025 q9 6847643464076407 q10 8264309530493049 q11 422 218 207 207 q12 386 227 236 227 q13 17996 368436413641 q14 250 215 218 215 q15 557 524 520 520 q16 473 420 423 420 q17 971 476 500 476 q18 7631683866756675 q19 1584138313981383 q20 711 348 324 324 q21 2808238824232388 q22 381 337 343 337 Total cold run time: 108513 ms Total hot run time: 38862 ms - Round 2, with runtime_filter_mode=off - q1 5158509751095097 q2 343 240 249 240 q3 3302331732893289 q4 2133203820452038 q5 5847582457955795 q6 216 127 127 127 q7 2348193619091909 q8 3349344934843449 q9 8837875087958750 q10 3795384138503841 q11 581 478 494 478 q12 795 617 632 617 q13 5809321532313215 q14 290 273 261 261 q15 570 521 526 521 q16 548 510 515 510 q17 1968177817401740 q18 8708841683258325 q19 1628157715731573 q20 2186196119391939 q21 5600540952975297 q22 548 492 505 492 Total cold run time: 64559 ms Total hot run time: 59503 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](Nereids) support complex literal cast in fe [doris]
morrySnow opened a new pull request, #29599: URL: https://github.com/apache/doris/pull/29599 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] The content of the Doris document's open source contribution guide is incorrect [doris]
Suzzt opened a new issue, #29600: URL: https://github.com/apache/doris/issues/29600 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version none ### What's Wrong? Hi, there is a small issue with the document here. The screenshot here should be the 'fork' button, but the screenshot here is the 'star' button. ### What You Expected? IMO, I think the fork button should be boxed or unchecked, but it should never be boxed as it can mislead others. ### How to Reproduce? The document address is https://doris.apache.org/community/how-to-contribute/pull-request https://github.com/apache/doris/assets/27397567/7be4da6c-f267-4e5a-95f4-21c671b0dfb9";> ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statistics)Do not collect min max for agg table value columns while doing sample analyze [doris]
morrySnow commented on PR #29483: URL: https://github.com/apache/doris/pull/29483#issuecomment-1878320918 run pipelinex_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) support complex literal cast in fe [doris]
morrySnow commented on PR #29599: URL: https://github.com/apache/doris/pull/29599#issuecomment-1878322767 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Bug](cast) cast to varchar when comparison predicate input literal and cast to c… [doris]
BiteThet opened a new pull request, #29601: URL: https://github.com/apache/doris/pull/29601 …ommon type failed ## Proposed changes cast to varchar when comparison predicate input literal and cast to common type failed ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org