(doris) branch branch-2.0 updated: [fix](mysql) Fix Connection Attributes Parsing Logic for Compatibility (#32177) (#32302)
This is an automated email from the ASF dual-hosted git repository.
zykkk pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new ba1d8fe30ac [fix](mysql) Fix Connection Attributes Parsing Logic for
Compatibility (#32177) (#32302)
ba1d8fe30ac is described below
commit ba1d8fe30ac6f5191ac585b96c9dc11ee0317f03
Author: zy-kkk
AuthorDate: Sat Mar 16 15:02:26 2024 +0800
[fix](mysql) Fix Connection Attributes Parsing Logic for Compatibility
(#32177) (#32302)
---
fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlAuthPacket.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlAuthPacket.java
b/fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlAuthPacket.java
index b112850b393..cf196f82be1 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlAuthPacket.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlAuthPacket.java
@@ -104,8 +104,9 @@ public class MysqlAuthPacket extends MysqlPacket {
// attribute map, no use now.
if (buffer.remaining() > 0 && capability.isConnectAttrs()) {
connectAttributes = Maps.newHashMap();
-long numPair = MysqlProto.readVInt(buffer);
-for (long i = 0; i < numPair; ++i) {
+long attrsLength = MysqlProto.readVInt(buffer);
+long initialPosition = buffer.position();
+while (buffer.position() - initialPosition < attrsLength) {
String key = new
String(MysqlProto.readLenEncodedString(buffer));
String value = new
String(MysqlProto.readLenEncodedString(buffer));
connectAttributes.put(key, value);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pick][fix](mysql) Fix Connection Attributes Parsing Logic for Compatibily [doris]
zy-kkk merged PR #32302: URL: https://github.com/apache/doris/pull/32302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Enhancement](inverted index) make compiler happy [doris]
airborne12 opened a new pull request, #32332: URL: https://github.com/apache/doris/pull/32332 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) make compiler happy [doris]
doris-robot commented on PR #32332: URL: https://github.com/apache/doris/pull/32332#issuecomment-2001883660 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) make compiler happy [doris]
airborne12 commented on PR #32332: URL: https://github.com/apache/doris/pull/32332#issuecomment-2001883688 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2001884779 TeamCity be ut coverage result: Function Coverage: 34.94% (8573/24538) Line Coverage: 26.63% (69492/260923) Region Coverage: 25.92% (36091/139251) Branch Coverage: 22.85% (18427/80626) Coverage Report: http://coverage.selectdb-in.cc/coverage/2c657aeb4cec7d358dddc20340844a0940cc0038_2c657aeb4cec7d358dddc20340844a0940cc0038/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) make compiler happy [doris]
github-actions[bot] commented on PR #32332: URL: https://github.com/apache/doris/pull/32332#issuecomment-2001885252 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2001886735 TPC-H: Total hot run time: 38256 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 2c657aeb4cec7d358dddc20340844a0940cc0038, data reload: false -- Round 1 -- q1 17634 421540874087 q2 2021147 152 147 q3 10595 1053887 887 q4 7783745 681 681 q5 7477259226502592 q6 185 122 123 122 q7 1212826 794 794 q8 9334200120132001 q9 7272655465396539 q10 8539358137493581 q11 446 216 226 216 q12 730 309 297 297 q13 17888 285328432843 q14 291 248 242 242 q15 501 459 449 449 q16 522 395 388 388 q17 935 593 516 516 q18 7151636164306361 q19 1600137813921378 q20 547 285 272 272 q21 6118357935733573 q22 319 305 290 290 Total cold run time: 109100 ms Total hot run time: 38256 ms - Round 2, with runtime_filter_mode=off - q1 4078406940984069 q2 328 226 220 220 q3 2942284827792779 q4 1829158115271527 q5 5258531953135313 q6 196 115 120 115 q7 2195185218651852 q8 3133324332383238 q9 8642864186288628 q10 3773379937893789 q11 517 438 440 438 q12 698 546 555 546 q13 16910 285028272827 q14 269 240 257 240 q15 489 432 447 432 q16 456 408 404 404 q17 1728147914761476 q18 7429731770107010 q19 1590150915321509 q20 1887169317181693 q21 4667478547574757 q22 519 453 462 453 Total cold run time: 69533 ms Total hot run time: 53315 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Optimize](inverted index) optimize inverted index bitmap copy [doris]
github-actions[bot] commented on PR #32279: URL: https://github.com/apache/doris/pull/32279#issuecomment-2001893111 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Optimize](inverted index) optimize inverted index bitmap copy [doris]
github-actions[bot] commented on PR #32279: URL: https://github.com/apache/doris/pull/32279#issuecomment-2001893123 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) make compiler happy [doris]
doris-robot commented on PR #32332: URL: https://github.com/apache/doris/pull/32332#issuecomment-2001895373 TPC-H: Total hot run time: 37947 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f427c13c2cb0eec99743becf5ecfd23cc39468fe, data reload: false -- Round 1 -- q1 17696 420241174117 q2 2017146 140 140 q3 10602 1054891 891 q4 7428723 671 671 q5 7482260425762576 q6 184 122 121 121 q7 1190800 808 800 q8 9355196319951963 q9 7177640263856385 q10 8521346836283468 q11 421 225 217 217 q12 754 311 279 279 q13 18069 283428642834 q14 264 242 246 242 q15 503 450 441 441 q16 499 387 383 383 q17 938 537 543 537 q18 7119635763756357 q19 1893146213921392 q20 542 295 286 286 q21 6168354435413541 q22 348 313 306 306 Total cold run time: 109170 ms Total hot run time: 37947 ms - Round 2, with runtime_filter_mode=off - q1 4108405140254025 q2 324 228 227 227 q3 2944283928222822 q4 1825157215081508 q5 5186518852325188 q6 198 114 116 114 q7 2210181918621819 q8 3139330832573257 q9 8556850384508450 q10 3681367736653665 q11 529 443 432 432 q12 719 545 544 544 q13 16925 285328552853 q14 284 242 257 242 q15 490 440 440 440 q16 454 409 402 402 q17 1731148914671467 q18 7447718570617061 q19 1638155215241524 q20 1879169617421696 q21 4839471745814581 q22 525 444 458 444 Total cold run time: 69631 ms Total hot run time: 52761 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)add 'HmsCommiter' to support inserting data into hive table [doris]
wuwenchi commented on PR #32283: URL: https://github.com/apache/doris/pull/32283#issuecomment-2001895504 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](hive)add 'HmsCommiter' to support inserting data into hive table [doris]
doris-robot commented on PR #32283: URL: https://github.com/apache/doris/pull/32283#issuecomment-2001904754 TPC-H: Total hot run time: 38218 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0fda07553377757c7dffcd9c9ed1b0564c06c4ed, data reload: false -- Round 1 -- q1 17670 418841144114 q2 2025163 144 144 q3 10592 1096879 879 q4 8130767 677 677 q5 7495265826252625 q6 187 126 124 124 q7 1222795 792 792 q8 9349201720042004 q9 7142644064416440 q10 8492349036303490 q11 433 258 217 217 q12 821 306 293 293 q13 18017 289228552855 q14 275 243 247 243 q15 501 450 448 448 q16 502 391 386 386 q17 937 545 537 537 q18 7280646563846384 q19 2106143114201420 q20 552 299 286 286 q21 6161356035583558 q22 350 302 318 302 Total cold run time: 110239 ms Total hot run time: 38218 ms - Round 2, with runtime_filter_mode=off - q1 4124409841044098 q2 323 224 217 217 q3 2943288027822782 q4 1900156215731562 q5 5226522352325223 q6 191 117 117 117 q7 2236184618181818 q8 3121330532443244 q9 8434853684998499 q10 3724369036593659 q11 536 446 432 432 q12 719 560 553 553 q13 16940 286428432843 q14 270 244 251 244 q15 477 443 444 443 q16 465 409 403 403 q17 1720150314511451 q18 7472715972117159 q19 1634151514891489 q20 1930169517021695 q21 4878474947774749 q22 534 450 439 439 Total cold run time: 69797 ms Total hot run time: 53119 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Optimize](inverted index) optimize inverted index bitmap copy [doris]
xiaokang merged PR #32279: URL: https://github.com/apache/doris/pull/32279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) make compiler happy [doris]
github-actions[bot] commented on PR #32332: URL: https://github.com/apache/doris/pull/32332#issuecomment-2001909332 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](inverted index) make compiler happy [doris]
github-actions[bot] commented on PR #32332: URL: https://github.com/apache/doris/pull/32332#issuecomment-2001909343 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (277b3cf4379 -> 7be5a5c355d)
This is an automated email from the ASF dual-hosted git repository. kxiao pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 277b3cf4379 [pipelineX](exchange) Make exchange buffer size configurable (#32201) add 7be5a5c355d [Optimize](inverted index) optimize inverted index bitmap copy (#32279) No new revisions were added by this update. Summary of changes: be/src/olap/comparison_predicate.h | 8 +-- be/src/olap/in_list_predicate.h| 6 +- be/src/olap/match_predicate.cpp| 8 +-- .../segment_v2/inverted_index_file_reader.cpp | 11 ++- .../rowset/segment_v2/inverted_index_file_reader.h | 1 + .../rowset/segment_v2/inverted_index_reader.cpp| 83 +++--- .../olap/rowset/segment_v2/inverted_index_reader.h | 22 +++--- be/src/olap/rowset/segment_v2/segment_iterator.cpp | 17 ++--- be/src/runtime/exec_env_init.cpp | 4 +- 9 files changed, 85 insertions(+), 75 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)implement hive table sink plan [doris]
morningman commented on PR #31765: URL: https://github.com/apache/doris/pull/31765#issuecomment-2001916571 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
HHoflittlefish777 commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001922643 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] The execution of multi_search_all_positions(x, null) leads to a BE crash [doris]
wyxxxcat commented on issue #32272: URL: https://github.com/apache/doris/issues/32272#issuecomment-2001928576 I cannot reproduce it and use version master ![Uploading 截图 2024-03-15 11-20-51.png…]() -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] The execution of multi_search_all_positions(x, null) leads to a BE crash [doris]
wyxxxcat commented on issue #32272: URL: https://github.com/apache/doris/issues/32272#issuecomment-2001929137 I cannot reproduce it and use version master  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001930197 TPC-H: Total hot run time: 49723 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b7807a444fb261d65c47cfab11b01d3e2bf1bb9e, data reload: false -- Round 1 -- q1 17905 434843764348 q2 2037148 134 134 q3 10452 190619261906 q4 10329 122412971224 q5 8493392439743924 q6 221 122 121 121 q7 1991156015881560 q8 9293268626822682 q9 10698 10433 10419 10419 q10 8636350635103506 q11 411 226 232 226 q12 460 292 299 292 q13 18363 397339843973 q14 344 327 326 326 q15 501 460 456 456 q16 685 580 588 580 q17 1108951 964 951 q18 7224685469136854 q19 1663157315401540 q20 504 301 301 301 q21 4410411740114011 q22 489 390 389 389 Total cold run time: 116217 ms Total hot run time: 49723 ms - Round 2, with runtime_filter_mode=off - q1 4300428342854283 q2 317 224 217 217 q3 4110411340934093 q4 2763273127302730 q5 7158698770496987 q6 239 116 117 116 q7 3240279028132790 q8 4312440643994399 q9 17025 16961 16876 16876 q10 4206422242274222 q11 757 671 690 671 q12 1019850 829 829 q13 7411369437453694 q14 442 425 439 425 q15 483 459 454 454 q16 740 691 690 690 q17 3730375837763758 q18 8814874387248724 q19 1671168316341634 q20 2389212721562127 q21 8452852785718527 q22 1014938 981 938 Total cold run time: 84592 ms Total hot run time: 79184 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001931387 TeamCity be ut coverage result: Function Coverage: 37.82% (8042/21264) Line Coverage: 29.49% (65623/222540) Region Coverage: 28.96% (33772/116607) Branch Coverage: 24.80% (17332/69880) Coverage Report: http://coverage.selectdb-in.cc/coverage/b7807a444fb261d65c47cfab11b01d3e2bf1bb9e_b7807a444fb261d65c47cfab11b01d3e2bf1bb9e/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](insert)implement hive table sink plan [doris]
doris-robot commented on PR #31765: URL: https://github.com/apache/doris/pull/31765#issuecomment-2001931907 TPC-H: Total hot run time: 38347 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 050190a810e213856c8c67387db687583cc8f060, data reload: false -- Round 1 -- q1 17649 445341224122 q2 2019149 142 142 q3 10593 1073908 908 q4 7772821 727 727 q5 7463263526852635 q6 187 127 124 124 q7 1203812 809 809 q8 9319195820271958 q9 7163647464416441 q10 8528352936583529 q11 427 226 226 226 q12 671 306 292 292 q13 18009 284928622849 q14 274 262 245 245 q15 500 465 452 452 q16 514 394 387 387 q17 966 541 549 541 q18 7302654564186418 q19 2216141513551355 q20 543 294 288 288 q21 6091361935983598 q22 386 301 318 301 Total cold run time: 109795 ms Total hot run time: 38347 ms - Round 2, with runtime_filter_mode=off - q1 4113408940864086 q2 315 227 216 216 q3 2946284928282828 q4 1868154516051545 q5 5240523952405239 q6 192 116 118 116 q7 2217185318301830 q8 3162329632463246 q9 8538851785548517 q10 3705363636553636 q11 544 428 452 428 q12 707 542 539 539 q13 16918 284728882847 q14 271 251 251 251 q15 488 442 445 442 q16 448 408 415 408 q17 1733146914531453 q18 7789719470447044 q19 1632158415251525 q20 1892168917061689 q21 4689470947894709 q22 551 457 484 457 Total cold run time: 69958 ms Total hot run time: 53051 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001933353 TPC-DS: Total hot run time: 200064 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b7807a444fb261d65c47cfab11b01d3e2bf1bb9e, data reload: false query1 916 394 381 381 query2 6537218121182118 query3 6919196 199 196 query4 21061 17827 17763 17763 query5 19718 650364126412 query6 301 215 219 215 query7 4152289 295 289 query8 286 238 293 238 query9 3083266825952595 query10 414 279 291 279 query11 11183 10671 10617 10617 query12 120 72 70 70 query13 5588627 594 594 query14 17620 13109 13422 13109 query15 359 224 241 224 query16 6449260 257 257 query17 15821447861 861 query18 2298399 391 391 query19 201 140 141 140 query20 78 74 75 74 query21 181 88 94 88 query22 5376511650135013 query23 32470 31895 31779 31779 query24 6981643165036431 query25 535 416 413 413 query26 522 157 161 157 query27 1865287 294 287 query28 6045220521602160 query29 2905280628972806 query30 243 158 159 158 query31 914 702 762 702 query32 70 59 54 54 query33 386 236 260 236 query34 841 463 470 463 query35 1104907 904 904 query36 1642133315071333 query37 84 59 57 57 query38 3083294328882888 query39 1369133913091309 query40 200 89 93 89 query41 35 36 31 31 query42 90 83 82 82 query43 595 593 541 541 query44 1136715 703 703 query45 238 223 223 223 query46 1230973 989 973 query47 2036159818901598 query48 974 659 656 656 query49 616 360 354 354 query50 854 618 613 613 query51 4759466847104668 query52 83 79 70 70 query53 437 309 309 309 query54 2628242924202420 query55 87 83 79 79 query56 213 188 211 188 query57 1153105611651056 query58 205 200 203 200 query59 3431313533223135 query60 194 189 178 178 query61 80 79 79 79 query62 886 461 471 461 query63 467 337 328 328 query64 2454147413381338 query65 3607353035213521 query66 799 365 359 359 query67 15943 17006 15254 15254 query68 7600629 644 629 query69 563 347 349 347 query70 1901156415331533 query71 391 295 300 295 query72 6406346633883388 query73 716 308 320 308 query74 6307592057875787 query75 4566371436353635 query76 4552116212001162 query77 538 235 248 235 query78 12613 11659 11621 11621 query79 8065646 635 635 query80 2575390 377 377 query81 519 227 228 227 query82 1526102 98 98 query83 184 126 126 126 query84 259 66 66 66 query85 1399275 273 273 query86 431 371 425 371 query87 3224303529762976 query88 5092228723022287 query89 343 305 285 285 query90 1741206 198 198 query91 152 112 111 111 query92 57 51 49 49 query93 4568585 589 585 query94 875 205 206 205 query95 1095105310351035 query96 644 316 323 316 query97 6464632763456327 query98 196 171 162 162 query99 2934848 952 848 Total cold run time: 309407 ms Total hot run time: 200064 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001934719 ClickBench: Total hot run time: 30.71 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b7807a444fb261d65c47cfab11b01d3e2bf1bb9e, data reload: false query1 0.020.020.01 query2 0.060.020.02 query3 0.240.050.04 query4 1.830.070.06 query5 0.540.530.52 query6 1.250.620.63 query7 0.010.010.01 query8 0.030.020.02 query9 0.530.460.48 query10 0.520.540.53 query11 0.110.080.08 query12 0.110.080.09 query13 0.620.610.60 query14 0.770.780.79 query15 0.760.750.78 query16 0.390.380.37 query17 1.001.030.96 query18 0.210.260.25 query19 1.861.811.83 query20 0.010.010.01 query21 15.50 0.540.55 query22 1.661.731.53 query23 17.23 1.060.97 query24 4.231.221.58 query25 0.410.100.04 query26 0.580.150.16 query27 0.040.040.04 query28 8.460.720.70 query29 12.63 2.292.27 query30 0.560.510.49 query31 2.810.390.37 query32 3.420.490.49 query33 3.053.083.05 query34 15.26 4.804.80 query35 4.884.834.86 query36 1.041.001.00 query37 0.060.050.04 query38 0.040.020.02 query39 0.020.020.01 query40 0.170.140.14 query41 0.060.010.02 query42 0.020.010.01 query43 0.020.010.02 Total cold run time: 103.02 s Total hot run time: 30.71 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001935582 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit b7807a444fb261d65c47cfab11b01d3e2bf1bb9e with default session variables Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 20.4 seconds inserted 1000 Rows, about 490K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
HHoflittlefish777 commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001937292 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server [doris]
hl10105210 closed issue #32330: [Bug] packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server URL: https://github.com/apache/doris/issues/32330 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
Mryange commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2001943405 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
github-actions[bot] commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2001944722 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001946276 TeamCity be ut coverage result: Function Coverage: 37.82% (8042/21264) Line Coverage: 29.49% (65627/222540) Region Coverage: 28.96% (33772/116607) Branch Coverage: 24.80% (17333/69880) Coverage Report: http://coverage.selectdb-in.cc/coverage/b43e1baa14ce00ae6209f3b68a8b108324f367bc_b43e1baa14ce00ae6209f3b68a8b108324f367bc/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001948135 TPC-H: Total hot run time: 49913 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b43e1baa14ce00ae6209f3b68a8b108324f367bc, data reload: false -- Round 1 -- q1 17670 441743964396 q2 2028148 138 138 q3 10495 192719211921 q4 10358 124913311249 q5 8640393939423939 q6 231 124 126 124 q7 2049157315801573 q8 9291272427262724 q9 10847 10509 10453 10453 q10 8615351834993499 q11 423 237 234 234 q12 471 293 300 293 q13 18335 396039603960 q14 352 319 318 318 q15 511 454 458 454 q16 706 591 582 582 q17 1136919 950 919 q18 7209686368676863 q19 1704157515251525 q20 528 313 293 293 q21 4477411640554055 q22 505 401 401 401 Total cold run time: 116581 ms Total hot run time: 49913 ms - Round 2, with runtime_filter_mode=off - q1 4368430942834283 q2 317 221 217 217 q3 4130416241354135 q4 2739276227512751 q5 7274711371647113 q6 237 126 122 122 q7 3211287328642864 q8 4325446644144414 q9 17076 16983 17037 16983 q10 4245422642934226 q11 765 648 663 648 q12 1003849 851 849 q13 7000374936973697 q14 450 410 421 410 q15 504 466 458 458 q16 780 695 708 695 q17 3837384738203820 q18 8840873187328731 q19 1700170716361636 q20 2349209820922092 q21 8419854683088308 q22 1044988 938 938 Total cold run time: 84613 ms Total hot run time: 79390 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001951199 TPC-DS: Total hot run time: 201713 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b43e1baa14ce00ae6209f3b68a8b108324f367bc, data reload: false query1 928 381 375 375 query2 6517231021422142 query3 6921200 194 194 query4 21289 17972 17868 17868 query5 19718 651465036503 query6 276 218 226 218 query7 4150298 307 298 query8 271 234 220 220 query9 3130268226142614 query10 406 298 297 297 query11 11218 10704 10551 10551 query12 130 74 71 71 query13 5577639 628 628 query14 17782 13592 13853 13592 query15 356 224 243 224 query16 6475271 262 262 query17 17581449873 873 query18 2329408 404 404 query19 202 143 149 143 query20 76 71 77 71 query21 185 95 96 95 query22 5188507650845076 query23 32907 32089 31930 31930 query24 6943661265096509 query25 529 412 403 403 query26 526 163 158 158 query27 1901296 291 291 query28 6031220321762176 query29 2865277226152615 query30 244 164 161 161 query31 898 721 726 721 query32 66 59 61 59 query33 392 236 257 236 query34 844 461 491 461 query35 1082899 954 899 query36 1597128916961289 query37 89 65 56 56 query38 3075287529342875 query39 1365132413221322 query40 198 92 94 92 query41 34 32 31 31 query42 94 85 83 83 query43 681 566 584 566 query44 1132711 707 707 query45 248 228 227 227 query46 1241954 962 954 query47 1752165418311654 query48 977 686 670 670 query49 610 367 382 367 query50 880 575 624 575 query51 4756463946704639 query52 90 82 82 82 query53 456 328 309 309 query54 2632242324532423 query55 86 79 79 79 query56 207 204 204 204 query57 1162106510941065 query58 202 199 210 199 query59 3524339934443399 query60 202 199 186 186 query61 84 85 85 85 query62 790 528 450 450 query63 467 345 335 335 query64 2375150114481448 query65 3619356236063562 query66 802 373 376 373 query67 15520 16007 15254 15254 query68 8584661 670 661 query69 585 343 335 335 query70 1786178216681668 query71 414 302 299 299 query72 6487343234583432 query73 743 320 327 320 query74 6252581058345810 query75 4744368337203683 query76 4822119712111197 query77 644 246 241 241 query78 12498 11466 11730 11466 query79 7606630 658 630 query80 1059392 392 392 query81 503 226 234 226 query82 150195 100 95 query83 166 133 128 128 query84 263 68 68 68 query85 937 276 272 272 query86 392 411 363 363 query87 3264299930362999 query88 4965233623412336 query89 367 334 290 290 query90 1850192 194 192 query91 146 116 113 113 query92 57 55 50 50 query93 4163561 589 561 query94 723 207 203 203 query95 1110103910381038 query96 644 326 318 318 query97 6466640763846384 query98 176 178 179 178 query99 2937884 867 867 Total cold run time: 307911 ms Total hot run time: 201713 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
[PR] [opt](like) opt like clause with concat(col, pattern_str) [doris]
sjyango opened a new pull request, #32333:
URL: https://github.com/apache/doris/pull/32333
## Proposed changes
Issue Number: close #xxx
opt like clause with concat(col, pattern_str), such as concat('%', col),
concat(col, '%'), concat('%', col, '%').
- before this pr:
https://github.com/apache/doris/assets/114391101/505ef331-f3c8-444c-ac87-11e11cae3d23";>
- after this pr:
https://github.com/apache/doris/assets/114391101/9a366f85-9cf6-4d60-912d-898c0eebde84";>
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](like) opt like clause with concat(col, pattern_str) [doris]
doris-robot commented on PR #32333: URL: https://github.com/apache/doris/pull/32333#issuecomment-2001951894 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001952627 ClickBench: Total hot run time: 30.48 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b43e1baa14ce00ae6209f3b68a8b108324f367bc, data reload: false query1 0.030.020.02 query2 0.060.020.02 query3 0.240.040.04 query4 1.830.070.07 query5 0.530.530.53 query6 1.270.620.63 query7 0.010.010.01 query8 0.040.020.02 query9 0.530.490.48 query10 0.520.530.53 query11 0.120.090.08 query12 0.110.090.09 query13 0.620.610.59 query14 0.780.810.77 query15 0.780.760.77 query16 0.360.370.37 query17 1.030.991.02 query18 0.230.260.23 query19 1.931.851.82 query20 0.010.010.01 query21 15.47 0.560.53 query22 2.272.111.77 query23 17.12 1.030.97 query24 6.760.631.01 query25 0.400.080.05 query26 0.680.170.15 query27 0.040.040.03 query28 6.870.760.72 query29 12.68 2.312.26 query30 0.560.540.51 query31 2.810.400.39 query32 3.380.500.51 query33 3.083.063.06 query34 15.25 4.794.77 query35 4.864.834.84 query36 1.031.001.00 query37 0.060.040.04 query38 0.030.020.02 query39 0.010.010.01 query40 0.160.140.14 query41 0.060.020.02 query42 0.020.020.02 query43 0.020.010.02 Total cold run time: 104.65 s Total hot run time: 30.48 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
doris-robot commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001953317 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit b43e1baa14ce00ae6209f3b68a8b108324f367bc with default session variables Stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 20.7 seconds inserted 1000 Rows, about 483K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2001953940 TPC-H: Total hot run time: 38366 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f463020199c9baf174d7f71d1c74972c580c6293, data reload: false -- Round 1 -- q1 17627 472241134113 q2 2022159 147 147 q3 10584 1115893 893 q4 7779759 688 688 q5 7462265427782654 q6 182 122 123 122 q7 1228836 790 790 q8 9338197719841977 q9 7288661065166516 q10 8521357736993577 q11 429 222 226 222 q12 738 300 299 299 q13 17961 286728532853 q14 277 243 244 243 q15 495 459 453 453 q16 530 404 396 396 q17 953 518 537 518 q18 7239658463436343 q19 1874146314081408 q20 552 293 277 277 q21 6097358635793579 q22 351 298 309 298 Total cold run time: 109527 ms Total hot run time: 38366 ms - Round 2, with runtime_filter_mode=off - q1 4083407241054072 q2 317 219 225 219 q3 2947285128312831 q4 1826152715431527 q5 5280531553475315 q6 190 126 119 119 q7 2265186418401840 q8 3171330833023302 q9 8696868486268626 q10 3796377937853779 q11 544 456 441 441 q12 720 537 568 537 q13 16888 285028992850 q14 278 244 257 244 q15 473 449 455 449 q16 440 425 422 422 q17 1719146614531453 q18 7560724171057105 q19 1609155315361536 q20 1890171217281712 q21 4850466147184661 q22 533 485 430 430 Total cold run time: 70075 ms Total hot run time: 53470 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](like) opt like clause with concat(col, pattern_str) [doris]
sjyango commented on PR #32333: URL: https://github.com/apache/doris/pull/32333#issuecomment-2001956228 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2001956821 TeamCity be ut coverage result: Function Coverage: 34.94% (8574/24539) Line Coverage: 26.63% (69496/260928) Region Coverage: 25.93% (36104/139255) Branch Coverage: 22.86% (18433/80628) Coverage Report: http://coverage.selectdb-in.cc/coverage/f463020199c9baf174d7f71d1c74972c580c6293_f463020199c9baf174d7f71d1c74972c580c6293/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](like) opt like clause with concat(col, pattern_str) [doris]
github-actions[bot] commented on code in PR #32333:
URL: https://github.com/apache/doris/pull/32333#discussion_r1527157317
##
be/src/vec/functions/like.cpp:
##
@@ -60,6 +60,92 @@
static const re2::RE2 LIKE_EQUALS_RE("(((_)|([^%_]))+)");
static const re2::RE2 LIKE_ALLPASS_RE("%+");
+struct VectorAllpassSearchState : public VectorLikeSearchState {
+VectorAllpassSearchState()
+: VectorLikeSearchState(PatternType::ALLPASS,
FunctionLikeBase::vector_allpass_fn) {}
+
+~VectorAllpassSearchState() = default;
+
+void pattern_match(const std::string& pattern_str) override {
+if (!pattern_str.empty() && RE2::FullMatch(pattern_str,
LIKE_ALLPASS_RE)) {
+_search_strings->insert_default();
+} else {
+_pattern_matched = false;
+}
+}
+};
+
+struct VectorEqualSearchState : public VectorLikeSearchState {
+VectorEqualSearchState()
+: VectorLikeSearchState(PatternType::EQUAL,
FunctionLikeBase::vector_equals_fn) {}
+
+~VectorEqualSearchState() = default;
+
+void pattern_match(const std::string& pattern_str) override {
+std::string search_string;
+if (pattern_str.empty() || RE2::FullMatch(pattern_str, LIKE_EQUALS_RE,
&search_string)) {
+FunctionLike::remove_escape_character(&search_string);
+_search_strings->insert_data(search_string.c_str(),
search_string.size());
+} else {
+_pattern_matched = false;
+}
+}
+};
+
+struct VectorSubStringSearchState : public VectorLikeSearchState {
+VectorSubStringSearchState()
+: VectorLikeSearchState(PatternType::SUBSTRING,
FunctionLikeBase::vector_substring_fn) {
+}
+
+~VectorSubStringSearchState() = default;
Review Comment:
warning: annotate this function with 'override' or (rarely) 'final'
[modernize-use-override]
```suggestion
~VectorSubStringSearchState() override = default;
```
##
be/src/vec/functions/like.cpp:
##
@@ -60,6 +60,92 @@
static const re2::RE2 LIKE_EQUALS_RE("(((_)|([^%_]))+)");
static const re2::RE2 LIKE_ALLPASS_RE("%+");
+struct VectorAllpassSearchState : public VectorLikeSearchState {
+VectorAllpassSearchState()
+: VectorLikeSearchState(PatternType::ALLPASS,
FunctionLikeBase::vector_allpass_fn) {}
+
+~VectorAllpassSearchState() = default;
+
+void pattern_match(const std::string& pattern_str) override {
+if (!pattern_str.empty() && RE2::FullMatch(pattern_str,
LIKE_ALLPASS_RE)) {
+_search_strings->insert_default();
+} else {
+_pattern_matched = false;
+}
+}
+};
+
+struct VectorEqualSearchState : public VectorLikeSearchState {
+VectorEqualSearchState()
+: VectorLikeSearchState(PatternType::EQUAL,
FunctionLikeBase::vector_equals_fn) {}
+
+~VectorEqualSearchState() = default;
Review Comment:
warning: annotate this function with 'override' or (rarely) 'final'
[modernize-use-override]
```suggestion
~VectorEqualSearchState() override = default;
```
##
be/src/vec/functions/like.cpp:
##
@@ -123,6 +251,29 @@
return Status::OK();
}
+Status FunctionLikeBase::constant_ends_with_fn_scalar(LikeSearchState* state,
const StringRef& val,
+ const StringRef& pattern,
+ unsigned char* result) {
+*result = (val.size >= state->search_string_sv.size) &&
+ (state->search_string_sv == val.substring(val.size -
state->search_string_sv.size,
+
state->search_string_sv.size));
+return Status::OK();
+}
+
+Status FunctionLikeBase::vector_ends_with_fn(const ColumnString& vals,
Review Comment:
warning: method 'vector_ends_with_fn' can be made static
[readability-convert-member-functions-to-static]
```suggestion
static Status FunctionLikeBase::vector_ends_with_fn(const ColumnString& vals,
```
##
be/src/vec/functions/like.cpp:
##
@@ -60,6 +60,92 @@
static const re2::RE2 LIKE_EQUALS_RE("(((_)|([^%_]))+)");
static const re2::RE2 LIKE_ALLPASS_RE("%+");
+struct VectorAllpassSearchState : public VectorLikeSearchState {
+VectorAllpassSearchState()
+: VectorLikeSearchState(PatternType::ALLPASS,
FunctionLikeBase::vector_allpass_fn) {}
+
+~VectorAllpassSearchState() = default;
+
+void pattern_match(const std::string& pattern_str) override {
+if (!pattern_str.empty() && RE2::FullMatch(pattern_str,
LIKE_ALLPASS_RE)) {
+_search_strings->insert_default();
+} else {
+_pattern_matched = false;
+}
+}
+};
+
+struct VectorEqualSearchState : public VectorLikeSearchState {
+VectorEqualSearchState()
+: VectorLikeSearchState(PatternType::EQUAL,
FunctionLikeBase::vector_equals_fn) {}
+
+
Re: [I] [Bug] How to resolve kafka consume timeout problem of ROUTINE LOAD in doris2.0.1? [doris]
kevinu2 commented on issue #27932: URL: https://github.com/apache/doris/issues/27932#issuecomment-2001958476 how to fix this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](like) opt like clause with concat(col, pattern_str) [doris]
doris-robot commented on PR #32333: URL: https://github.com/apache/doris/pull/32333#issuecomment-2001963499 TPC-H: Total hot run time: 38353 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit dd2248ae77ab4f222504fdf3fbbbaab4672e990f, data reload: false -- Round 1 -- q1 17712 424441614161 q2 2023160 154 154 q3 10656 1070902 902 q4 7780767 724 724 q5 7473265125612561 q6 186 119 121 119 q7 1215825 804 804 q8 9331205420362036 q9 7171647064096409 q10 8451349736003497 q11 428 220 217 217 q12 634 308 294 294 q13 1 285028612850 q14 275 252 248 248 q15 492 459 443 443 q16 488 399 390 390 q17 957 542 537 537 q18 7369646365126463 q19 5681148014271427 q20 556 287 270 270 q21 6274360235583558 q22 364 306 289 289 Total cold run time: 113293 ms Total hot run time: 38353 ms - Round 2, with runtime_filter_mode=off - q1 4146406140794061 q2 330 229 228 228 q3 2990288528512851 q4 1830151815711518 q5 5249528752715271 q6 195 116 115 115 q7 2279185018801850 q8 3168331433103310 q9 8616856185308530 q10 3715365937043659 q11 547 441 444 441 q12 739 575 549 549 q13 16944 289128312831 q14 295 244 258 244 q15 490 442 448 442 q16 472 429 408 408 q17 1743151514711471 q18 7420712970587058 q19 1638155015391539 q20 1900171217411712 q21 4812475747264726 q22 567 450 456 450 Total cold run time: 70085 ms Total hot run time: 53264 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function) fixed the get_json_string function [doris]
cambyzju commented on PR #32149: URL: https://github.com/apache/doris/pull/32149#issuecomment-2001975524 cherry-pick https://github.com/apache/doris/pull/32150 to 1.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](function) fixed the get_json_string function [doris]
cambyzju merged PR #32149: URL: https://github.com/apache/doris/pull/32149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [fix](function) fixed the get_json_string function (#32149)
This is an automated email from the ASF dual-hosted git repository.
cambyzju pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 64e332b1ea1 [fix](function) fixed the get_json_string function (#32149)
64e332b1ea1 is described below
commit 64e332b1ea1036b694184f5f33d41726a1cc979f
Author: lw112 <131352377+felixw...@users.noreply.github.com>
AuthorDate: Sat Mar 16 20:41:46 2024 +0800
[fix](function) fixed the get_json_string function (#32149)
---
be/src/vec/functions/function_json.cpp | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/be/src/vec/functions/function_json.cpp
b/be/src/vec/functions/function_json.cpp
index d4c46d829a2..11d75110a3d 100644
--- a/be/src/vec/functions/function_json.cpp
+++ b/be/src/vec/functions/function_json.cpp
@@ -23,6 +23,7 @@
#include
#include "exprs/json_functions.h"
+#include "vec/io/io_helper.h"
#ifdef __AVX2__
#include "util/jsonb_parser_simd.h"
#else
@@ -350,7 +351,7 @@ struct GetJsonString {
rapidjson::Value* root = nullptr;
root = get_json_object(json_string, path_string,
&document);
-const int max_string_len = (l_size <= 65535) ? 65535 : (l_size +
1);
+const int max_string_len = DEFAULT_MAX_JSON_SIZE;
if (root == nullptr || root->IsNull()) {
StringOP::push_null_string(i, res_data, res_offsets, null_map);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
github-actions[bot] commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001977056 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.0](load) fix load channel may memory leak [doris]
github-actions[bot] commented on PR #32301: URL: https://github.com/apache/doris/pull/32301#issuecomment-2001977063 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) enhance auto resume to keep routine load stable [doris]
github-actions[bot] commented on PR #32239: URL: https://github.com/apache/doris/pull/32239#issuecomment-2001981411 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) enhance auto resume to keep routine load stable [doris]
github-actions[bot] commented on PR #32239: URL: https://github.com/apache/doris/pull/32239#issuecomment-2001981416 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) optimize allocate task to be algorithm for load balance [doris]
github-actions[bot] commented on PR #32021: URL: https://github.com/apache/doris/pull/32021#issuecomment-2001981771 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [docs](docs) Fix flink doris connector typo of v2.0 [doris]
vinlee19 commented on PR #32218: URL: https://github.com/apache/doris/pull/32218#issuecomment-2001983691 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Cloud) Support drop storage vault in Meta Service [doris]
gavinchou commented on code in PR #32228:
URL: https://github.com/apache/doris/pull/32228#discussion_r1527172066
##
cloud/test/meta_service_test.cpp:
##
@@ -5088,6 +5082,113 @@ TEST(MetaServiceTest, AddHdfsInfoTest) {
SyncPoint::get_instance()->clear_all_call_backs();
}
+TEST(MetaServiceTest, DropHdfsInfoTest) {
+auto meta_service = get_meta_service();
+
+auto sp = cloud::SyncPoint::get_instance();
+sp->enable_processing();
+sp->set_call_back("encrypt_ak_sk:get_encryption_key_ret",
+ [](void* p) { *reinterpret_cast(p) = 0; });
+sp->set_call_back("encrypt_ak_sk:get_encryption_key", [](void* p) {
+*reinterpret_cast(p) =
"selectdbselectdbselectdbselectdb";
+});
+sp->set_call_back("encrypt_ak_sk:get_encryption_key_id",
+ [](void* p) { *reinterpret_cast(p) = 1; });
+
+std::unique_ptr txn;
+ASSERT_EQ(meta_service->txn_kv()->create_txn(&txn), TxnErrorCode::TXN_OK);
+std::string key;
+std::string val;
+InstanceKeyInfo key_info {"test_instance"};
+instance_key(key_info, &key);
+
+ObjectStoreInfoPB obj_info;
+obj_info.set_id("1");
+obj_info.set_ak("ak");
+obj_info.set_sk("sk");
+StorageVaultPB vault;
+vault.set_name("test_hdfs_vault");
+vault.set_id("2");
+InstanceInfoPB instance;
+instance.add_obj_info()->CopyFrom(obj_info);
+instance.add_storage_vault_names(vault.name());
+instance.add_resource_ids(vault.id());
+val = instance.SerializeAsString();
+txn->put(key, val);
+txn->put(storage_vault_key({instance.instance_id(), "2"}),
vault.SerializeAsString());
+ASSERT_EQ(txn->commit(), TxnErrorCode::TXN_OK);
+
+auto get_test_instance = [&](InstanceInfoPB& i) {
+std::string key;
+std::string val;
+std::unique_ptr txn;
+ASSERT_EQ(meta_service->txn_kv()->create_txn(&txn),
TxnErrorCode::TXN_OK);
+InstanceKeyInfo key_info {"test_instance"};
+instance_key(key_info, &key);
+ASSERT_EQ(txn->get(key, &val), TxnErrorCode::TXN_OK);
+i.ParseFromString(val);
+};
+
+// update failed because has no storage vault set
+{
+AlterObjStoreInfoRequest req;
+req.set_cloud_unique_id("test_cloud_unique_id");
+req.set_op(AlterObjStoreInfoRequest::DROP_HDFS_INFO);
+
+brpc::Controller cntl;
+AlterObjStoreInfoResponse res;
+meta_service->alter_obj_store_info(
+reinterpret_cast<::google::protobuf::RpcController*>(&cntl),
&req, &res, nullptr);
+ASSERT_EQ(res.status().code(), MetaServiceCode::INVALID_ARGUMENT) <<
res.status().msg();
+}
+
+// update failed because vault name does not exist
+{
+AlterObjStoreInfoRequest req;
+req.set_cloud_unique_id("test_cloud_unique_id");
+req.set_op(AlterObjStoreInfoRequest::DROP_HDFS_INFO);
+StorageVaultPB hdfs;
+hdfs.set_name("test_hdfs_vault_not_found");
+HdfsVaultInfo params;
+
+hdfs.mutable_hdfs_info()->CopyFrom(params);
+req.mutable_hdfs()->CopyFrom(hdfs);
+
+brpc::Controller cntl;
+AlterObjStoreInfoResponse res;
+meta_service->alter_obj_store_info(
+reinterpret_cast<::google::protobuf::RpcController*>(&cntl),
&req, &res, nullptr);
+ASSERT_EQ(res.status().code(),
MetaServiceCode::STORAGE_VAULT_NOT_FOUND)
+<< res.status().msg();
+}
+
+// update successfully
+{
+AlterObjStoreInfoRequest req;
+req.set_cloud_unique_id("test_cloud_unique_id");
+req.set_op(AlterObjStoreInfoRequest::DROP_HDFS_INFO);
+StorageVaultPB hdfs;
+hdfs.set_name("test_hdfs_vault");
+HdfsVaultInfo params;
+
+hdfs.mutable_hdfs_info()->CopyFrom(params);
+req.mutable_hdfs()->CopyFrom(hdfs);
+
+brpc::Controller cntl;
+AlterObjStoreInfoResponse res;
+meta_service->alter_obj_store_info(
+reinterpret_cast<::google::protobuf::RpcController*>(&cntl),
&req, &res, nullptr);
+ASSERT_EQ(res.status().code(), MetaServiceCode::OK) <<
res.status().msg();
+InstanceInfoPB instance;
+get_test_instance(instance);
+ASSERT_EQ(instance.resource_ids().size(), 0);
+ASSERT_EQ(instance.storage_vault_names().size(), 0);
Review Comment:
read the InstanceInfoPB again and read the removed key to ensure all things
go right
##
cloud/src/meta-service/meta_service_resource.cpp:
##
@@ -322,16 +328,32 @@ static int add_hdfs_storage_valut(InstanceInfoPB&
instance, Transaction* txn,
hdfs_param.set_id(vault_id);
std::string val = hdfs_param.SerializeAsString();
txn->put(key, val);
-LOG_INFO("try to put storage vault_id={}, vault_name={}, err={}",
vault_id, hdfs_param.name());
+LOG_INFO("try to put storage vault_id={}, vault_name={}", vault_id,
hdfs_param.name());
instance.mutable_resource_ids()->Ad
Re: [PR] [refactor](chore) replace HashMapWithStackMemory with std::unordered_map [doris]
yiguolei commented on PR #32309: URL: https://github.com/apache/doris/pull/32309#issuecomment-2001988303 @cambyzju we want to calculate the memory using memtracker, but the hash table is used in segment cache, we could not using an allocator to calculate memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) optimize routine load task limit [doris]
github-actions[bot] commented on PR #32282: URL: https://github.com/apache/doris/pull/32282#issuecomment-2001989081 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](routine-load) optimize routine load task limit [doris]
github-actions[bot] commented on PR #32282: URL: https://github.com/apache/doris/pull/32282#issuecomment-2001989090 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Cloud) Support drop storage vault in Meta Service [doris]
github-actions[bot] commented on code in PR #32228:
URL: https://github.com/apache/doris/pull/32228#discussion_r1527175059
##
cloud/test/meta_service_test.cpp:
##
@@ -5088,6 +5082,143 @@ TEST(MetaServiceTest, AddHdfsInfoTest) {
SyncPoint::get_instance()->clear_all_call_backs();
}
+TEST(MetaServiceTest, DropHdfsInfoTest) {
Review Comment:
warning: function 'TEST' exceeds recommended size/complexity thresholds
[readability-function-size]
```cpp
TEST(MetaServiceTest, DropHdfsInfoTest) {
^
```
Additional context
**cloud/test/meta_service_test.cpp:5084:** 135 lines including whitespace
and comments (threshold 80)
```cpp
TEST(MetaServiceTest, DropHdfsInfoTest) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Cloud) Support drop storage vault in Meta Service [doris]
github-actions[bot] commented on PR #32228: URL: https://github.com/apache/doris/pull/32228#issuecomment-2001992037 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Cloud) Support drop storage vault in Meta Service [doris]
ByteYue commented on PR #32228: URL: https://github.com/apache/doris/pull/32228#issuecomment-2001995354 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Cloud) Support drop storage vault in Meta Service [doris]
github-actions[bot] commented on code in PR #32228:
URL: https://github.com/apache/doris/pull/32228#discussion_r1527178065
##
cloud/test/meta_service_test.cpp:
##
@@ -5088,6 +5082,248 @@ TEST(MetaServiceTest, AddHdfsInfoTest) {
SyncPoint::get_instance()->clear_all_call_backs();
}
+TEST(MetaServiceTest, DropHdfsInfoTest) {
Review Comment:
warning: function 'TEST' exceeds recommended size/complexity thresholds
[readability-function-size]
```cpp
TEST(MetaServiceTest, DropHdfsInfoTest) {
^
```
Additional context
**cloud/test/meta_service_test.cpp:5084:** 240 lines including whitespace
and comments (threshold 80)
```cpp
TEST(MetaServiceTest, DropHdfsInfoTest) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance][fix] add ccr downstreamurl for ccr [doris]
dataroaring merged PR #32325: URL: https://github.com/apache/doris/pull/32325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [enhance][fix] add ccr downstreamurl for ccr (#32325)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 6d45d88273b [enhance][fix] add ccr downstreamurl for ccr (#32325) 6d45d88273b is described below commit 6d45d88273b5454d974490527e46fd43e9cce097 Author: XuJianxu AuthorDate: Sat Mar 16 22:11:12 2024 +0800 [enhance][fix] add ccr downstreamurl for ccr (#32325) Co-authored-by: 胥剑旭 --- regression-test/conf/regression-conf.groovy | 5 + 1 file changed, 5 insertions(+) diff --git a/regression-test/conf/regression-conf.groovy b/regression-test/conf/regression-conf.groovy index c6764a2ce86..e8256233742 100644 --- a/regression-test/conf/regression-conf.groovy +++ b/regression-test/conf/regression-conf.groovy @@ -29,6 +29,11 @@ targetJdbcUrl = "jdbc:mysql://127.0.0.1:9030/?useLocalSessionState=true&allowLoa jdbcUser = "root" jdbcPassword = "" +ccrDownstreamUrl = "jdbc:mysql://127.0.0.1:9030/?useLocalSessionState=true&allowLoadLocalInfile=true" +ccrDownstreamUser = "root" +ccrDownstreamPassword = "" +ccrDownstreamFeThriftAddress = "127.0.0.1:9020" + feSourceThriftAddress = "127.0.0.1:9020" feTargetThriftAddress = "127.0.0.1:9020" syncerAddress = "127.0.0.1:9190" - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] Only support subquery in binary predicate in case statement [doris]
iwanttobepowerful commented on issue #31300: URL: https://github.com/apache/doris/issues/31300#issuecomment-2002003801 try add this ```sql /*+SET_VAR(disable_nereids_rules='TRANSPOSE_LOGICAL_SEMI_JOIN_AGG_PROJECT, TRANSPOSE_LOGICAL_SEMI_JOIN_AGG')*/ ``` @wmtbnbo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Cloud) Support drop storage vault in Meta Service [doris]
doris-robot commented on PR #32228: URL: https://github.com/apache/doris/pull/32228#issuecomment-2002004723 TeamCity be ut coverage result: Function Coverage: 34.94% (8572/24535) Line Coverage: 26.64% (69472/260789) Region Coverage: 25.92% (36083/139190) Branch Coverage: 22.87% (18426/80578) Coverage Report: http://coverage.selectdb-in.cc/coverage/e99fbeed29cfe7eb41a124e5a8dfc77e425013f9_e99fbeed29cfe7eb41a124e5a8dfc77e425013f9/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](test) exclude somecases not support in cloud mode [doris]
dataroaring commented on PR #32108: URL: https://github.com/apache/doris/pull/32108#issuecomment-2002004794 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](Cloud) Support drop storage vault in Meta Service [doris]
doris-robot commented on PR #32228: URL: https://github.com/apache/doris/pull/32228#issuecomment-2002005232 TPC-H: Total hot run time: 38541 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e99fbeed29cfe7eb41a124e5a8dfc77e425013f9, data reload: false -- Round 1 -- q1 17640 423041064106 q2 2017155 143 143 q3 10601 1096903 903 q4 8117757 706 706 q5 7475274227012701 q6 186 125 123 123 q7 1174825 806 806 q8 9335203819821982 q9 7146644364436443 q10 8855354736873547 q11 429 234 221 221 q12 618 316 295 295 q13 18015 291029502910 q14 289 250 256 250 q15 514 463 454 454 q16 518 391 397 391 q17 952 542 619 542 q18 7296650364676467 q19 5238137014621370 q20 548 302 285 285 q21 6429364235963596 q22 372 300 331 300 Total cold run time: 113764 ms Total hot run time: 38541 ms - Round 2, with runtime_filter_mode=off - q1 4227417741834177 q2 337 235 237 235 q3 3150298730052987 q4 1927166116361636 q5 5412540554765405 q6 193 116 119 116 q7 2345198618461846 q8 3191332432813281 q9 8682857385628562 q10 3727368436623662 q11 541 432 439 432 q12 706 538 552 538 q13 17061 285828692858 q14 293 247 256 247 q15 477 446 457 446 q16 461 410 407 407 q17 1737148014471447 q18 7488725871207120 q19 4460157315511551 q20 1928171317001700 q21 4899474845574557 q22 509 446 455 446 Total cold run time: 73751 ms Total hot run time: 53656 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Enhancement]Make client_cache_helper reopen func thread-safe [doris]
Bears0haunt opened a new pull request, #32334: URL: https://github.com/apache/doris/pull/32334 1. for thread safe, Find and delete should be synchronized 2. but when client getting open, there might be I/O Tradeoffs need to be considered ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement]Make client_cache_helper reopen func thread-safe [doris]
doris-robot commented on PR #32334: URL: https://github.com/apache/doris/pull/32334#issuecomment-2002008689 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement]Make client_cache_helper reopen func thread-safe [doris]
Bears0haunt commented on PR #32334: URL: https://github.com/apache/doris/pull/32334#issuecomment-2002008730 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](chore) replace HashMapWithStackMemory with std::unordered_map [doris]
eldenmoon commented on PR #32309: URL: https://github.com/apache/doris/pull/32309#issuecomment-2002009313 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement]Make client_cache_helper reopen func thread-safe [doris]
github-actions[bot] commented on PR #32334: URL: https://github.com/apache/doris/pull/32334#issuecomment-2002010130 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Enhancement] when partition column is datetime, date can work in create table command [doris]
xiedeyantu opened a new pull request, #32335: URL: https://github.com/apache/doris/pull/32335 ## Proposed changes Issue Number: close #32323 ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] when partition column is datetime, date can work in create table command [doris]
doris-robot commented on PR #32335: URL: https://github.com/apache/doris/pull/32335#issuecomment-2002013858 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] when partition column is datetime, date can work in create table command [doris]
xiedeyantu commented on PR #32335: URL: https://github.com/apache/doris/pull/32335#issuecomment-2002016417 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement]Make client_cache_helper reopen func thread-safe [doris]
doris-robot commented on PR #32334: URL: https://github.com/apache/doris/pull/32334#issuecomment-2002018904 TeamCity be ut coverage result: Function Coverage: 34.95% (8574/24535) Line Coverage: 26.65% (69501/260770) Region Coverage: 25.94% (36107/139181) Branch Coverage: 22.88% (18436/80572) Coverage Report: http://coverage.selectdb-in.cc/coverage/d99e0c5644b01be291991b00505939356bd22fb8_d99e0c5644b01be291991b00505939356bd22fb8/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [behavior change](output) change float output format [doris]
Mryange commented on PR #32049: URL: https://github.com/apache/doris/pull/32049#issuecomment-2002019661 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [behavior change](output) change float output format [doris]
github-actions[bot] commented on PR #32049: URL: https://github.com/apache/doris/pull/32049#issuecomment-2002021211 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](chore) replace HashMapWithStackMemory with std::unordered_map [doris]
doris-robot commented on PR #32309: URL: https://github.com/apache/doris/pull/32309#issuecomment-2002021564 TPC-H: Total hot run time: 38370 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8af5cf038106fb7e7d82253d5757a2f5803a06f0, data reload: false -- Round 1 -- q1 17663 427841374137 q2 2028155 146 146 q3 10648 1131900 900 q4 7778802 713 713 q5 7520276926812681 q6 187 123 122 122 q7 1208823 816 816 q8 9340200820162008 q9 7208645863776377 q10 8501354236313542 q11 436 233 217 217 q12 599 301 294 294 q13 17805 287728672867 q14 273 247 266 247 q15 507 456 453 453 q16 511 394 387 387 q17 982 577 525 525 q18 7323661764176417 q19 4545140314951403 q20 538 297 290 290 q21 6265353735633537 q22 352 311 291 291 Total cold run time: 112217 ms Total hot run time: 38370 ms - Round 2, with runtime_filter_mode=off - q1 4157409240944092 q2 326 225 219 219 q3 3008288028542854 q4 1854156715881567 q5 5250528452325232 q6 193 117 119 117 q7 2253183218531832 q8 3177329333083293 q9 8627857385438543 q10 3707368836673667 q11 537 441 434 434 q12 713 542 554 542 q13 16908 291928582858 q14 274 258 263 258 q15 486 451 444 444 q16 451 411 401 401 q17 1756151314841484 q18 7490715770737073 q19 1619152915471529 q20 1919171517001700 q21 4778483946654665 q22 541 441 471 441 Total cold run time: 70024 ms Total hot run time: 53245 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
github-actions[bot] commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2002022885 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
Mryange commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2002023848 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
github-actions[bot] commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2002024587 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [behavior change](output) change float output format [doris]
doris-robot commented on PR #32049: URL: https://github.com/apache/doris/pull/32049#issuecomment-2002025175 TeamCity be ut coverage result: Function Coverage: 34.95% (8574/24535) Line Coverage: 26.65% (69487/260780) Region Coverage: 25.93% (36087/139182) Branch Coverage: 22.86% (18423/80574) Coverage Report: http://coverage.selectdb-in.cc/coverage/68d927450d2e0bee100ccb3276e59e9379056496_68d927450d2e0bee100ccb3276e59e9379056496/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](chore) replace HashMapWithStackMemory with std::unordered_map [doris]
doris-robot commented on PR #32309: URL: https://github.com/apache/doris/pull/32309#issuecomment-2002025410 TeamCity be ut coverage result: Function Coverage: 34.95% (8574/24530) Line Coverage: 26.65% (69499/260778) Region Coverage: 25.94% (36106/139181) Branch Coverage: 22.87% (18432/80578) Coverage Report: http://coverage.selectdb-in.cc/coverage/8af5cf038106fb7e7d82253d5757a2f5803a06f0_8af5cf038106fb7e7d82253d5757a2f5803a06f0/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] when partition column is datetime, date can work in create table command [doris]
doris-robot commented on PR #32335: URL: https://github.com/apache/doris/pull/32335#issuecomment-2002026409 TPC-H: Total hot run time: 38000 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 71ffe10618e18155d32f7a7d4738dd192b8efb9c, data reload: false -- Round 1 -- q1 17647 419541004100 q2 2019149 138 138 q3 10608 1070903 903 q4 7786750 675 675 q5 7450262125772577 q6 183 123 122 122 q7 1199826 797 797 q8 9338194919991949 q9 7213642164106410 q10 8529349435823494 q11 432 213 214 213 q12 791 285 299 285 q13 18102 281628582816 q14 283 252 246 246 q15 499 452 441 441 q16 503 381 386 381 q17 942 525 529 525 q18 7093648364676467 q19 3278144313671367 q20 539 290 268 268 q21 6019358635223522 q22 345 304 329 304 Total cold run time: 110798 ms Total hot run time: 38000 ms - Round 2, with runtime_filter_mode=off - q1 4100407540414041 q2 323 224 226 224 q3 2966280227862786 q4 1794150815071507 q5 5204522452325224 q6 193 115 117 115 q7 181618601816 q8 3133326332653263 q9 8481849785158497 q10 3719363436573634 q11 538 431 431 431 q12 702 553 529 529 q13 16922 288828132813 q14 281 242 251 242 q15 484 454 433 433 q16 445 397 400 397 q17 1711148114601460 q18 7403713070717071 q19 1607154615051505 q20 1910175016821682 q21 4784465247184652 q22 527 470 474 470 Total cold run time: 69449 ms Total hot run time: 52792 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2002029431 TPC-H: Total hot run time: 38865 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit bf9db3a2f2e9a2b223a167e80ac223edc58013a7, data reload: false -- Round 1 -- q1 17696 449241794179 q2 2026154 148 148 q3 10764 1132906 906 q4 7872793 741 741 q5 7467280828652808 q6 186 125 122 122 q7 1184837 819 819 q8 9379203719911991 q9 7272656365826563 q10 8587360236723602 q11 423 227 219 219 q12 652 306 292 292 q13 17792 288028682868 q14 282 254 248 248 q15 498 461 439 439 q16 512 398 402 398 q17 963 554 637 554 q18 7449645164156415 q19 4767147414081408 q20 555 292 292 292 q21 6157355538773555 q22 368 336 298 298 Total cold run time: 112851 ms Total hot run time: 38865 ms - Round 2, with runtime_filter_mode=off - q1 4180428440854085 q2 327 224 241 224 q3 2991287528522852 q4 1868149815611498 q5 5298533253435332 q6 195 116 119 116 q7 2253190418971897 q8 3148329532853285 q9 8724873287578732 q10 3810380037403740 q11 545 450 463 450 q12 709 540 549 540 q13 16917 290628272827 q14 280 248 273 248 q15 487 446 451 446 q16 449 407 429 407 q17 1749147414931474 q18 7553724171807180 q19 2096153615481536 q20 1927171217091709 q21 4769477446564656 q22 535 453 461 453 Total cold run time: 70810 ms Total hot run time: 53687 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [behavior change](output) change float output format [doris]
doris-robot commented on PR #32049: URL: https://github.com/apache/doris/pull/32049#issuecomment-2002029652 TPC-H: Total hot run time: 38193 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 68d927450d2e0bee100ccb3276e59e9379056496, data reload: false -- Round 1 -- q1 17682 415741304130 q2 2012156 145 145 q3 10593 1070885 885 q4 7767741 725 725 q5 7448273627532736 q6 185 125 123 123 q7 1165802 800 800 q8 9325198819481948 q9 7197643464176417 q10 8507344635953446 q11 428 229 224 224 q12 706 290 294 290 q13 18000 284928552849 q14 290 255 243 243 q15 500 465 450 450 q16 504 396 394 394 q17 946 519 611 519 q18 7168641664156415 q19 4109143014761430 q20 538 281 278 278 q21 6097345035503450 q22 348 332 296 296 Total cold run time: 111515 ms Total hot run time: 38193 ms - Round 2, with runtime_filter_mode=off - q1 4111407240174017 q2 319 215 215 215 q3 2969283328142814 q4 1894157715271527 q5 5224523952165216 q6 193 113 115 113 q7 2201181318731813 q8 3148327132923271 q9 8500852785188518 q10 3693372836563656 q11 549 459 433 433 q12 741 557 578 557 q13 16916 287528312831 q14 274 243 250 243 q15 485 454 446 446 q16 457 415 401 401 q17 1727148614531453 q18 7442719470707070 q19 1598152514751475 q20 1884172817011701 q21 4974486547904790 q22 515 463 457 457 Total cold run time: 69814 ms Total hot run time: 53017 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](nereids)unify outputTupleDesc and projection [doris]
doris-robot commented on PR #32128: URL: https://github.com/apache/doris/pull/32128#issuecomment-2002031504 TeamCity be ut coverage result: Function Coverage: 34.93% (8571/24539) Line Coverage: 26.63% (69472/260926) Region Coverage: 25.92% (36089/139255) Branch Coverage: 22.85% (18424/80628) Coverage Report: http://coverage.selectdb-in.cc/coverage/bf9db3a2f2e9a2b223a167e80ac223edc58013a7_bf9db3a2f2e9a2b223a167e80ac223edc58013a7/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](like) opt like clause with concat(col, pattern_str) [doris]
yiguolei commented on code in PR #32333:
URL: https://github.com/apache/doris/pull/32333#discussion_r1527365618
##
regression-test/suites/correctness_p0/test_like_predicate_with_concat.groovy:
##
@@ -0,0 +1,90 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_like_predicate_with_concat") {
+sql """ DROP TABLE IF EXISTS `test_like_predicate_with_concat` """
+
+sql """
+CREATE TABLE IF NOT EXISTS `test_like_predicate_with_concat` (
+`id` int,
+`value_col` string,
+`pattern_col` string
+) ENGINE=OLAP
+DISTRIBUTED BY HASH(`id`) BUCKETS 4
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 1"
+);
+"""
+
+sql """
+INSERT INTO `test_like_predicate_with_concat` VALUES
+(0, 'prefix0_infix0_suffix0', 'prefix0'),
Review Comment:
The test is too simple. I think you should add some special chars such as $
^ % in the col value. And it could at the beginning of the value or ending of
the value or middle of the value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](nothing) Test run p0 [doris]
github-actions[bot] commented on PR #24563: URL: https://github.com/apache/doris/pull/24563#issuecomment-2002197544 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](docs) Change an error in the array_enumerate_uniq function documentation [doris]
github-actions[bot] commented on PR #24552: URL: https://github.com/apache/doris/pull/24552#issuecomment-2002197555 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 04/11: [Chore](FE)Remove unused components (#32295)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit bf820302707d10efa2d2b46cb376b49b05cb8592
Author: Calvin Kirs
AuthorDate: Fri Mar 15 21:01:30 2024 +0800
[Chore](FE)Remove unused components (#32295)
The tomcat-embed-el dependency is primarily used for standardizing EL
functionality, which we don't require in our application. Therefore, we can
safely remove it.
---
fe/pom.xml | 10 ++
1 file changed, 10 insertions(+)
diff --git a/fe/pom.xml b/fe/pom.xml
index e1d4f44d752..d96d4697880 100644
--- a/fe/pom.xml
+++ b/fe/pom.xml
@@ -467,6 +467,12 @@ under the License.
org.springframework.boot
spring-boot-starter-jetty
${spring.version}
+
+
+org.apache.tomcat
+tomcat-embed-el
+
+
org.springframework.boot
@@ -478,6 +484,10 @@ under the License.
spring-boot-starter-web
${spring.version}
+
+org.springframework.boot
+spring-boot-starter-tomcat
+
validator
hibernate-validator
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated (9ad196f1890 -> c34f5045c88)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from 9ad196f1890 Revert "[fix](cloud) ignore some case in cloud mode
(#32261)"
new a90a1a76f16 [bugfix](profile) support multi execution profile for
brokerload (#32280)
new f64a9a33f8e [fix](Nereids): don't pushdown project when project
contains both side of join (#32214)
new 844a1b53b74 [fix](retry) Set query encounter rpc exception default
retry times to 3 (#28555)
new bf820302707 [Chore](FE)Remove unused components (#32295)
new a15bf3057f4 [Fix](nereids) remove duplicate expr in grouping set
(#32290)
new 83ab61ad226 Add QUEUE_START_TIME/QUEUE_END_TIME/QUERY_STATUS column
for active_queries (#32259)
new c5ffeff8330 [fix](s3 client)add default ca cert list for s3 client to
avoid problem:'curlCode:77' (#32285)
new 5ceccb5ba5d [fix](compatibility) should enable windown funnel mode
from 2.0 (#32284)
new bf9332a2759 [docs](docs) Fix invalid link and typo of Master Branch
(#32213)
new 4bf202db042 [pipelineX](exchange) Make exchange buffer size
configurable (#32201)
new c34f5045c88 fix compile
The 11 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/agent/be_exec_version_manager.h | 3 +-
be/src/common/config.cpp | 7 ++
be/src/common/config.h | 4 +
.../schema_active_queries_scanner.cpp | 8 +-
be/src/pipeline/exec/exchange_sink_buffer.cpp | 6 +-
be/src/pipeline/exec/exchange_sink_buffer.h| 1 -
be/src/pipeline/exec/scan_operator.cpp | 1 +
be/src/util/s3_util.cpp| 23 ++
be/src/util/s3_util.h | 2 +
.../aggregate_function_window_funnel.h | 4 +-
.../main/java/org/apache/doris/common/Config.java | 2 +-
.../java/org/apache/doris/catalog/SchemaTable.java | 5 +-
.../doris/common/profile/AggregatedProfile.java| 41 --
.../org/apache/doris/common/profile/Profile.java | 40 ++
.../apache/doris/load/loadv2/LoadLoadingTask.java | 2 +-
.../nereids/rules/analysis/NormalizeRepeat.java| 11 +++
.../join/PushDownProjectThroughSemiJoin.java | 19 +
.../doris/nereids/trees/plans/algebra/Repeat.java | 3 -
.../trees/plans/commands/InsertExecutor.java | 2 +-
.../nereids/trees/plans/logical/LogicalRepeat.java | 4 +
.../main/java/org/apache/doris/qe/Coordinator.java | 4 +
.../java/org/apache/doris/qe/QeProcessorImpl.java | 29 ++-
.../org/apache/doris/qe/QueryStatisticsItem.java | 6 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 6 +-
.../doris/resource/workloadgroup/QueryQueue.java | 6 +-
.../doris/resource/workloadgroup/QueueToken.java | 33 +++-
.../doris/tablefunction/MetadataGenerator.java | 90 +-
.../join/PushDownProjectThroughSemiJoinTest.java | 32
fe/pom.xml | 10 +++
.../remove_duplicate_expr_in_grouping_set.out} | 21 +++--
.../remove_duplicate_expr_in_grouping_set.groovy} | 28 ---
.../schema_table/test_active_queries.groovy| 6 +-
32 files changed, 301 insertions(+), 158 deletions(-)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/profile/AggregatedProfile.java
copy regression-test/data/{load/insert/test_publish_one_succ.out =>
nereids_rules_p0/grouping_sets/remove_duplicate_expr_in_grouping_set.out} (52%)
copy
regression-test/suites/nereids_rules_p0/{cte/test_cte_reuse_with_window.groovy
=> grouping_sets/remove_duplicate_expr_in_grouping_set.groovy} (58%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 02/11: [fix](Nereids): don't pushdown project when project contains both side of join (#32214)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit f64a9a33f8e28d8f13af476cb8b508fd35067203
Author: 谢健
AuthorDate: Fri Mar 15 19:21:26 2024 +0800
[fix](Nereids): don't pushdown project when project contains both side of
join (#32214)
---
.../join/PushDownProjectThroughSemiJoin.java | 19 +
.../join/PushDownProjectThroughSemiJoinTest.java | 32 ++
2 files changed, 51 insertions(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoin.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoin.java
index 0629092f451..66735d39b98 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoin.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoin.java
@@ -38,6 +38,7 @@ import java.util.stream.Collectors;
/**
* Rule for pushdown project through left-semi/anti join
* Just push down project inside join to avoid to push the top of Join-Cluster.
+ * Note this rule is only used to push down project between join for join
ordering.
*
* Join Join
* ||
@@ -61,6 +62,9 @@ public class PushDownProjectThroughSemiJoin implements
ExplorationRuleFactory {
.whenNot(j -> j.left().child().hasDistributeHint())
.then(topJoin -> {
LogicalProject>
project = topJoin.left();
+if (projectBothJoinSide(project)) {
+return null;
+}
Plan newLeft = pushdownProject(project);
return topJoin.withChildren(newLeft, topJoin.right());
}).toRule(RuleType.PUSH_DOWN_PROJECT_THROUGH_SEMI_JOIN_LEFT),
@@ -72,12 +76,27 @@ public class PushDownProjectThroughSemiJoin implements
ExplorationRuleFactory {
.whenNot(j -> j.right().child().hasDistributeHint())
.then(topJoin -> {
LogicalProject>
project = topJoin.right();
+if (projectBothJoinSide(project)) {
+return null;
+}
Plan newRight = pushdownProject(project);
return topJoin.withChildren(topJoin.left(), newRight);
}).toRule(RuleType.PUSH_DOWN_PROJECT_THROUGH_SEMI_JOIN_RIGHT)
);
}
+private boolean projectBothJoinSide(LogicalProject> project) {
+// if project contains both side of join, it can't be pushed.
+// such as:
+// Project(l, null as r)
+// -- L(l) left anti join R(r)
+LogicalJoin join = project.child();
+Set projectOutput = project.getOutputSet();
+boolean containLeft =
join.left().getOutput().stream().anyMatch(projectOutput::contains);
+boolean containRight =
join.right().getOutput().stream().anyMatch(projectOutput::contains);
+return containRight && containLeft;
+}
+
private Plan pushdownProject(LogicalProject> project) {
LogicalJoin join = project.child();
Set conditionLeftSlots = CBOUtils.joinChildConditionSlots(join,
true);
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoinTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoinTest.java
index 8c5a29f0d55..bccd3056d35 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoinTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/PushDownProjectThroughSemiJoinTest.java
@@ -22,6 +22,7 @@ import org.apache.doris.nereids.trees.expressions.Add;
import org.apache.doris.nereids.trees.expressions.Alias;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
import org.apache.doris.nereids.trees.plans.JoinType;
import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan;
import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
@@ -133,4 +134,35 @@ class PushDownProjectThroughSemiJoinTest implements
MemoPatternMatchSupported {
)
);
}
+
+@Test
+void testProjectLiteral() {
+List projectExprs = ImmutableList.of(
+new Alias(new Add(scan1.getOutput().get(0), Literal.of(1)),
"alias"),
+new Alias(scan2.getOutput().get(0).getExprId(), new
NullLiteral(
(doris) 01/11: [bugfix](profile) support multi execution profile for brokerload (#32280)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit a90a1a76f168709d7695efaf1cbb4b49d2abc9ce
Author: yiguolei <676222...@qq.com>
AuthorDate: Fri Mar 15 17:41:47 2024 +0800
[bugfix](profile) support multi execution profile for brokerload (#32280)
The bug is introduced by #27184
Profile Format is :
Summary
MergedProfile
ExecutionProfile1
ExecutionProfile2
...
There maybe multiple execution profiles for broker load.
---
.../doris/common/profile/AggregatedProfile.java| 41 -
.../org/apache/doris/common/profile/Profile.java | 40 +++--
.../apache/doris/load/loadv2/LoadLoadingTask.java | 2 +-
.../trees/plans/commands/InsertExecutor.java | 2 +-
.../commands/insert/AbstractInsertExecutor.java| 194 +
.../java/org/apache/doris/qe/StmtExecutor.java | 6 +-
6 files changed, 222 insertions(+), 63 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/profile/AggregatedProfile.java
b/fe/fe-core/src/main/java/org/apache/doris/common/profile/AggregatedProfile.java
deleted file mode 100644
index f8481b21e65..000
---
a/fe/fe-core/src/main/java/org/apache/doris/common/profile/AggregatedProfile.java
+++ /dev/null
@@ -1,41 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-package org.apache.doris.common.profile;
-
-import org.apache.doris.common.util.RuntimeProfile;
-
-import java.util.Map;
-
-/**
-* AggregatedProfile is part of a query profile.
-* It contains the aggregated information of a query.
-*/
-public class AggregatedProfile {
-
-public static final String PROFILE_NAME = "MergedProfile";
-private ExecutionProfile executionProfile;
-
-public AggregatedProfile(RuntimeProfile rootProfile, ExecutionProfile
executionProfile) {
-this.executionProfile = executionProfile;
-}
-
-public RuntimeProfile getAggregatedFragmentsProfile(Map
planNodeMap) {
-return executionProfile.getAggregatedFragmentsProfile(planNodeMap);
-}
-
-}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/profile/Profile.java
b/fe/fe-core/src/main/java/org/apache/doris/common/profile/Profile.java
index 19a51a1aaa6..5f3ed601630 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/profile/Profile.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/profile/Profile.java
@@ -21,11 +21,13 @@ import org.apache.doris.common.util.ProfileManager;
import org.apache.doris.common.util.RuntimeProfile;
import org.apache.doris.planner.Planner;
+import com.google.common.collect.Lists;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import java.util.List;
import java.util.Map;
/**
@@ -33,20 +35,23 @@ import java.util.Map;
* following structure: root profile: // summary of this profile, such as start
* time, end time, query id, etc. [SummaryProfile] // each execution profile is
* a complete execution of a query, a job may contain multiple queries.
- * [List]
+ * [List].
+ * There maybe multi execution profiles for one job, for example broker load
job.
+ * It will create one execution profile for every single load task.
*
* SummaryProfile: Summary: Execution Summary:
*
*
- * ExecutionProfile: Fragment 0: Fragment 1: ...
+ * ExecutionProfile1: Fragment 0: Fragment 1: ...
+ * ExecutionProfile2: Fragment 0: Fragment 1: ...
+ *
*/
public class Profile {
private static final Logger LOG = LogManager.getLogger(Profile.class);
private static final int MergedProfileLevel = 1;
private RuntimeProfile rootProfile;
private SummaryProfile summaryProfile;
-private AggregatedProfile aggregatedProfile;
-private ExecutionProfile executionProfile;
+private List executionProfiles = Lists.newArrayList();
private boolean isFinished;
private Map planNodeMap;
@@ -59,14 +64,13 @@ public class Profile {
this.isFinished = !isEnable;
}
-public void setExecutionProfile(ExecutionProfile executionProfile) {
+publi
(doris) 05/11: [Fix](nereids) remove duplicate expr in grouping set (#32290)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit a15bf3057f446aea49be8f1570b8cb593f56c5b7
Author: feiniaofeiafei <53502832+feiniaofeia...@users.noreply.github.com>
AuthorDate: Fri Mar 15 21:36:16 2024 +0800
[Fix](nereids) remove duplicate expr in grouping set (#32290)
db reported a error " expression duplicate in grouping set" when there are
duplicate expression in grouping set.
e.g.select a from mal_test1 group by grouping sets((a,a))
This pr removes duplicate expr in grouping set :
select a from mal_test1 group by grouping sets((a))
---
.../nereids/rules/analysis/NormalizeRepeat.java| 11 +++
.../doris/nereids/trees/plans/algebra/Repeat.java | 3 --
.../nereids/trees/plans/logical/LogicalRepeat.java | 4 +++
.../remove_duplicate_expr_in_grouping_set.out | 19 +++
.../remove_duplicate_expr_in_grouping_set.groovy | 37 ++
5 files changed, 71 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeRepeat.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeRepeat.java
index 9451cc40bc2..e90528a9d4a 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeRepeat.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/NormalizeRepeat.java
@@ -84,6 +84,7 @@ public class NormalizeRepeat extends OneAnalysisRuleFactory {
return RuleType.NORMALIZE_REPEAT.build(
logicalRepeat(any()).when(LogicalRepeat::canBindVirtualSlot).then(repeat -> {
checkRepeatLegality(repeat);
+repeat = removeDuplicateColumns(repeat);
// add virtual slot, LogicalAggregate and LogicalProject for
normalize
LogicalAggregate agg = normalizeRepeat(repeat);
return dealSlotAppearBothInAggFuncAndGroupingSets(agg);
@@ -91,6 +92,16 @@ public class NormalizeRepeat extends OneAnalysisRuleFactory {
);
}
+private LogicalRepeat removeDuplicateColumns(LogicalRepeat
repeat) {
+List> groupingSets = repeat.getGroupingSets();
+ImmutableList.Builder> builder =
ImmutableList.builder();
+for (List sets : groupingSets) {
+List newList =
ImmutableList.copyOf(ImmutableSet.copyOf(sets));
+builder.add(newList);
+}
+return repeat.withGroupSets(builder.build());
+}
+
private void checkRepeatLegality(LogicalRepeat repeat) {
checkGroupingSetsSize(repeat);
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/algebra/Repeat.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/algebra/Repeat.java
index 29258b6bbab..e729f2a7cb3 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/algebra/Repeat.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/algebra/Repeat.java
@@ -175,9 +175,6 @@ public interface Repeat extends
Aggregate {
if (index == null) {
throw new AnalysisException("Can not find grouping set
expression in output: " + expression);
}
-if (groupingSetIndex.contains(index)) {
-throw new AnalysisException("expression duplicate in
grouping set: " + expression);
-}
groupingSetIndex.add(index);
}
groupingSetsIndex.add(groupingSetIndex);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalRepeat.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalRepeat.java
index 1c8a468b44b..5e0311afe1b 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalRepeat.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalRepeat.java
@@ -158,6 +158,10 @@ public class LogicalRepeat
extends LogicalUnary withGroupSets(List>
groupingSets) {
+return new LogicalRepeat<>(groupingSets, outputExpressions, child());
+}
+
public LogicalRepeat
withGroupSetsAndOutput(List> groupingSets,
List outputExpressionList) {
return new LogicalRepeat<>(groupingSets, outputExpressionList,
child());
diff --git
a/regression-test/data/nereids_rules_p0/grouping_sets/remove_duplicate_expr_in_grouping_set.out
b/regression-test/data/nereids_rules_p0/grouping_sets/remove_duplicate_expr_in_grouping_set.out
new file mode 100644
index 000..5956255d7bd
--- /dev/null
+++
b/regression-test/data/nereids_rules_p0/grouping_sets/remove_duplicate_expr_in_grouping_set.out
@@ -0,0 +1,19 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !test_col --
+\N 6
+1 20
+2 8
+3 20
+4
(doris) 09/11: [docs](docs) Fix invalid link and typo of Master Branch (#32213)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git commit bf9332a275975cc6d9f386c6a3ab4716c682 Author: KassieZ <139741991+kass...@users.noreply.github.com> AuthorDate: Sat Mar 16 09:03:35 2024 +0800 [docs](docs) Fix invalid link and typo of Master Branch (#32213) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 07/11: [fix](s3 client)add default ca cert list for s3 client to avoid problem:'curlCode:77' (#32285)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit c5ffeff833060add948eb98cea7fa538b9fcd918
Author: ryanzryu <143597717+ryanz...@users.noreply.github.com>
AuthorDate: Fri Mar 15 22:19:58 2024 +0800
[fix](s3 client)add default ca cert list for s3 client to avoid
problem:'curlCode:77' (#32285)
Co-authored-by: ryanzryu
---
be/src/common/config.cpp | 6 ++
be/src/common/config.h | 3 +++
be/src/util/s3_util.cpp | 23 +++
be/src/util/s3_util.h| 2 ++
4 files changed, 34 insertions(+)
diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp
index 3d44528ea36..63ed49eca24 100644
--- a/be/src/common/config.cpp
+++ b/be/src/common/config.cpp
@@ -1190,6 +1190,12 @@ DEFINE_mBool(check_segment_when_build_rowset_meta,
"false");
DEFINE_mInt32(max_s3_client_retry, "10");
+// ca_cert_file is in this path by default, Normally no modification is
required
+// ca cert default path is different from different OS
+DEFINE_mString(ca_cert_file_paths,
+
"/etc/pki/tls/certs/ca-bundle.crt;/etc/ssl/certs/ca-certificates.crt;"
+ "/etc/ssl/ca-bundle.pem");
+
// clang-format off
#ifdef BE_TEST
// test s3
diff --git a/be/src/common/config.h b/be/src/common/config.h
index a5219c42590..a0a20450d5d 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -1267,6 +1267,9 @@ DECLARE_mInt32(max_s3_client_retry);
// write as inverted index tmp directory
DECLARE_String(tmp_file_dir);
+// the file paths(one or more) of CA cert, splite using ";" aws s3 lib use it
to init s3client
+DECLARE_mString(ca_cert_file_paths);
+
#ifdef BE_TEST
// test s3
DECLARE_String(test_s3_resource);
diff --git a/be/src/util/s3_util.cpp b/be/src/util/s3_util.cpp
index d09b808868b..063cc16c67b 100644
--- a/be/src/util/s3_util.cpp
+++ b/be/src/util/s3_util.cpp
@@ -30,6 +30,7 @@
#include
#include
+#include
#include
#include
#include
@@ -114,6 +115,18 @@ S3ClientFactory::S3ClientFactory() {
return std::make_shared(logLevel);
};
Aws::InitAPI(_aws_options);
+_ca_cert_file_path = get_valid_ca_cert_path();
+}
+
+string S3ClientFactory::get_valid_ca_cert_path() {
+vector vec_ca_file_path =
doris::split(config::ca_cert_file_paths, ";");
+vector::iterator it = vec_ca_file_path.begin();
+for (; it != vec_ca_file_path.end(); ++it) {
+if (std::filesystem::exists(*it)) {
+return *it;
+}
+}
+return "";
}
S3ClientFactory::~S3ClientFactory() {
@@ -157,6 +170,16 @@ std::shared_ptr
S3ClientFactory::create(const S3Conf& s3_conf
Aws::Client::ClientConfiguration aws_config =
S3ClientFactory::getClientConfiguration();
aws_config.endpointOverride = s3_conf.endpoint;
aws_config.region = s3_conf.region;
+std::string ca_cert = get_valid_ca_cert_path();
+if ("" != _ca_cert_file_path) {
+aws_config.caFile = _ca_cert_file_path;
+} else {
+// config::ca_cert_file_paths is valmutable,get newest value if file
path invaild
+_ca_cert_file_path = get_valid_ca_cert_path();
+if ("" != _ca_cert_file_path) {
+aws_config.caFile = _ca_cert_file_path;
+}
+}
if (s3_conf.max_connections > 0) {
aws_config.maxConnections = s3_conf.max_connections;
} else {
diff --git a/be/src/util/s3_util.h b/be/src/util/s3_util.h
index a47b5620548..74af0121d20 100644
--- a/be/src/util/s3_util.h
+++ b/be/src/util/s3_util.h
@@ -133,10 +133,12 @@ public:
private:
S3ClientFactory();
+static std::string get_valid_ca_cert_path();
Aws::SDKOptions _aws_options;
std::mutex _lock;
std::unordered_map> _cache;
+std::string _ca_cert_file_path;
};
} // end namespace doris
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 03/11: [fix](retry) Set query encounter rpc exception default retry times to 3 (#28555)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 844a1b53b74bebd3a2dc2384071e778ef7a2c793
Author: deardeng <565620...@qq.com>
AuthorDate: Fri Mar 15 19:38:38 2024 +0800
[fix](retry) Set query encounter rpc exception default retry times to 3
(#28555)
---
fe/fe-common/src/main/java/org/apache/doris/common/Config.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 4dd10de1cea..2c5a36e9f85 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -911,7 +911,7 @@ public class Config extends ConfigBase {
* You may reduce this number to avoid Avalanche disaster.
*/
@ConfField(mutable = true)
-public static int max_query_retry_time = 1;
+public static int max_query_retry_time = 3;
/**
* The number of point query retries in executor.
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 08/11: [fix](compatibility) should enable windown funnel mode from 2.0 (#32284)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 5ceccb5ba5dc01856ae6790d9f5762d40679fefe
Author: Jerry Hu
AuthorDate: Sat Mar 16 08:54:33 2024 +0800
[fix](compatibility) should enable windown funnel mode from 2.0 (#32284)
---
be/src/agent/be_exec_version_manager.h| 3 ++-
be/src/vec/aggregate_functions/aggregate_function_window_funnel.h | 4 ++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/be/src/agent/be_exec_version_manager.h
b/be/src/agent/be_exec_version_manager.h
index 0290e3bee32..470d78ee981 100644
--- a/be/src/agent/be_exec_version_manager.h
+++ b/be/src/agent/be_exec_version_manager.h
@@ -60,8 +60,9 @@ private:
*e. add repeat_max_num in repeat function
* 3: start from doris 2.0 (by some mistakes)
*a. aggregation function do not serialize bitmap to string.
+ *b. support window funnel mode.
* 4: start from doris 2.1
- *a. support window funnel mode from 2.0
+ *a. ignore this line, window funnel mode should be enabled from 2.0.
*b. array contains/position/countequal function return nullable in less
situations.
*c. cleared old version of Version 2.
*d. unix_timestamp function support timestamp with float for datetimev2,
and change nullable mode.
diff --git a/be/src/vec/aggregate_functions/aggregate_function_window_funnel.h
b/be/src/vec/aggregate_functions/aggregate_function_window_funnel.h
index ed4c9aed059..9356cfd4b68 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_window_funnel.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_window_funnel.h
@@ -270,8 +270,8 @@ public:
void create(AggregateDataPtr __restrict place) const override {
auto data = new (place) WindowFunnelState();
-/// support window funnel mode from 2.1. See
`BeExecVersionManager::max_be_exec_version`
-data->enable_mode = version >= USE_NEW_SERDE;
+/// support window funnel mode from 2.0. See
`BeExecVersionManager::max_be_exec_version`
+data->enable_mode = version >= 3;
}
String get_name() const override { return "window_funnel"; }
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
