Re: [PR] [improve](nested_types)support nested type with agg replace_if_not_null [doris]
xiaokang merged PR #38304: URL: https://github.com/apache/doris/pull/38304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
amorynan commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249602266 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](bash) Make sure that the `prof_prefix` defined in be.conf is not overwritten in start_be.sh [doris]
doris-robot commented on PR #38354: URL: https://github.com/apache/doris/pull/38354#issuecomment-2249602257 TPC-H: Total hot run time: 39590 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ba33fa9344811d2e9f805f5d8970ab7abefa517f, data reload: false -- Round 1 -- q1 17701 434542444244 q2 2014191 181 181 q3 10463 113510371037 q4 10188 786 799 786 q5 7541269926862686 q6 219 136 136 136 q7 954 593 597 593 q8 9218205820862058 q9 8875651165576511 q10 8730378437393739 q11 463 231 233 231 q12 462 226 225 225 q13 19158 301129822982 q14 280 241 233 233 q15 513 475 484 475 q16 492 379 381 379 q17 962 596 655 596 q18 7973745973167316 q19 8340149114291429 q20 724 319 325 319 q21 4869315640433156 q22 353 278 288 278 Total cold run time: 120492 ms Total hot run time: 39590 ms - Round 2, with runtime_filter_mode=off - q1 4406422042134213 q2 389 270 258 258 q3 3180296829352935 q4 2036170117171701 q5 5582549055345490 q6 231 131 132 131 q7 2238185217811781 q8 3304339134223391 q9 8797894487578757 q10 4049382838573828 q11 606 499 487 487 q12 813 633 644 633 q13 17136 317531723172 q14 324 285 302 285 q15 539 482 489 482 q16 506 444 437 437 q17 1790152115161516 q18 8109796679047904 q19 1734144615161446 q20 2170187018791870 q21 7877481449364814 q22 586 524 506 506 Total cold run time: 76402 ms Total hot run time: 56037 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix unstable refresh catalog/db ut [doris]
github-actions[bot] commented on PR #38352: URL: https://github.com/apache/doris/pull/38352#issuecomment-2249605874 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix unstable refresh catalog/db ut [doris]
github-actions[bot] commented on PR #38352: URL: https://github.com/apache/doris/pull/38352#issuecomment-2249605928 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature][external catalog/lakesoul] LakeSoul Catalog support Filter Pushdown & Cdc data handling & S3 data access [doris]
doris-robot commented on PR #37979: URL: https://github.com/apache/doris/pull/37979#issuecomment-2249605561 TPC-DS: Total hot run time: 174357 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2360a214572d1e6ddf4e2ba28b1868d1f32eaf7a, data reload: false query1 908 383 364 364 query2 6432196519601960 query3 6647204 217 204 query4 28431 17398 17458 17398 query5 3755480 471 471 query6 263 180 150 150 query7 4584289 278 278 query8 244 190 191 190 query9 8428241223852385 query10 430 293 274 274 query11 12717 10163 10125 10125 query12 120 86 85 85 query13 1649378 369 369 query14 10507 770182927701 query15 232 166 170 166 query16 7601465 444 444 query17 1599588 542 542 query18 1812290 283 283 query19 210 158 155 155 query20 90 86 83 83 query21 217 134 133 133 query22 4256409139543954 query23 34599 33671 33496 33496 query24 10937 297929102910 query25 618 407 385 385 query26 701 153 152 152 query27 2271276 276 276 query28 6166208820612061 query29 922 662 638 638 query30 260 162 154 154 query31 987 774 772 772 query32 97 53 63 53 query33 715 342 333 333 query34 925 520 513 513 query35 885 781 754 754 query36 1152959 929 929 query37 138 85 86 85 query38 2925276827612761 query39 871 811 788 788 query40 201 119 121 119 query41 50 42 43 42 query42 117 103 105 103 query43 525 488 492 488 query44 1193745 748 745 query45 208 174 174 174 query46 1087729 742 729 query47 1852176717791767 query48 360 294 287 287 query49 831 406 411 406 query50 788 386 406 386 query51 6797668667096686 query52 102 92 91 91 query53 369 294 301 294 query54 916 447 448 447 query55 78 73 75 73 query56 281 263 263 263 query57 1187104611021046 query58 255 251 256 251 query59 3051270327502703 query60 315 282 283 282 query61 97 94 94 94 query62 773 681 631 631 query63 320 285 295 285 query64 9183224074042240 query65 3182308831103088 query66 696 333 324 324 query67 15507 15016 14882 14882 query68 4705558 595 558 query69 633 450 355 355 query70 1191109210291029 query71 420 282 286 282 query72 8610521756435217 query73 761 327 329 327 query74 6070568757175687 query75 3474268926892689 query76 3439983 910 910 query77 635 326 326 326 query78 11577 949389598959 query79 6444533 520 520 query80 1541476 473 473 query81 610 229 219 219 query82 816 142 136 136 query83 336 167 168 167 query84 276 85 89 85 query85 1358314 343 314 query86 458 310 292 292 query87 3310313231273127 query88 4600237323722372 query89 498 393 386 386 query90 1942198 192 192 query91 136 99 101 99 query92 59 49 50 49 query93 4820521 515 515 query94 1309305 295 295 query95 410 329 318 318 query96 611 280 278 278 query97 3211302630273026 query98 211 211 195 195 query99 1590126012691260 Total cold run time: 292683 ms Total hot run time: 174357 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
SWJTU-ZhangLei commented on PR #38243: URL: https://github.com/apache/doris/pull/38243#issuecomment-2249608295 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature][external catalog/lakesoul] LakeSoul Catalog support Filter Pushdown & Cdc data handling & S3 data access [doris]
doris-robot commented on PR #37979: URL: https://github.com/apache/doris/pull/37979#issuecomment-2249614294 ClickBench: Total hot run time: 31.01 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 2360a214572d1e6ddf4e2ba28b1868d1f32eaf7a, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.230.050.06 query4 1.680.080.08 query5 0.490.490.48 query6 1.130.730.73 query7 0.020.010.01 query8 0.060.050.05 query9 0.550.480.49 query10 0.540.540.53 query11 0.150.120.11 query12 0.150.120.12 query13 0.590.580.58 query14 0.750.790.77 query15 0.870.820.83 query16 0.360.360.40 query17 0.991.000.99 query18 0.230.220.21 query19 1.911.781.75 query20 0.010.020.01 query21 15.54 0.830.67 query22 4.786.072.13 query23 18.28 1.371.33 query24 2.180.220.23 query25 0.140.090.08 query26 0.300.210.20 query27 0.460.230.23 query28 13.24 1.021.00 query29 12.64 3.303.34 query30 0.240.060.05 query31 2.890.410.40 query32 3.230.480.47 query33 2.852.952.93 query34 16.87 4.304.32 query35 4.424.444.49 query36 0.650.490.46 query37 0.190.160.16 query38 0.160.150.15 query39 0.050.040.03 query40 0.150.120.12 query41 0.100.050.05 query42 0.060.050.05 query43 0.050.050.04 Total cold run time: 110.3 s Total hot run time: 31.01 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
github-actions[bot] commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249611394 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
github-actions[bot] commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249612844 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](bash) Make sure that the `prof_prefix` defined in be.conf is not overwritten in start_be.sh [doris]
doris-robot commented on PR #38354: URL: https://github.com/apache/doris/pull/38354#issuecomment-2249619865 TPC-DS: Total hot run time: 173725 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit ba33fa9344811d2e9f805f5d8970ab7abefa517f, data reload: false query1 911 380 365 365 query2 6425198819181918 query3 6646207 215 207 query4 28367 17504 17620 17504 query5 3597464 474 464 query6 257 176 171 171 query7 4583285 291 285 query8 235 190 201 190 query9 8557237923842379 query10 446 288 279 279 query11 11777 10065 10173 10065 query12 117 84 94 84 query13 1641372 379 372 query14 10157 759970907090 query15 223 171 171 171 query16 7771471 464 464 query17 1607591 544 544 query18 1955286 280 280 query19 199 156 159 156 query20 95 84 81 81 query21 214 123 126 123 query22 4507412339703970 query23 34339 33972 33709 33709 query24 10640 290029112900 query25 594 387 389 387 query26 707 155 149 149 query27 2233274 289 274 query28 6018208120572057 query29 906 653 649 649 query30 261 155 157 155 query31 965 801 759 759 query32 96 52 53 52 query33 642 334 332 332 query34 900 487 510 487 query35 918 746 741 741 query36 1126990 988 988 query37 145 85 83 83 query38 2918290328262826 query39 887 894 871 871 query40 206 128 121 121 query41 49 45 46 45 query42 114 99 103 99 query43 530 489 498 489 query44 1092724 726 724 query45 207 174 176 174 query46 1097727 714 714 query47 1852178518041785 query48 361 293 284 284 query49 843 400 405 400 query50 786 386 384 384 query51 6874671367206713 query52 100 94 89 89 query53 355 308 283 283 query54 911 448 448 448 query55 73 73 72 72 query56 282 280 269 269 query57 1153104310391039 query58 247 252 267 252 query59 2979281626892689 query60 313 265 277 265 query61 95 97 95 95 query62 826 626 633 626 query63 321 290 286 286 query64 9297220816831683 query65 3180310331413103 query66 750 329 330 329 query67 15400 14826 14865 14826 query68 7033545 543 543 query69 685 463 371 371 query70 1194114110731073 query71 488 278 272 272 query72 6951537357635373 query73 812 323 326 323 query74 6265569056815681 query75 4001270627132706 query76 4119929 1004929 query77 702 316 319 316 query78 9806899598238995 query79 5042527 527 527 query80 1109481 476 476 query81 571 221 229 221 query82 792 139 135 135 query83 208 178 175 175 query84 272 91 85 85 query85 1287315 302 302 query86 347 311 318 311 query87 3245309430983094 query88 4685236323682363 query89 498 383 378 378 query90 1493191 190 190 query91 126 163 100 100 query92 59 49 48 48 query93 5312521 503 503 query94 591 294 285 285 query95 397 315 324 315 query96 606 277 275 275 query97 3167309730263026 query98 231 198 200 198 query99 1556127012691269 Total cold run time: 288525 ms Total hot run time: 173725 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled [doris]
eldenmoon merged PR #37900: URL: https://github.com/apache/doris/pull/37900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled (#37900)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 1b3d1678c23 [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled (#37900) 1b3d1678c23 is described below commit 1b3d1678c23af03fcdd5ee78614a4bffbeec2dad Author: lihangyu <15605149...@163.com> AuthorDate: Thu Jul 25 15:14:10 2024 +0800 [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled (#37900) …e handled --- .../main/java/org/apache/doris/qe/PointQueryExecutor.java | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java b/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java index 2ce047c9ddc..5aa07246570 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java +++ b/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java @@ -47,6 +47,7 @@ import org.apache.doris.thrift.TStatusCode; import com.google.common.base.Preconditions; import com.google.common.base.Strings; import com.google.common.collect.Lists; +import com.google.common.collect.Maps; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.apache.thrift.TDeserializer; @@ -57,6 +58,7 @@ import java.util.Collections; import java.util.HashSet; import java.util.Iterator; import java.util.List; +import java.util.Map; import java.util.Set; import java.util.concurrent.ExecutionException; import java.util.concurrent.Future; @@ -177,16 +179,18 @@ public class PointQueryExecutor implements CoordInterface { void addKeyTuples( InternalService.PTabletKeyLookupRequest.Builder requestBuilder) { // TODO handle IN predicates +Map columnExpr = Maps.newHashMap(); KeyTuple.Builder kBuilder = KeyTuple.newBuilder(); for (Expr expr : shortCircuitQueryContext.scanNode.getConjuncts()) { BinaryPredicate predicate = (BinaryPredicate) expr; Expr left = predicate.getChild(0); Expr right = predicate.getChild(1); -// ignore delete sign conjuncts only collect key conjuncts -if (left instanceof SlotRef && ((SlotRef) left).getColumnName().equalsIgnoreCase(Column.DELETE_SIGN)) { -continue; -} -kBuilder.addKeyColumnRep(right.getStringValue()); +SlotRef columnSlot = left.unwrapSlotRef(); +columnExpr.put(columnSlot.getColumnName(), right); +} +// add key tuple in keys order +for (Column column : shortCircuitQueryContext.scanNode.getOlapTable().getBaseSchemaKeyColumns()) { + kBuilder.addKeyColumnRep(columnExpr.get(column.getName()).getStringValue()); } requestBuilder.addKeyTuples(kBuilder); } - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled [doris]
github-actions[bot] commented on PR #37900: URL: https://github.com/apache/doris/pull/37900#issuecomment-2249621197 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft] [doris]
doris-robot commented on PR #38305: URL: https://github.com/apache/doris/pull/38305#issuecomment-2249623305 TPC-H: Total hot run time: 39794 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0f6ea131249f387b333078f31d062d5c6a48c642, data reload: false -- Round 1 -- q1 17630 433043014301 q2 2005185 183 183 q3 10473 121210921092 q4 10181 830 769 769 q5 7601267927052679 q6 225 138 136 136 q7 948 591 589 589 q8 9213206120932061 q9 8933657065596559 q10 8926378537053705 q11 451 239 235 235 q12 451 233 222 222 q13 17904 297529612961 q14 277 236 243 236 q15 526 478 482 478 q16 503 397 383 383 q17 969 671 680 671 q18 8077753173197319 q19 8204143114261426 q20 706 326 319 319 q21 4850319232863192 q22 344 278 281 278 Total cold run time: 119397 ms Total hot run time: 39794 ms - Round 2, with runtime_filter_mode=off - q1 4387425742604257 q2 382 275 254 254 q3 3054292430142924 q4 1944170917291709 q5 5629556454735473 q6 219 131 138 131 q7 2282183518511835 q8 3249341934043404 q9 8817883688438836 q10 4158368838733688 q11 603 524 495 495 q12 831 643 633 633 q13 17223 314531833145 q14 310 285 282 282 q15 527 474 511 474 q16 497 440 428 428 q17 1805151815311518 q18 8190796678917891 q19 1761149214461446 q20 2934191618791879 q21 5155469348454693 q22 574 519 509 509 Total cold run time: 74531 ms Total hot run time: 55904 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compaction) fix missing flush column during compaction [doris]
doris-robot commented on PR #38356: URL: https://github.com/apache/doris/pull/38356#issuecomment-2249623339 TPC-H: Total hot run time: 39630 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 903c6282d8858f886bc39855c3fa6b8447ef3474, data reload: false -- Round 1 -- q1 17619 429142684268 q2 2029192 185 185 q3 10468 114711191119 q4 10188 812 699 699 q5 7517270426442644 q6 218 136 141 136 q7 943 596 594 594 q8 9231203320652033 q9 8756652265316522 q10 8810375637583756 q11 468 232 240 232 q12 461 218 219 218 q13 18747 298329712971 q14 283 233 245 233 q15 516 468 470 468 q16 480 377 376 376 q17 961 661 783 661 q18 8002747473747374 q19 7935137413881374 q20 702 322 326 322 q21 4883317332633173 q22 335 278 272 272 Total cold run time: 119552 ms Total hot run time: 39630 ms - Round 2, with runtime_filter_mode=off - q1 4441425842064206 q2 366 283 269 269 q3 3075291629452916 q4 1977175418201754 q5 5608555954695469 q6 222 135 140 135 q7 2256191318091809 q8 3230345333993399 q9 8741886188448844 q10 4177374337853743 q11 595 499 501 499 q12 812 641 642 641 q13 15907 317132313171 q14 318 284 289 284 q15 534 498 482 482 q16 525 433 446 433 q17 1817154314961496 q18 8200811378357835 q19 1729139814641398 q20 2273187519011875 q21 10605 490946154615 q22 606 506 503 503 Total cold run time: 78014 ms Total hot run time: 55776 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
amorynan commented on PR #38150: URL: https://github.com/apache/doris/pull/38150#issuecomment-2249624954 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](bash) Make sure that the `prof_prefix` defined in `JEMALLOC_CONF` is not overwritten in start_be.sh [doris]
doris-robot commented on PR #38354: URL: https://github.com/apache/doris/pull/38354#issuecomment-2249628693 ClickBench: Total hot run time: 30.18 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit ba33fa9344811d2e9f805f5d8970ab7abefa517f, data reload: false query1 0.040.040.04 query2 0.070.040.03 query3 0.220.050.05 query4 1.670.070.07 query5 0.490.490.48 query6 1.120.720.73 query7 0.020.010.02 query8 0.050.040.05 query9 0.550.480.49 query10 0.550.540.54 query11 0.150.110.12 query12 0.150.120.12 query13 0.580.580.58 query14 0.760.790.81 query15 0.840.810.82 query16 0.370.370.36 query17 0.951.050.97 query18 0.220.210.22 query19 1.931.801.77 query20 0.010.010.01 query21 15.40 0.800.66 query22 3.967.311.44 query23 18.29 1.461.28 query24 2.140.220.22 query25 0.160.090.08 query26 0.300.210.21 query27 0.450.230.23 query28 13.31 1.021.00 query29 12.60 3.323.28 query30 0.270.060.06 query31 2.860.400.40 query32 3.270.490.47 query33 2.852.882.91 query34 17.15 4.334.33 query35 4.474.424.40 query36 0.650.460.46 query37 0.180.150.15 query38 0.160.150.15 query39 0.050.030.04 query40 0.160.110.12 query41 0.100.060.06 query42 0.060.050.05 query43 0.040.050.04 Total cold run time: 109.62 s Total hot run time: 30.18 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix unstable refresh catalog/db ut [doris]
morningman merged PR #38352: URL: https://github.com/apache/doris/pull/38352 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](test) fix unstable refresh catalog/db ut (#38352)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 18e6af5ed7f [fix](test) fix unstable refresh catalog/db ut (#38352) 18e6af5ed7f is described below commit 18e6af5ed7f094b6e10b9ee710c184900c22e334 Author: Mingyu Chen AuthorDate: Thu Jul 25 15:19:44 2024 +0800 [fix](test) fix unstable refresh catalog/db ut (#38352) Followup #38244 fix some unstable fe ut after set `use_meta_cache` as true --- .../src/test/java/org/apache/doris/catalog/RefreshDbTest.java | 3 +++ .../test/java/org/apache/doris/datasource/RefreshCatalogTest.java | 7 +-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java b/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java index fd920e9632c..fd7a6b1a24b 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java @@ -91,6 +91,9 @@ public class RefreshDbTest extends TestWithFeService { } long l3 = db1.getLastUpdateTime(); Assertions.assertTrue(l3 == l2); +// when use_meta_cache is true, the table will be recreated after refresh. +// so we need to get table again +table = db1.getTable("tbl11").get(); Assertions.assertFalse(table.isObjectCreated()); test1.getDbNullable("db1").getTables(); Assertions.assertFalse(table.isObjectCreated()); diff --git a/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java b/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java index 87906261b24..34177f49382 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java @@ -145,9 +145,12 @@ public class RefreshCatalogTest extends TestWithFeService { // not triggered init method long l3 = test2.getLastUpdateTime(); Assertions.assertTrue(l3 == l2); -// Assertions.assertFalse(table.isObjectCreated()); +// when use_meta_cache is true, the table will be recreated after refresh. +// so we need to get table again +table = (TestExternalTable) test2.getDbNullable("db1").getTable("tbl11").get(); +Assertions.assertFalse(table.isObjectCreated()); test2.getDbNullable("db1").getTables(); -// Assertions.assertFalse(table.isObjectCreated()); +Assertions.assertFalse(table.isObjectCreated()); try { DdlExecutor.execute(Env.getCurrentEnv(), refreshCatalogStmt); } catch (Exception e) { - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
doris-robot commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249631012 TeamCity be ut coverage result: Function Coverage: 37.81% (8117/21466) Line Coverage: 29.48% (66546/225741) Region Coverage: 28.95% (34299/118466) Branch Coverage: 24.83% (17622/70960) Coverage Report: http://coverage.selectdb-in.cc/coverage/dcf18ba45f3f802fc50dba752dffe83fb6227428_dcf18ba45f3f802fc50dba752dffe83fb6227428/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](JsonReader) support sepecify `$.` as root column in json path [doris]
xiaokang commented on code in PR #38213: URL: https://github.com/apache/doris/pull/38213#discussion_r1690951663 ## be/src/vec/exec/format/json/new_json_reader.cpp: ## @@ -1657,7 +1657,19 @@ Status NewJsonReader::_simdjson_write_columns_by_jsonpath( return st; } } -if (i >= _parsed_jsonpaths.size() || st.is()) { +if (i < _parsed_jsonpaths.size() && JsonFunctions::is_root_path(_parsed_jsonpaths[i])) { +// Indicate that the jsonpath is "$.", read the full root json object, insert the original doc directly Review Comment: Does this feature, that load all fields in JSON to variant column, match the need for users? I think usually users only need some fields in JSON, not all fields. ## regression-test/data/load_p0/stream_load/test_json_load.out: ## @@ -250,3 +250,9 @@ testk2_value -- !select29 -- 10 \N + +-- !select30 -- +12345 {"k1":12345,"k2":"1","k3":11,"k4":[1]} {"k1":12345,"k2":"1","k3":11,"k4":[1]} 11 +12346 {"k1":12346,"k2":"2","k4":[2]} {"k1":12346,"k2":"2","k4":[2]} \N +12347 {"k1":12347,"k3":"3","k5":{"k51":1024,"":[1]}} {"k1":12347,"k3":"3","k5":{"k51":1024,"":[1]}} 3 +12347 {"k1":12347,"k3":"3","k4":[2]} {"k1":12347,"k3":"3","k4":[2]} 3 Review Comment: unstable output since k1 is the same for row 3 and 4. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) ensure error log upload to S3 before returning load result [doris]
sollhui closed pull request #38353: [fix](cloud) ensure error log upload to S3 before returning load result URL: https://github.com/apache/doris/pull/38353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
github-actions[bot] commented on PR #38150: URL: https://github.com/apache/doris/pull/38150#issuecomment-2249635976 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](JsonReader) support sepecify `$.` as root column in json path [doris]
eldenmoon commented on code in PR #38213: URL: https://github.com/apache/doris/pull/38213#discussion_r1690956423 ## be/src/vec/exec/format/json/new_json_reader.cpp: ## @@ -1657,7 +1657,19 @@ Status NewJsonReader::_simdjson_write_columns_by_jsonpath( return st; } } -if (i >= _parsed_jsonpaths.size() || st.is()) { +if (i < _parsed_jsonpaths.size() && JsonFunctions::is_root_path(_parsed_jsonpaths[i])) { +// Indicate that the jsonpath is "$.", read the full root json object, insert the original doc directly Review Comment: some user need this feature, map all columns to a variant column -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
doris-robot commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249639340 TPC-H: Total hot run time: 49757 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, data reload: false -- Round 1 -- q1 17667 438343064306 q2 2073155 144 144 q3 10271 191121811911 q4 10126 126113131261 q5 8487395839393939 q6 232 146 123 123 q7 2051158315931583 q8 9311275827042704 q9 10582 10484 10228 10228 q10 8658353434943494 q11 427 234 251 234 q12 471 304 295 295 q13 18323 393240213932 q14 351 318 318 318 q15 509 466 466 466 q16 677 578 563 563 q17 1126973 974 973 q18 7278697968286828 q19 1793163916111611 q20 530 307 314 307 q21 4370410240944094 q22 531 443 456 443 Total cold run time: 115844 ms Total hot run time: 49757 ms - Round 2, with runtime_filter_mode=off - q1 4296433042584258 q2 322 230 221 221 q3 4146413741104110 q4 2759273427332733 q5 7142712570807080 q6 236 121 119 119 q7 3239282628482826 q8 4377449545124495 q9 16805 16708 16692 16692 q10 4220425542474247 q11 796 681 697 681 q12 1027859 837 837 q13 6948372537043704 q14 460 433 432 432 q15 500 452 460 452 q16 744 682 665 665 q17 3871391237993799 q18 8775869188068691 q19 1714169916961696 q20 2364209021132090 q21 8641843185008431 q22 1075980 1013980 Total cold run time: 84457 ms Total hot run time: 79239 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
liugddx commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2249635287 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](function) support split_by_regexp function [doris]
zhangstar333 commented on PR #38259: URL: https://github.com/apache/doris/pull/38259#issuecomment-2249638751 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft] [doris]
doris-robot commented on PR #38305: URL: https://github.com/apache/doris/pull/38305#issuecomment-2249641731 TPC-DS: Total hot run time: 174710 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 0f6ea131249f387b333078f31d062d5c6a48c642, data reload: false query1 897 371 370 370 query2 6434206018861886 query3 6655204 211 204 query4 28284 17352 17410 17352 query5 3755500 460 460 query6 267 184 173 173 query7 4572281 281 281 query8 240 191 191 191 query9 8489240423702370 query10 435 285 260 260 query11 11173 10041 10152 10041 query12 111 85 97 85 query13 1639371 361 361 query14 8579809283418092 query15 234 169 170 169 query16 7185422 433 422 query17 1596560 553 553 query18 1140288 295 288 query19 198 156 158 156 query20 92 82 82 82 query21 203 126 128 126 query22 4551417341234123 query23 34176 33859 33904 33859 query24 10996 287029422870 query25 609 414 393 393 query26 704 153 150 150 query27 2230274 283 274 query28 7100214221222122 query29 899 667 625 625 query30 281 157 156 156 query31 1001819 787 787 query32 94 55 54 54 query33 758 344 352 344 query34 950 512 507 507 query35 888 786 787 786 query36 1151980 976 976 query37 166 83 78 78 query38 2918278528092785 query39 867 804 816 804 query40 203 122 117 117 query41 44 42 43 42 query42 119 93 106 93 query43 533 463 501 463 query44 1177737 723 723 query45 207 175 177 175 query46 1120709 737 709 query47 1872177618271776 query48 367 287 289 287 query49 849 405 402 402 query50 780 385 383 383 query51 6836664366026602 query52 101 94 92 92 query53 354 285 285 285 query54 933 448 438 438 query55 75 73 70 70 query56 285 279 263 263 query57 1102108610471047 query58 251 269 270 269 query59 3035263727342637 query60 300 270 274 270 query61 94 93 91 91 query62 790 655 662 655 query63 319 287 284 284 query64 9151224216551655 query65 3168310031103100 query66 766 329 323 323 query67 15631 15050 15113 15050 query68 4570547 540 540 query69 662 416 352 352 query70 1209107711531077 query71 457 279 276 276 query72 8059607254505450 query73 758 325 322 322 query74 6047573557105710 query75 3522268526852685 query76 3215920 938 920 query77 690 316 302 302 query78 9593905189068906 query79 2728522 520 520 query80 2414520 467 467 query81 605 222 219 219 query82 1250137 134 134 query83 285 168 167 167 query84 273 91 92 91 query85 1345309 301 301 query86 447 316 325 316 query87 3279311231453112 query88 4105234723482347 query89 497 392 381 381 query90 1944190 186 186 query91 126 101 100 100 query92 60 52 50 50 query93 3466522 516 516 query94 1285309 281 281 query95 404 316 314 314 query96 603 270 273 270 query97 3234312130273027 query98 225 205 193 193 query99 1553126912621262 Total cold run time: 281889 ms Total hot run time: 174710 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
SWJTU-ZhangLei commented on PR #38243: URL: https://github.com/apache/doris/pull/38243#issuecomment-2249642741 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compaction) fix missing flush column during compaction [doris]
doris-robot commented on PR #38356: URL: https://github.com/apache/doris/pull/38356#issuecomment-2249641771 TPC-DS: Total hot run time: 174035 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 903c6282d8858f886bc39855c3fa6b8447ef3474, data reload: false query1 914 373 363 363 query2 6435197919301930 query3 6635205 220 205 query4 28284 17424 17360 17360 query5 3694482 473 473 query6 267 184 183 183 query7 4590288 285 285 query8 255 188 198 188 query9 8485240823752375 query10 442 302 272 272 query11 12727 991899529918 query12 113 85 86 85 query13 1638378 371 371 query14 10393 764376237623 query15 214 166 176 166 query16 7129454 448 448 query17 1137559 521 521 query18 1917275 276 275 query19 195 143 149 143 query20 89 80 81 80 query21 204 135 127 127 query22 4403409239793979 query23 34121 33763 33619 33619 query24 10013 293329062906 query25 595 403 383 383 query26 703 153 150 150 query27 2253274 278 274 query28 5962209020732073 query29 903 632 627 627 query30 246 149 148 148 query31 983 767 766 766 query32 98 53 62 53 query33 631 334 330 330 query34 903 476 506 476 query35 856 756 740 740 query36 1147997 990 990 query37 144 82 81 81 query38 2965289928662866 query39 915 844 866 844 query40 215 127 135 127 query41 48 47 50 47 query42 115 103 106 103 query43 523 492 489 489 query44 1068732 731 731 query45 209 174 177 174 query46 1086737 720 720 query47 1829179417751775 query48 387 290 293 290 query49 874 399 420 399 query50 774 389 387 387 query51 6767662766326627 query52 104 90 98 90 query53 356 298 289 289 query54 806 439 439 439 query55 74 73 73 73 query56 291 264 262 262 query57 1131108710591059 query58 241 239 291 239 query59 2893276727462746 query60 324 275 271 271 query61 100 100 95 95 query62 799 651 645 645 query63 335 286 289 286 query64 9133219316551655 query65 3192310231043102 query66 829 322 336 322 query67 15516 15092 14970 14970 query68 4522547 545 545 query69 559 397 356 356 query70 1222114611821146 query71 387 272 272 272 query72 7135545060905450 query73 752 324 320 320 query74 6310566456515651 query75 3380266027252660 query76 2137989 904 904 query77 438 307 295 295 query78 10847 10182 91339133 query79 4547526 521 521 query80 1061466 531 466 query81 559 227 215 215 query82 742 137 133 133 query83 197 166 167 166 query84 273 89 82 82 query85 1147329 299 299 query86 460 310 330 310 query87 3254309730923092 query88 4454235323772353 query89 506 397 383 383 query90 1821192 190 190 query91 127 99 97 97 query92 65 48 52 48 query93 5247523 511 511 query94 1166287 295 287 query95 409 309 312 309 query96 610 277 274 274 query97 3208304730253025 query98 223 195 188 188 query99 1510124612591246 Total cold run time: 282892 ms Total hot run time: 174035 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Bug](dry-run) make dryrun result set collect multi batch's QueryStatistics [doris]
github-actions[bot] commented on PR #38303: URL: https://github.com/apache/doris/pull/38303#issuecomment-2249643934 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
github-actions[bot] commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2249645732 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compaction) fix missing flush column during compaction [doris]
doris-robot commented on PR #38356: URL: https://github.com/apache/doris/pull/38356#issuecomment-2249651268 ClickBench: Total hot run time: 30.17 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 903c6282d8858f886bc39855c3fa6b8447ef3474, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.230.040.04 query4 1.680.070.07 query5 0.490.480.48 query6 1.130.770.74 query7 0.020.010.01 query8 0.050.050.04 query9 0.560.500.49 query10 0.540.540.54 query11 0.160.110.11 query12 0.150.120.12 query13 0.600.590.58 query14 0.760.790.78 query15 0.870.820.82 query16 0.360.360.37 query17 1.020.990.98 query18 0.220.220.21 query19 1.801.691.66 query20 0.020.010.01 query21 15.39 0.760.66 query22 4.706.991.64 query23 18.34 1.331.18 query24 2.120.230.23 query25 0.150.090.08 query26 0.300.200.21 query27 0.460.230.23 query28 13.30 1.021.00 query29 12.55 3.283.26 query30 0.250.060.05 query31 2.880.390.39 query32 3.250.480.47 query33 2.902.912.93 query34 16.97 4.354.32 query35 4.384.404.44 query36 0.640.460.47 query37 0.180.150.16 query38 0.160.150.15 query39 0.050.040.03 query40 0.160.120.12 query41 0.100.050.04 query42 0.060.050.05 query43 0.040.040.04 Total cold run time: 110.11 s Total hot run time: 30.17 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft] [doris]
doris-robot commented on PR #38305: URL: https://github.com/apache/doris/pull/38305#issuecomment-2249651278 ClickBench: Total hot run time: 30.48 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 0f6ea131249f387b333078f31d062d5c6a48c642, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.220.050.04 query4 1.680.070.08 query5 0.480.470.48 query6 1.120.720.72 query7 0.020.010.02 query8 0.050.040.05 query9 0.540.500.49 query10 0.550.540.55 query11 0.140.110.11 query12 0.150.120.13 query13 0.600.580.58 query14 0.750.780.80 query15 0.860.810.81 query16 0.360.380.37 query17 1.041.031.06 query18 0.230.220.22 query19 1.901.721.81 query20 0.010.010.00 query21 15.40 0.790.66 query22 4.407.401.78 query23 18.35 1.401.25 query24 2.150.220.24 query25 0.150.090.09 query26 0.310.220.21 query27 0.450.230.23 query28 13.20 1.011.01 query29 12.59 3.303.25 query30 0.240.060.05 query31 2.880.390.38 query32 3.260.470.47 query33 2.982.922.88 query34 17.15 4.354.37 query35 4.474.434.40 query36 0.650.460.47 query37 0.180.160.15 query38 0.160.150.14 query39 0.040.040.04 query40 0.150.120.13 query41 0.090.050.05 query42 0.060.050.05 query43 0.050.050.04 Total cold run time: 110.19 s Total hot run time: 30.48 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] (cloud-merge) Supports online capacity expansion and contraction [doris]
github-actions[bot] commented on PR #37484: URL: https://github.com/apache/doris/pull/37484#issuecomment-2249653939 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhance](RateLimit) Add bvar to monitor OSS's rate limit sleep time and failure time [doris]
github-actions[bot] commented on code in PR #38294: URL: https://github.com/apache/doris/pull/38294#discussion_r1690979539 ## cloud/src/recycler/s3_accessor.h: ## @@ -17,10 +17,13 @@ #pragma once +#include Review Comment: warning: 'bvar/latency_recorder.h' file not found [clang-diagnostic-error] ```cpp #include ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] (cloud-merge) Supports online capacity expansion and contraction [doris]
github-actions[bot] commented on PR #37484: URL: https://github.com/apache/doris/pull/37484#issuecomment-2249653992 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
github-actions[bot] commented on code in PR #38243: URL: https://github.com/apache/doris/pull/38243#discussion_r1690977477 ## cloud/src/meta-service/txn_lazy_committer.h: ## @@ -0,0 +1,65 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#pragma once + +#include Review Comment: warning: 'gen_cpp/cloud.pb.h' file not found [clang-diagnostic-error] ```cpp #include ^ ``` ## cloud/src/meta-service/txn_lazy_committer.cpp: ## @@ -0,0 +1,208 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#include "txn_lazy_committer.h" + +#include "common/logging.h" +#include "common/util.h" +#include "meta-service/keys.h" +#include "meta-service/meta_service_helper.h" + +namespace doris::cloud { + +extern void scan_tmp_rowset( +const std::string& instance_id, int64_t txn_id, std::shared_ptr txn_kv, +MetaServiceCode& code, std::string& msg, int64_t* db_id, +std::vector>* tmp_rowsets_meta); + +extern void convert_tmp_rowsets( +const std::string& instance_id, int64_t txn_id, std::shared_ptr txn_kv, +MetaServiceCode& code, std::string& msg, int64_t db_id, +std::vector>& tmp_rowsets_meta, +std::unordered_map& tablet_ids); + +extern void make_committed_txn_visible(const std::string& instance_id, int64_t db_id, + int64_t txn_id, std::shared_ptr txn_kv, + MetaServiceCode& code, std::string& msg); + +TxnLazyCommitTask::TxnLazyCommitTask(const std::string& instance_id, int64_t txn_id, Review Comment: warning: function 'TxnLazyCommitTask' exceeds recommended size/complexity thresholds [readability-function-size] ```cpp TxnLazyCommitTask::TxnLazyCommitTask(const std::string& instance_id, int64_t txn_id, ^ ``` Additional context **cloud/src/meta-service/txn_lazy_committer.cpp:41:** 130 lines including whitespace and comments (threshold 80) ```cpp TxnLazyCommitTask::TxnLazyCommitTask(const std::string& instance_id, int64_t txn_id, ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) Pick some group commit pr [doris]
doris-robot commented on PR #38320: URL: https://github.com/apache/doris/pull/38320#issuecomment-2249654974 TeamCity be ut coverage result: Function Coverage: 36.44% (9254/25398) Line Coverage: 28.00% (75644/270191) Region Coverage: 26.81% (38878/145023) Branch Coverage: 23.54% (19731/83808) Coverage Report: http://coverage.selectdb-in.cc/coverage/c65a8ccd4284eb7473106b27b3391dffe7878073_c65a8ccd4284eb7473106b27b3391dffe7878073/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]
liaoxin01 opened a new pull request, #38359: URL: https://github.com/apache/doris/pull/38359 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]
doris-robot commented on PR #38359: URL: https://github.com/apache/doris/pull/38359#issuecomment-2249657240 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]
liaoxin01 commented on PR #38359: URL: https://github.com/apache/doris/pull/38359#issuecomment-2249657417 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
doris-robot commented on PR #38150: URL: https://github.com/apache/doris/pull/38150#issuecomment-2249657999 TPC-H: Total hot run time: 49917 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit cb1808a3924f62b57926e08509bb31647acff685, data reload: false -- Round 1 -- q1 18138 444343294329 q2 2074156 157 156 q3 10298 187619051876 q4 10102 125013011250 q5 8420383039353830 q6 239 125 150 125 q7 2072165316191619 q8 9594273227402732 q9 12916 10304 10308 10304 q10 8612352335563523 q11 413 239 258 239 q12 473 303 303 303 q13 18360 396440203964 q14 346 339 322 322 q15 513 456 456 456 q16 660 579 577 577 q17 1138948 945 945 q18 7306696070006960 q19 1797166216021602 q20 526 318 314 314 q21 4404414240504050 q22 530 441 449 441 Total cold run time: 118931 ms Total hot run time: 49917 ms - Round 2, with runtime_filter_mode=off - q1 4375432943034303 q2 322 230 222 222 q3 4221415041464146 q4 2754273527452735 q5 7202712971607129 q6 243 125 123 123 q7 3191278428052784 q8 4381448544344434 q9 16830 16761 16750 16750 q10 4221427842594259 q11 751 697 667 667 q12 1032882 864 864 q13 6709379137563756 q14 446 424 425 424 q15 512 454 459 454 q16 729 681 674 674 q17 3920391838703870 q18 8805878387818781 q19 1761166817061668 q20 2382217220842084 q21 8546840985248409 q22 1046101010081008 Total cold run time: 84379 ms Total hot run time: 79544 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
doris-robot commented on PR #38150: URL: https://github.com/apache/doris/pull/38150#issuecomment-2249661323 TeamCity be ut coverage result: Function Coverage: 37.81% (8117/21466) Line Coverage: 29.48% (66552/225741) Region Coverage: 28.96% (34303/118467) Branch Coverage: 24.84% (17625/70960) Coverage Report: http://coverage.selectdb-in.cc/coverage/cb1808a3924f62b57926e08509bb31647acff685_cb1808a3924f62b57926e08509bb31647acff685/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) fix outof range exception after calling function element_at for variant type [doris]
eldenmoon merged PR #38318: URL: https://github.com/apache/doris/pull/38318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
doris-robot commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2249666440 TPC-H: Total hot run time: 40224 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit ec848b55520d1a4c70c338b096ea396162b3fed2, data reload: false -- Round 1 -- q1 18849 444044114411 q2 2799198 201 198 q3 12542 118711531153 q4 10592 829 712 712 q5 7590268926712671 q6 229 141 137 137 q7 953 598 593 593 q8 9219207820972078 q9 8620658865916588 q10 8805377837583758 q11 448 232 233 232 q12 401 220 220 220 q13 17749 298129722972 q14 271 241 237 237 q15 524 479 487 479 q16 484 379 374 374 q17 961 722 672 672 q18 8119758174657465 q19 8546146614931466 q20 769 328 320 320 q21 4885320233103202 q22 347 288 286 286 Total cold run time: 123702 ms Total hot run time: 40224 ms - Round 2, with runtime_filter_mode=off - q1 4426425642414241 q2 377 271 270 270 q3 2999273628012736 q4 1910158515701570 q5 5293534053445340 q6 219 134 135 134 q7 2116173017521730 q8 3252333833083308 q9 8431841484498414 q10 3882365137343651 q11 565 510 482 482 q12 746 603 591 591 q13 17409 301129612961 q14 306 264 266 264 q15 525 477 468 468 q16 466 425 404 404 q17 1749148914721472 q18 7582740273547354 q19 1662166115991599 q20 2020178817891788 q21 4836464845884588 q22 580 488 502 488 Total cold run time: 71351 ms Total hot run time: 53853 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
doris-robot commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249662215 TPC-DS: Total hot run time: 203935 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, data reload: false query1 930 424 382 382 query2 6528277226392639 query3 6922210 206 206 query4 19794 18058 17930 17930 query5 19714 649464796479 query6 284 220 226 220 query7 4162300 303 300 query8 439 478 427 427 query9 3135268126142614 query10 409 317 307 307 query11 11311 10719 10775 10719 query12 121 81 76 76 query13 5604698 712 698 query14 17701 13041 13298 13041 query15 364 236 245 236 query16 6486295 271 271 query17 16911463897 897 query18 2289416 406 406 query19 215 149 151 149 query20 80 77 80 77 query21 187 100 96 96 query22 5352507550295029 query23 32416 31909 31903 31903 query24 7139645664946456 query25 517 449 430 430 query26 528 164 155 155 query27 1905298 298 298 query28 6176237423412341 query29 3012286126352635 query30 248 167 172 167 query31 887 731 720 720 query32 71 62 62 62 query33 410 250 264 250 query34 851 467 482 467 query35 1103904 901 901 query36 1222119711371137 query37 89 60 62 60 query38 3069295529752955 query39 1374133213201320 query40 206 106 90 90 query41 46 45 45 45 query42 83 82 80 80 query43 812 630 785 630 query44 1113713 703 703 query45 248 235 231 231 query46 1208938 973 938 query47 1849172319201723 query48 1001707 711 707 query49 613 371 372 371 query50 862 615 649 615 query51 4675470246704670 query52 103 81 98 81 query53 442 346 316 316 query54 2655243424132413 query55 97 86 89 86 query56 225 237 201 201 query57 1242113911561139 query58 219 193 210 193 query59 4275385540733855 query60 209 205 214 205 query61 97 105 93 93 query62 784 449 486 449 query63 484 344 337 337 query64 2554154914771477 query65 3650352335723523 query66 781 382 368 368 query67 15598 16732 16067 16067 query68 8288657 662 657 query69 554 359 346 346 query70 1553153513791379 query71 374 304 327 304 query72 6462347934743474 query73 733 333 325 325 query74 6300580258515802 query75 4532366637053666 query76 4638118111901181 query77 593 268 257 257 query78 12615 11661 12033 11661 query79 9203658 659 658 query80 1919400 409 400 query81 504 236 232 232 query82 1689101 96 96 query83 173 133 138 133 query84 261 71 69 69 query85 1118324 336 324 query86 324 297 298 297 query87 3263306030493049 query88 5229233823462338 query89 387 290 285 285 query90 1721206 210 206 query91 171 154 138 138 query92 59 54 56 54 query93 5298542 613 542 query94 673 209 211 209 query95 1092106710471047 query96 636 338 333 333 query97 6515637062826282 query98 193 173 177 173 query99 2922956 974 956 Total cold run time: 310858 ms Total hot run time: 203935 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [Fix](Variant) fix outof range exception after calling function element_at for variant type [doris]
github-actions[bot] commented on PR #38318: URL: https://github.com/apache/doris/pull/38318#issuecomment-2249664531 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Fix](Variant) fix outof range exception after calling function element_at for variant type (#38318)
This is an automated email from the ASF dual-hosted git repository. eldenmoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 20c4cadcbbe [Fix](Variant) fix outof range exception after calling function element_at for variant type (#38318) 20c4cadcbbe is described below commit 20c4cadcbbeb966c32423a01d0f8dda1e5176ac3 Author: lihangyu <15605149...@163.com> AuthorDate: Thu Jul 25 15:40:03 2024 +0800 [Fix](Variant) fix outof range exception after calling function element_at for variant type (#38318) …nt_at for variant type src subcolumns empty but src row count may not be 0, so need to padding rows to at least src.size() and finlize at last ``` [CANCELLED][E6] Parameter out of bound in IColumnStr::insert_range_from me ! ``` --- be/src/vec/functions/function_variant_element.cpp | 10 +- regression-test/data/variant_p0/rqg/rqg3.out | 130 ++ regression-test/suites/variant_p0/rqg/rqg3.sql| 43 +++ 3 files changed, 182 insertions(+), 1 deletion(-) diff --git a/be/src/vec/functions/function_variant_element.cpp b/be/src/vec/functions/function_variant_element.cpp index 84ddc3b8046..faa56ccb1e3 100644 --- a/be/src/vec/functions/function_variant_element.cpp +++ b/be/src/vec/functions/function_variant_element.cpp @@ -28,6 +28,7 @@ #include "common/status.h" #include "exprs/json_functions.h" #include "simdjson.h" +#include "util/defer_op.h" #include "vec/columns/column.h" #include "vec/columns/column_nullable.h" #include "vec/columns/column_object.h" @@ -102,8 +103,11 @@ private: static Status get_element_column(const ColumnObject& src, const ColumnPtr& index_column, ColumnPtr* result) { std::string field_name = index_column->get_data_at(0).to_string(); +Defer finalize([&]() { (*result)->assume_mutable()->finalize(); }); if (src.empty()) { *result = ColumnObject::create(true); +// src subcolumns empty but src row count may not be 0 +(*result)->assume_mutable()->insert_many_defaults(src.size()); return Status::OK(); } if (src.is_scalar_variant() && @@ -135,8 +139,10 @@ private: PathInData path(field_name); ColumnObject::Subcolumns subcolumns = mutable_ptr->get_subcolumns(); const auto* node = subcolumns.find_exact(path); -auto result_col = ColumnObject::create(true, false /*should not create root*/); +MutableColumnPtr result_col; if (node != nullptr) { +// Create without root, since root will be added +result_col = ColumnObject::create(true, false /*should not create root*/); std::vector nodes; PathsInData paths; ColumnObject::Subcolumns::get_leaves_of_node(node, nodes, paths); @@ -162,6 +168,8 @@ private: auto container = ColumnObject::create(std::move(new_subcolumns), true); result_col->insert_range_from(*container, 0, container->size()); } else { +// Create with root, otherwise the root type maybe type Nothing +result_col = ColumnObject::create(true); result_col->insert_many_defaults(src.size()); } *result = result_col->get_ptr(); diff --git a/regression-test/data/variant_p0/rqg/rqg3.out b/regression-test/data/variant_p0/rqg/rqg3.out new file mode 100644 index 000..d39b0e288dc --- /dev/null +++ b/regression-test/data/variant_p0/rqg/rqg3.out @@ -0,0 +1,130 @@ +-- This file is automatically generated. You should know what you did if you want to edit this +-- !rqg3 -- +0 + +-- !rqg3_2 -- +0 + +-- !rqg3_3 -- +0 + +-- !rqg3_4 -- +0 + +-- !rqg3_5 -- +0 + +-- !rqg3_6 -- +0 + +-- !rqg3_7 -- +0 + +-- !rqg3_8 -- +0 + +-- !rqg3_9 -- +0 + +-- !rqg3_10 -- +0 + +-- !rqg3_11 -- +10 + +-- !rqg3_12 -- +10 + +-- !rqg3_13 -- +10 + +-- !rqg3_14 -- +10 + +-- !rqg3_15 -- +10 + +-- !rqg3_16 -- +10 + +-- !rqg3_17 -- +10 + +-- !rqg3_18 -- +10 + +-- !rqg3_19 -- +21 + +-- !rqg3_20 -- +21 + +-- !rqg3_21 -- +21 + +-- !rqg3_22 -- +21 + +-- !rqg3_23 -- +21 + +-- !rqg3_24 -- +21 + +-- !rqg3_25 -- +25 + +-- !rqg3_26 -- +25 + +-- !rqg3_27 -- +25 + +-- !rqg3_28 -- +25 + +-- !rqg3_29 -- +25 + +-- !rqg3_30 -- +25 + +-- !rqg3_31 -- +6 + +-- !rqg3_32 -- +6 + +-- !rqg3_33 -- +6 + +-- !rqg3_34 -- +6 + +-- !rqg3_35 -- +6 + +-- !rqg3_36 -- +6 + +-- !rqg3_37 -- +7 + +-- !rqg3_38 -- +7 + +-- !rqg3_39 -- +7 + +-- !rqg3_40 -- +7 + +-- !rqg3_41 -- +7 + +-- !rqg3_42 -- +7 + +-- !rqg3_43 -- +0 \N c \N \N \N + diff --git a/regression-test/suites/variant_p0/rqg/rqg3.sql b/regression-test/suites/variant_p0/rqg/rqg3.sql new file mode 100644 index 000..d7fde8f1a1c --- /dev/null +++ b/regression-te
Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]
github-actions[bot] commented on PR #38359: URL: https://github.com/apache/doris/pull/38359#issuecomment-2249669519 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](columns)replace logFatal with exception and del some useless code in columns [doris]
eldenmoon commented on code in PR #38035: URL: https://github.com/apache/doris/pull/38035#discussion_r1690991296 ## be/src/common/exception.h: ## @@ -139,3 +139,48 @@ inline const std::string& Exception::to_string() const { } \ } \ } while (0); + +#define HANDLE_EXCEPTION_IF_CATCH_EXCEPTION(stmt, handle_exception) \ +do { \ +try { \ +doris::enable_thread_catch_bad_alloc++; \ +Defer defer {[&]() { doris::enable_thread_catch_bad_alloc--; }}; \ +{ \ +Status _status_ = (stmt); \ +if (UNLIKELY(!_status_.ok())) { \ +return _status_; \ Review Comment: call handle_exception() after `!_status_.ok()` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](columns)replace logFatal with exception and del some useless code in columns [doris]
eldenmoon commented on code in PR #38035: URL: https://github.com/apache/doris/pull/38035#discussion_r1690991296 ## be/src/common/exception.h: ## @@ -139,3 +139,48 @@ inline const std::string& Exception::to_string() const { } \ } \ } while (0); + +#define HANDLE_EXCEPTION_IF_CATCH_EXCEPTION(stmt, handle_exception) \ +do { \ +try { \ +doris::enable_thread_catch_bad_alloc++; \ +Defer defer {[&]() { doris::enable_thread_catch_bad_alloc--; }}; \ +{ \ +Status _status_ = (stmt); \ +if (UNLIKELY(!_status_.ok())) { \ +return _status_; \ Review Comment: call handle_exception() after `!_status_.ok()` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
doris-robot commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249671431 ClickBench: Total hot run time: 30.46 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, data reload: false query1 0.030.020.03 query2 0.070.030.02 query3 0.250.050.04 query4 1.770.070.07 query5 0.540.530.52 query6 1.340.620.61 query7 0.020.010.01 query8 0.030.030.03 query9 0.520.480.49 query10 0.530.530.54 query11 0.120.090.09 query12 0.110.080.09 query13 0.620.610.61 query14 0.770.780.81 query15 0.790.760.76 query16 0.360.360.37 query17 1.021.010.98 query18 0.230.250.25 query19 1.881.851.84 query20 0.020.010.01 query21 15.46 0.570.56 query22 2.002.131.40 query23 17.27 1.030.79 query24 4.242.981.07 query25 0.370.100.06 query26 0.640.150.16 query27 0.040.040.03 query28 6.880.770.73 query29 12.60 2.282.34 query30 0.610.530.53 query31 2.830.390.38 query32 3.350.490.50 query33 3.113.063.06 query34 15.23 4.784.78 query35 4.844.844.82 query36 1.051.011.02 query37 0.060.040.05 query38 0.030.020.02 query39 0.020.010.01 query40 0.160.130.14 query41 0.060.010.02 query42 0.020.020.01 query43 0.030.010.01 Total cold run time: 101.92 s Total hot run time: 30.46 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]
doris-robot commented on PR #38149: URL: https://github.com/apache/doris/pull/38149#issuecomment-2249677118 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428 with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 21.4 seconds inserted 1000 Rows, about 467K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
doris-robot commented on PR #38150: URL: https://github.com/apache/doris/pull/38150#issuecomment-2249681154 TPC-DS: Total hot run time: 203766 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit cb1808a3924f62b57926e08509bb31647acff685, data reload: false query1 940 424 380 380 query2 6550272028272720 query3 6930219 218 218 query4 20233 18161 17912 17912 query5 19727 655966466559 query6 290 213 223 213 query7 4160308 325 308 query8 411 443 442 442 query9 3186272226542654 query10 420 304 284 284 query11 11293 10754 10712 10712 query12 123 84 73 73 query13 5602723 701 701 query14 17634 13373 13444 13373 query15 360 238 242 238 query16 6484287 261 261 query17 17141441872 872 query18 2327436 415 415 query19 213 146 149 146 query20 83 79 80 79 query21 193 99 94 94 query22 5228501549844984 query23 32643 32120 32126 32120 query24 6938647564746474 query25 523 436 439 436 query26 534 165 160 160 query27 1884308 307 307 query28 6177238423632363 query29 2785275526962696 query30 241 166 175 166 query31 937 732 764 732 query32 69 67 63 63 query33 411 256 262 256 query34 847 500 494 494 query35 1125934 934 934 query36 1332111610801080 query37 89 65 60 60 query38 3111290729582907 query39 1364132113391321 query40 201 90 97 90 query41 48 45 44 44 query42 97 89 86 86 query43 713 706 658 658 query44 1149717 727 717 query45 244 236 242 236 query46 1221975 966 966 query47 1817183917411741 query48 1024719 719 719 query49 643 387 378 378 query50 862 613 589 589 query51 4729471446614661 query52 96 87 91 87 query53 445 325 318 318 query54 2639249424752475 query55 90 83 84 83 query56 248 230 221 221 query57 1248110611321106 query58 209 215 208 208 query59 4446413038983898 query60 225 231 212 212 query61 99 93 102 93 query62 835 464 444 444 query63 492 344 344 344 query64 2507153814011401 query65 3672356235733562 query66 830 392 389 389 query67 17329 16153 15017 15017 query68 8609645 659 645 query69 568 356 357 356 query70 1707138814351388 query71 394 303 328 303 query72 6604344934973449 query73 737 328 324 324 query74 6356586359705863 query75 4978373537433735 query76 5026116111501150 query77 756 272 259 259 query78 12583 11632 11684 11632 query79 7157633 634 633 query80 1202416 412 412 query81 501 237 232 232 query82 1496101 99 99 query83 178 140 134 134 query84 258 73 70 70 query85 1021331 335 331 query86 342 291 297 291 query87 3240302429812981 query88 5076237323622362 query89 353 285 315 285 query90 1880207 210 207 query91 176 140 141 140 query92 58 54 53 53 query93 3047580 539 539 query94 729 216 213 213 query95 1138106810561056 query96 645 335 331 331 query97 6557641763656365 query98 187 175 168 168 query99 2863810 860 810 Total cold run time: 309691 ms Total hot run time: 203766 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
doris-robot commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2249686411 TPC-DS: Total hot run time: 172709 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit ec848b55520d1a4c70c338b096ea396162b3fed2, data reload: false query1 926 371 359 359 query2 6448193518461846 query3 6651210 211 210 query4 28433 17694 17401 17401 query5 4178480 480 480 query6 266 169 165 165 query7 4598293 284 284 query8 259 204 200 200 query9 8487242523832383 query10 441 277 269 269 query11 10593 999010030 9990 query12 134 92 83 83 query13 1640387 369 369 query14 9442732476107324 query15 217 166 163 163 query16 7788466 440 440 query17 1250546 528 528 query18 1907271 291 271 query19 194 146 146 146 query20 89 83 81 81 query21 205 136 130 130 query22 4410404740414041 query23 33935 33416 33180 33180 query24 12099 288327822782 query25 668 372 396 372 query26 1822149 150 149 query27 2873267 271 267 query28 7545200819961996 query29 1136616 613 613 query30 289 152 147 147 query31 979 755 751 751 query32 94 51 53 51 query33 773 327 334 327 query34 908 469 496 469 query35 843 739 722 722 query36 938 944 938 query37 210 78 77 77 query38 2842272627482726 query39 858 825 820 820 query40 266 122 116 116 query41 51 46 46 46 query42 126 108 110 108 query43 508 473 476 473 query44 1255776 724 724 query45 207 172 174 172 query46 1100748 723 723 query47 1879178717921787 query48 372 297 284 284 query49 1191421 413 413 query50 800 392 399 392 query51 6792666266396639 query52 109 94 94 94 query53 352 287 281 281 query54 1027447 447 447 query55 79 73 73 73 query56 302 265 265 265 query57 1177104910721049 query58 258 250 250 250 query59 2787283828212821 query60 312 275 307 275 query61 97 94 95 94 query62 822 663 646 646 query63 321 290 283 283 query64 10483 221016421642 query65 3454310631683106 query66 1353346 323 323 query67 15330 15155 14889 14889 query68 4556544 560 544 query69 483 339 352 339 query70 1203108210881082 query71 400 349 271 271 query72 7008545861235458 query73 760 326 325 325 query74 6089559256655592 query75 3430269327022693 query76 2765989 917 917 query77 489 305 298 298 query78 10863 902596539025 query79 2595530 521 521 query80 2126470 464 464 query81 586 225 227 225 query82 826 134 133 133 query83 301 170 172 170 query84 273 89 88 88 query85 2155321 301 301 query86 387 325 286 286 query87 3309307130913071 query88 4303236723652365 query89 474 369 380 369 query90 1616196 192 192 query91 132 100 103 100 query92 64 50 51 50 query93 2885518 509 509 query94 692 283 292 283 query95 403 316 321 316 query96 601 274 271 271 query97 3226303630373036 query98 209 201 195 195 query99 1598125512741255 Total cold run time: 287158 ms Total hot run time: 172709 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
doris-robot commented on PR #38150: URL: https://github.com/apache/doris/pull/38150#issuecomment-2249691493 ClickBench: Total hot run time: 30.47 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit cb1808a3924f62b57926e08509bb31647acff685, data reload: false query1 0.020.030.02 query2 0.070.030.03 query3 0.250.050.04 query4 1.800.070.07 query5 0.540.520.52 query6 1.230.880.61 query7 0.020.010.01 query8 0.030.020.02 query9 0.530.480.48 query10 0.540.540.54 query11 0.120.090.08 query12 0.120.090.10 query13 0.620.620.60 query14 0.790.790.79 query15 0.760.750.76 query16 0.370.360.38 query17 1.001.031.02 query18 0.230.240.25 query19 1.851.851.85 query20 0.010.020.01 query21 15.48 0.560.56 query22 2.282.161.73 query23 17.11 1.130.95 query24 7.970.720.54 query25 0.380.090.07 query26 0.720.160.14 query27 0.040.040.05 query28 5.970.730.75 query29 12.68 2.292.31 query30 0.550.530.52 query31 2.820.380.38 query32 3.380.490.49 query33 3.073.043.07 query34 15.26 4.794.76 query35 4.834.834.85 query36 1.051.011.01 query37 0.060.040.05 query38 0.040.020.02 query39 0.030.020.02 query40 0.160.140.14 query41 0.070.020.01 query42 0.020.020.01 query43 0.020.020.02 Total cold run time: 104.89 s Total hot run time: 30.47 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]
Mryange opened a new pull request, #38360: URL: https://github.com/apache/doris/pull/38360 ## Proposed changes The previous code only considered the x86 architecture, and _mm_movemask_epi8 does not have a corresponding instruction in ARM. According to the article below, we need to abstract the overall logic. [link](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/porting-x86-vector-bitmask-optimizations-to-arm-neon?CommentId=af187ac6-ae00-4e4d-bbf0-e142187aa92e) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]
doris-robot commented on PR #38360: URL: https://github.com/apache/doris/pull/38360#issuecomment-2249695441 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]
Mryange commented on PR #38360: URL: https://github.com/apache/doris/pull/38360#issuecomment-2249695589 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]
doris-robot commented on PR #38215: URL: https://github.com/apache/doris/pull/38215#issuecomment-2249696750 ClickBench: Total hot run time: 30.68 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit ec848b55520d1a4c70c338b096ea396162b3fed2, data reload: false query1 0.040.030.03 query2 0.100.040.03 query3 0.220.050.05 query4 1.680.080.07 query5 0.500.490.49 query6 1.140.740.73 query7 0.020.010.01 query8 0.050.040.05 query9 0.550.490.49 query10 0.540.550.55 query11 0.160.100.11 query12 0.150.130.12 query13 0.590.590.59 query14 0.760.790.77 query15 0.850.810.84 query16 0.360.350.37 query17 0.951.000.94 query18 0.220.200.21 query19 1.791.821.73 query20 0.010.000.01 query21 15.39 0.800.64 query22 4.147.191.94 query23 18.29 1.391.30 query24 2.100.230.23 query25 0.160.090.09 query26 0.300.210.21 query27 0.460.230.23 query28 13.23 1.021.01 query29 12.60 3.283.25 query30 0.250.060.06 query31 2.880.390.39 query32 3.310.480.46 query33 2.842.972.92 query34 16.81 4.424.41 query35 4.494.394.43 query36 0.660.470.49 query37 0.190.160.16 query38 0.160.140.14 query39 0.040.030.03 query40 0.150.120.12 query41 0.090.050.06 query42 0.060.050.04 query43 0.040.050.04 Total cold run time: 109.32 s Total hot run time: 30.68 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]
doris-robot commented on PR #38150: URL: https://github.com/apache/doris/pull/38150#issuecomment-2249697525 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit cb1808a3924f62b57926e08509bb31647acff685 with default session variables Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 21.3 seconds inserted 1000 Rows, about 469K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]
LiBinfeng-01 opened a new pull request, #38361: URL: https://github.com/apache/doris/pull/38361 cherry-pick: #38135 explain shape plan select * from table2 where Null; explain shape plan select * from table2 where false; in this case, null literal can be regard as same as false literal ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]
doris-robot commented on PR #38361: URL: https://github.com/apache/doris/pull/38361#issuecomment-2249702599 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]
LiBinfeng-01 commented on PR #38361: URL: https://github.com/apache/doris/pull/38361#issuecomment-2249702724 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] be必须要关闭swap才能运行,我这边是在自己的笔记本上跑,无法提供大内存 [doris]
xyismy commented on issue #37767: URL: https://github.com/apache/doris/issues/37767#issuecomment-2249706124 本地测试可以自行修改 bin/start_be.sh 脚本,找到下面代码注释它( doris 版本不同,代码不同,可以自己找对应的 swap 检查代码),启动 be 时就不会检查 swap 了 if [[ "$(swapon -s | wc -l)" -gt 1 ]]; then echo "Please disable swap memory before installation." exit 1 fi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]
github-actions[bot] commented on PR #38360: URL: https://github.com/apache/doris/pull/38360#issuecomment-2249707377 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]
LiBinfeng-01 opened a new pull request, #38362: URL: https://github.com/apache/doris/pull/38362 cherry-pick: #38135 explain shape plan select * from table2 where Null; explain shape plan select * from table2 where false; in this case, null literal can be regard as same as false literal ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]
doris-robot commented on PR #38362: URL: https://github.com/apache/doris/pull/38362#issuecomment-2249713617 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]
LiBinfeng-01 commented on PR #38362: URL: https://github.com/apache/doris/pull/38362#issuecomment-2249713718 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](function) support split_by_regexp function [doris]
doris-robot commented on PR #38259: URL: https://github.com/apache/doris/pull/38259#issuecomment-2249716123 TPC-H: Total hot run time: 39752 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit daafe3c0b29f3119dd33734cf1b2b9efb44ded91, data reload: false -- Round 1 -- q1 18187 469442514251 q2 2028193 194 193 q3 10534 121611031103 q4 10222 830 839 830 q5 7519265826652658 q6 220 139 141 139 q7 968 596 625 596 q8 9222210021102100 q9 8744653265316531 q10 8910380838043804 q11 476 243 244 243 q12 404 238 234 234 q13 17770 296429802964 q14 271 239 233 233 q15 531 474 484 474 q16 488 406 390 390 q17 980 651 771 651 q18 7978761673167316 q19 5715134313931343 q20 698 316 331 316 q21 4925310332353103 q22 355 281 280 280 Total cold run time: 117145 ms Total hot run time: 39752 ms - Round 2, with runtime_filter_mode=off - q1 4369423642754236 q2 364 272 266 266 q3 3006288629212886 q4 2016167116571657 q5 571154835483 q6 224 134 134 134 q7 2180186218321832 q8 3293340334553403 q9 8796877489208774 q10 4055386437673767 q11 620 498 493 493 q12 849 656 638 638 q13 15956 325631633163 q14 310 277 305 277 q15 550 505 521 505 q16 495 425 431 425 q17 1831154315091509 q18 7991784379517843 q19 1722152414991499 q20 2101188918781878 q21 5046509847714771 q22 615 489 507 489 Total cold run time: 72100 ms Total hot run time: 55928 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](restore) Reset next version for restored partitions [doris]
w41ter merged PR #38343: URL: https://github.com/apache/doris/pull/38343 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [fix](restore) Reset next version for restored partitions (#38343)
This is an automated email from the ASF dual-hosted git repository. w41ter pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new 3d6689d32cc [fix](restore) Reset next version for restored partitions (#38343) 3d6689d32cc is described below commit 3d6689d32cc84cd96f5c2e76d4378d1586a1344e Author: walter AuthorDate: Thu Jul 25 16:07:54 2024 +0800 [fix](restore) Reset next version for restored partitions (#38343) Cherry-pick #38321 --- .../java/org/apache/doris/backup/RestoreJob.java | 7 -- .../java/org/apache/doris/catalog/OlapTable.java | 4 .../org/apache/doris/backup/RestoreJobTest.java| 27 ++ 3 files changed, 36 insertions(+), 2 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java index 2e20a9fa223..57696bc5a84 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java +++ b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java @@ -82,6 +82,7 @@ import org.apache.doris.thrift.TStorageMedium; import org.apache.doris.thrift.TStorageType; import org.apache.doris.thrift.TTaskType; +import com.google.common.annotations.VisibleForTesting; import com.google.common.base.Joiner; import com.google.common.base.Preconditions; import com.google.common.collect.ArrayListMultimap; @@ -1112,8 +1113,9 @@ public class RestoreJob extends AbstractJob { // reset remote partition. // reset all id in remote partition, but DO NOT modify any exist catalog objects. -private Partition resetPartitionForRestore(OlapTable localTbl, OlapTable remoteTbl, String partName, - ReplicaAllocation replicaAlloc) { +@VisibleForTesting +protected Partition resetPartitionForRestore(OlapTable localTbl, OlapTable remoteTbl, String partName, + ReplicaAllocation replicaAlloc) { Preconditions.checkState(localTbl.getPartition(partName) == null); Partition remotePart = remoteTbl.getPartition(partName); Preconditions.checkNotNull(remotePart); @@ -1143,6 +1145,7 @@ public class RestoreJob extends AbstractJob { // save version info for creating replicas long visibleVersion = remotePart.getVisibleVersion(); +remotePart.setNextVersion(visibleVersion + 1); LOG.info("reset partition {} for restore, visible version: {}, old partition id: {}", newPartId, visibleVersion, oldPartId); diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java index de93324891a..8a4a9fee95f 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java +++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java @@ -348,6 +348,10 @@ public class OlapTable extends Table implements MTMVRelatedTableIf { } } +public void setPartitionInfo(PartitionInfo info) { +partitionInfo = info; +} + public boolean hasMaterializedIndex(String indexName) { return indexNameToId.containsKey(indexName); } diff --git a/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java b/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java index ff60a6e8b90..a9e0a981a12 100644 --- a/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java +++ b/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java @@ -24,10 +24,13 @@ import org.apache.doris.backup.BackupJobInfo.BackupPartitionInfo; import org.apache.doris.backup.BackupJobInfo.BackupTabletInfo; import org.apache.doris.catalog.Database; import org.apache.doris.catalog.Env; +import org.apache.doris.catalog.HashDistributionInfo; import org.apache.doris.catalog.MaterializedIndex; import org.apache.doris.catalog.MaterializedIndex.IndexExtState; import org.apache.doris.catalog.OlapTable; import org.apache.doris.catalog.Partition; +import org.apache.doris.catalog.PartitionInfo; +import org.apache.doris.catalog.PartitionType; import org.apache.doris.catalog.ReplicaAllocation; import org.apache.doris.catalog.Resource; import org.apache.doris.catalog.Table; @@ -51,6 +54,7 @@ import mockit.Injectable; import mockit.Mock; import mockit.MockUp; import mockit.Mocked; +import org.junit.Assert; import org.junit.Before; import org.junit.Test; @@ -276,4 +280,27 @@ public class RestoreJobTest { System.out.println("tbl signature: " + tbl.getSignature(BackupHandler.SIGNATURE_VERSION, partNames)); } +@Test +public void testResetPartitionVisibleAndNextVersionForRestore() throws Exception { +long visibleVersion = 1234; +long remotePartId = 123; +String partName = "p20240723"; +
[PR] [regression](cluster key) move cluster key case to a new directory [doris]
mymeiyi opened a new pull request, #38363: URL: https://github.com/apache/doris/pull/38363 move cluster key case from `unique_with_mow_p0` directory to `unique_with_mow_p0_c` directory, because we will add more case like `unique_with_mow_p0`, now the directory is nested and confusing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](cluster key) move cluster key case to a new directory [doris]
doris-robot commented on PR #38363: URL: https://github.com/apache/doris/pull/38363#issuecomment-2249718807 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](cluster key) move cluster key case to a new directory [doris]
mymeiyi commented on PR #38363: URL: https://github.com/apache/doris/pull/38363#issuecomment-2249723923 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](columns)replace logFatal with exception and del some useless code in columns [doris]
github-actions[bot] commented on PR #38035: URL: https://github.com/apache/doris/pull/38035#issuecomment-2249729658 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](bloom filter) Forbid small bloom filter [doris]
Gabriel39 merged PR #38349: URL: https://github.com/apache/doris/pull/38349 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (20c4cadcbbe -> 73a710fb0bd)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 20c4cadcbbe [Fix](Variant) fix outof range exception after calling function element_at for variant type (#38318) add 73a710fb0bd [Improvement](bloom filter) Forbid small bloom filter (#38349) No new revisions were added by this update. Summary of changes: be/src/exprs/bloom_filter_func.h | 20 be/src/exprs/runtime_filter.cpp | 3 +++ be/src/exprs/runtime_filter.h| 1 + .../java/org/apache/doris/qe/SessionVariable.java| 3 ++- gensrc/thrift/PaloInternalService.thrift | 4 +++- 5 files changed, 25 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]
eldenmoon opened a new pull request, #38364: URL: https://github.com/apache/doris/pull/38364 …subcolumns` ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](auth)fix show routine load db is null [doris]
zddr opened a new pull request, #38365: URL: https://github.com/apache/doris/pull/38365 cause by: https://github.com/apache/doris/pull/27861 when label contains dbName, will loss ``` show routine load for xxx.yyy ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]
doris-robot commented on PR #38364: URL: https://github.com/apache/doris/pull/38364#issuecomment-2249737823 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](group commit) Pick some group commit pr [doris]
mymeiyi commented on PR #38320: URL: https://github.com/apache/doris/pull/38320#issuecomment-2249736944 run feut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth)fix show routine load db is null [doris]
doris-robot commented on PR #38365: URL: https://github.com/apache/doris/pull/38365#issuecomment-2249738726 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](function) support split_by_regexp function [doris]
doris-robot commented on PR #38259: URL: https://github.com/apache/doris/pull/38259#issuecomment-2249739868 TPC-DS: Total hot run time: 173451 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit daafe3c0b29f3119dd33734cf1b2b9efb44ded91, data reload: false query1 908 371 360 360 query2 6443201619071907 query3 6654205 217 205 query4 28082 17471 17639 17471 query5 3685477 478 477 query6 264 157 155 155 query7 4594290 285 285 query8 233 192 198 192 query9 8599240923782378 query10 458 294 284 284 query11 11472 10025 99949994 query12 117 81 83 81 query13 1635403 371 371 query14 10444 768175417541 query15 225 168 168 168 query16 7789474 469 469 query17 1600546 514 514 query18 1968275 267 267 query19 188 146 146 146 query20 85 98 78 78 query21 204 133 129 129 query22 4330419340584058 query23 34378 33802 33534 33534 query24 11026 289728552855 query25 605 377 397 377 query26 1108148 151 148 query27 2370280 283 280 query28 6430208620552055 query29 881 692 629 629 query30 259 159 158 158 query31 981 772 737 737 query32 96 52 57 52 query33 763 317 337 317 query34 947 482 486 482 query35 859 763 759 759 query36 1142989 981 981 query37 140 84 105 84 query38 2951285828292829 query39 933 858 821 821 query40 201 118 120 118 query41 44 45 44 44 query42 117 99 97 97 query43 537 490 496 490 query44 1271727 725 725 query45 205 171 174 171 query46 1080715 701 701 query47 1839174817741748 query48 369 289 289 289 query49 830 402 415 402 query50 773 397 392 392 query51 6787676565436543 query52 101 95 95 95 query53 354 296 294 294 query54 923 453 445 445 query55 74 73 75 73 query56 313 269 263 263 query57 1128104010541040 query58 262 247 297 247 query59 2990266725792579 query60 297 278 272 272 query61 96 93 96 93 query62 804 654 640 640 query63 317 303 292 292 query64 9219222116761676 query65 3168312331643123 query66 745 333 334 333 query67 15385 15051 14763 14763 query68 4547539 536 536 query69 488 334 334 334 query70 1203115611601156 query71 427 287 271 271 query72 7401572155715571 query73 766 326 321 321 query74 6318564657245646 query75 3629267426752674 query76 2730971 895 895 query77 613 310 316 310 query78 9603901589448944 query79 2799516 521 516 query80 1651483 473 473 query81 582 217 226 217 query82 907 142 139 139 query83 325 169 175 169 query84 277 89 89 89 query85 1794383 307 307 query86 484 325 303 303 query87 3276307031083070 query88 3812235323672353 query89 485 393 392 392 query90 1787195 195 195 query91 129 102 100 100 query92 61 52 51 51 query93 3073516 501 501 query94 1146283 290 283 query95 409 315 327 315 query96 597 270 274 270 query97 3264302630913026 query98 222 190 196 190 query99 1744125812791258 Total cold run time: 282624 ms Total hot run time: 173451 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]
doris-robot commented on PR #38359: URL: https://github.com/apache/doris/pull/38359#issuecomment-2249742905 TPC-H: Total hot run time: 40110 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 55208846a86b386c1969cde484bd7e26c33f23e8, data reload: false -- Round 1 -- q1 18223 542444514451 q2 2828194 191 191 q3 11108 129610311031 q4 11173 755 895 755 q5 7806276227442744 q6 227 140 138 138 q7 988 601 589 589 q8 9208208621242086 q9 8688657866026578 q10 8603378638493786 q11 472 231 235 231 q12 396 222 225 222 q13 17780 299529552955 q14 284 247 237 237 q15 532 473 486 473 q16 504 386 397 386 q17 978 613 628 613 q18 8115755574717471 q19 7026141013981398 q20 676 327 330 327 q21 4975316932213169 q22 337 279 281 279 Total cold run time: 120927 ms Total hot run time: 40110 ms - Round 2, with runtime_filter_mode=off - q1 4388440842254225 q2 381 267 273 267 q3 3025278327532753 q4 1939166416001600 q5 5332536053215321 q6 222 127 131 127 q7 2142174517271727 q8 3191335833083308 q9 8397837184018371 q10 3911368536653665 q11 586 490 504 490 q12 769 610 569 569 q13 17620 297830492978 q14 326 273 271 271 q15 512 468 467 467 q16 481 412 427 412 q17 1778147614721472 q18 7781745074567450 q19 1646138215071382 q20 1993179617821782 q21 4925475947164716 q22 564 493 476 476 Total cold run time: 71909 ms Total hot run time: 53829 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
doris-robot commented on PR #38243: URL: https://github.com/apache/doris/pull/38243#issuecomment-2249743123 TPC-H: Total hot run time: 39710 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit bf6aad6e7d1b3229f6fdc1b41de15dc9b7d01bfc, data reload: false -- Round 1 -- q1 18903 633542784278 q2 2011188 197 188 q3 10567 112910771077 q4 10335 730 832 730 q5 7563271126732673 q6 224 137 136 136 q7 950 597 584 584 q8 9216206820492049 q9 8626652665306526 q10 8716375637583756 q11 483 238 245 238 q12 411 226 224 224 q13 17774 298229882982 q14 284 237 263 237 q15 522 486 493 486 q16 511 392 400 392 q17 986 696 695 695 q18 8084744273017301 q19 4948144214161416 q20 642 329 317 317 q21 4837315432013154 q22 344 273 271 271 Total cold run time: 116937 ms Total hot run time: 39710 ms - Round 2, with runtime_filter_mode=off - q1 4377420342014201 q2 360 268 262 262 q3 3047290129562901 q4 1993165417201654 q5 5585553154895489 q6 220 145 129 129 q7 2223188418281828 q8 3276339933853385 q9 8726884088168816 q10 4142379839203798 q11 606 510 523 510 q12 811 659 657 657 q13 15727 323031883188 q14 322 291 295 291 q15 556 484 503 484 q16 500 440 422 422 q17 1845155015201520 q18 8033795577097709 q19 1778154415341534 q20 2240188018791879 q21 8261489148574857 q22 609 541 519 519 Total cold run time: 75237 ms Total hot run time: 56033 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](function) support split_by_regexp function [doris]
doris-robot commented on PR #38259: URL: https://github.com/apache/doris/pull/38259#issuecomment-2249750848 ClickBench: Total hot run time: 30.96 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit daafe3c0b29f3119dd33734cf1b2b9efb44ded91, data reload: false query1 0.050.040.04 query2 0.090.040.04 query3 0.220.060.05 query4 1.680.080.07 query5 0.500.480.48 query6 1.120.730.73 query7 0.020.010.02 query8 0.050.040.04 query9 0.550.490.50 query10 0.550.540.54 query11 0.150.120.12 query12 0.160.130.12 query13 0.610.590.59 query14 0.770.770.81 query15 0.850.830.80 query16 0.350.360.34 query17 0.970.970.97 query18 0.230.220.21 query19 1.811.681.74 query20 0.010.010.01 query21 15.41 0.780.66 query22 4.207.262.12 query23 18.26 1.371.31 query24 2.110.240.21 query25 0.160.090.08 query26 0.280.200.21 query27 0.460.230.23 query28 13.35 1.011.00 query29 12.63 3.553.48 query30 0.260.070.06 query31 2.850.390.39 query32 3.280.470.46 query33 2.942.882.94 query34 17.09 4.344.33 query35 4.404.404.40 query36 0.650.460.50 query37 0.180.150.16 query38 0.160.150.15 query39 0.040.040.04 query40 0.140.110.13 query41 0.100.050.05 query42 0.060.050.06 query43 0.050.040.04 Total cold run time: 109.8 s Total hot run time: 30.96 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](http action)Add http action to show nested inverted index file [doris]
github-actions[bot] commented on code in PR #38272: URL: https://github.com/apache/doris/pull/38272#discussion_r1691046960 ## be/src/olap/rowset/beta_rowset.cpp: ## @@ -701,4 +702,127 @@ Status BetaRowset::calc_file_crc(uint32_t* crc_value, int64_t* file_count) { return Status::OK(); } +Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value, Review Comment: warning: function 'show_nested_index_file' exceeds recommended size/complexity thresholds [readability-function-size] ```cpp Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value, ^ ``` Additional context **be/src/olap/rowset/beta_rowset.cpp:704:** 120 lines including whitespace and comments (threshold 80) ```cpp Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value, ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]
github-actions[bot] commented on PR #38364: URL: https://github.com/apache/doris/pull/38364#issuecomment-2249750710 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Bug] (mtmv) Unable to find a suitable base table for partitioning when there are both valid and invalid expressions [doris]
liutang123 opened a new issue, #38366: URL: https://github.com/apache/doris/issues/38366 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 2.1 ### What's Wrong? reproduce: ``` CREATE TABLE `test_t` ( `id` VARCHAR(36) NOT NULL COMMENT 'id', `created_time` DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '' ) ENGINE=OLAP DUPLICATE KEY(`id`) COMMENT '' PARTITION BY RANGE(`created_time`) (PARTITION P_2024071713 VALUES [('2024-07-17 13:00:00'), ('2024-07-17 14:00:00')), PARTITION P_2024071714 VALUES [('2024-07-17 14:00:00'), ('2024-07-17 15:00:00'))) DISTRIBUTED BY HASH(`id`) BUCKETS AUTO PROPERTIES ( "replication_allocation" = "tag.location.default: 1" ); CREATE MATERIALIZED VIEW test_mtmv BUILD IMMEDIATE REFRESH AUTO ON MANUAL partition by(date_trunc(created_time_minute, 'day')) DISTRIBUTED BY RANDOM BUCKETS 3 PROPERTIES ('replication_num' = '1') AS select id ,date_trunc(created_time, 'minute') as created_time_minute ,min(created_time) as start_time ,if(count(id) > 0, 1, 0) as status from test_t group by id, date_trunc(created_time, 'minute'); ``` An error occurs: ``` ERROR 1105 (HY000): errCode = 2, detailMessage = Unable to find a suitable base table for partitioning, the fail reason is can't not find valid partition track column, because column to check use invalid implicit expression, invalid expression is min(created_time#12) ``` ### What You Expected? `test_t` should be selected as base table. `date_trunc(created_time, 'minute')` should pass the check and `min(created_time)` should be ignore. ### How to Reproduce? Refer to the description above. ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]
github-actions[bot] commented on PR #38364: URL: https://github.com/apache/doris/pull/38364#issuecomment-2249753344 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix] fix arrow read timestamp bug [doris-flink-connector]
JNSimba opened a new pull request, #446: URL: https://github.com/apache/doris-flink-connector/pull/446 # Proposed changes Currently, when reading arrow, since the be layer hard-codes all datetimev2 scales to 6, arrow returns TimeStampMicroVector, which results in scale parsing errors when arrow is displayed. Refer to https://github.com/apache/doris/issues/38174 This PR removes the strong consistency restriction of the type and is compatible with the logic of the scale repaired above ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]
doris-robot commented on PR #38360: URL: https://github.com/apache/doris/pull/38360#issuecomment-2249757100 TPC-H: Total hot run time: 40187 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6635a3866fd28147784237653e3621b85a570a71, data reload: false -- Round 1 -- q1 18191 501844814481 q2 2488225 194 194 q3 11465 1194989 989 q4 11179 745 856 745 q5 7700275227392739 q6 228 139 145 139 q7 969 624 621 621 q8 9243207721002077 q9 8719661166536611 q10 8774375037953750 q11 477 236 241 236 q12 396 232 231 231 q13 17811 297530272975 q14 281 234 247 234 q15 535 479 491 479 q16 482 392 391 391 q17 978 624 737 624 q18 8141747174317431 q19 6977139513771377 q20 692 330 340 330 q21 4935324332603243 q22 351 298 290 290 Total cold run time: 121012 ms Total hot run time: 40187 ms - Round 2, with runtime_filter_mode=off - q1 4479429842794279 q2 379 262 268 262 q3 3020271927712719 q4 1900160916171609 q5 5282532753555327 q6 217 133 133 133 q7 2074165516951655 q8 3212335733113311 q9 8409844383618361 q10 3881366536703665 q11 617 494 497 494 q12 766 637 611 611 q13 17540 302430053005 q14 305 269 276 269 q15 510 485 474 474 q16 484 428 431 428 q17 1776149014791479 q18 7596750272597259 q19 1668161916141614 q20 1983176917641764 q21 4972472248164722 q22 586 520 505 505 Total cold run time: 71656 ms Total hot run time: 53945 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](inverted index) disable range query in StringTypeInvertedIndexReader [doris]
github-actions[bot] commented on PR #38218: URL: https://github.com/apache/doris/pull/38218#issuecomment-2249759777 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix] (mtmv) Choose a valid partition column when there are both valid and invalid expressions [doris]
doris-robot commented on PR #38367: URL: https://github.com/apache/doris/pull/38367#issuecomment-2249761648 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix] (mtmv) Choose a valid partition column when there are both valid and invalid expressions [doris]
liutang123 commented on PR #38367: URL: https://github.com/apache/doris/pull/38367#issuecomment-2249762242 Will add regression tests later -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]
doris-robot commented on PR #38359: URL: https://github.com/apache/doris/pull/38359#issuecomment-2249766067 TPC-DS: Total hot run time: 173066 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 55208846a86b386c1969cde484bd7e26c33f23e8, data reload: false query1 928 385 361 361 query2 6445188518151815 query3 6664211 214 211 query4 27519 17329 17346 17329 query5 4241479 469 469 query6 265 185 159 159 query7 4587291 280 280 query8 235 197 198 197 query9 8391241923892389 query10 441 294 275 275 query11 11812 10146 10044 10044 query12 145 83 89 83 query13 1643381 369 369 query14 10273 788576977697 query15 215 167 165 165 query16 7630470 505 470 query17 1612553 551 551 query18 1526287 283 283 query19 208 152 160 152 query20 93 82 83 82 query21 210 134 127 127 query22 4190412139803980 query23 33830 33304 33169 33169 query24 11970 285728692857 query25 660 390 395 390 query26 1752151 150 150 query27 2765272 283 272 query28 7404202920212021 query29 1048666 634 634 query30 288 151 153 151 query31 970 736 741 736 query32 101 54 54 54 query33 785 356 337 337 query34 895 474 484 474 query35 867 740 737 737 query36 1095922 922 922 query37 258 79 81 79 query38 2849278927582758 query39 854 813 855 813 query40 284 122 121 121 query41 48 46 48 46 query42 127 97 104 97 query43 498 468 458 458 query44 1306728 736 728 query45 214 179 179 179 query46 1097780 744 744 query47 1861179017971790 query48 369 293 298 293 query49 1272409 408 408 query50 780 395 390 390 query51 6839659666946596 query52 101 96 90 90 query53 363 289 290 289 query54 976 442 448 442 query55 77 72 75 72 query56 281 271 266 266 query57 1209107110611061 query58 267 256 240 240 query59 2791257125502550 query60 306 278 283 278 query61 96 97 97 97 query62 838 663 667 663 query63 320 301 290 290 query64 10496 220816861686 query65 3193310830963096 query66 1292364 329 329 query67 15629 15077 14830 14830 query68 8511554 563 554 query69 691 460 357 357 query70 1218115311511151 query71 527 284 270 270 query72 8920576454765476 query73 1306325 322 322 query74 6161576857245724 query75 4592267127432671 query76 4987910 946 910 query77 745 306 297 297 query78 9592923389338933 query79 8160521 519 519 query80 1022481 485 481 query81 599 222 218 218 query82 292 133 130 130 query83 300 168 160 160 query84 272 92 84 84 query85 1034309 346 309 query86 355 313 339 313 query87 3339310631203106 query88 4790237223732372 query89 522 389 376 376 query90 2074197 192 192 query91 129 104 100 100 query92 64 51 50 50 query93 5845517 510 510 query94 1338283 258 258 query95 410 319 312 312 query96 607 271 271 271 query97 3216307730383038 query98 223 209 188 188 query99 1552124312971243 Total cold run time: 303987 ms Total hot run time: 173066 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
doris-robot commented on PR #38243: URL: https://github.com/apache/doris/pull/38243#issuecomment-2249765436 TPC-DS: Total hot run time: 173437 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit bf6aad6e7d1b3229f6fdc1b41de15dc9b7d01bfc, data reload: false query1 906 365 362 362 query2 6420205019471947 query3 6633206 227 206 query4 26326 17597 17202 17202 query5 3582485 478 478 query6 286 176 179 176 query7 4585297 286 286 query8 250 193 191 191 query9 8529241723852385 query10 438 291 276 276 query11 12568 10058 99399939 query12 120 91 84 84 query13 1654372 370 370 query14 10167 740676717406 query15 208 161 164 161 query16 7673462 443 443 query17 1378532 543 532 query18 1933281 273 273 query19 199 146 162 146 query20 94 87 79 79 query21 209 128 123 123 query22 4407400840784008 query23 34386 33495 33531 33495 query24 11288 301429792979 query25 593 384 448 384 query26 774 157 158 157 query27 2210285 274 274 query28 6128208720732073 query29 917 632 638 632 query30 260 160 150 150 query31 979 783 777 777 query32 100 56 54 54 query33 767 331 347 331 query34 933 488 487 487 query35 867 749 754 749 query36 11351006957 957 query37 140 81 82 81 query38 2968290328092809 query39 901 874 829 829 query40 202 123 116 116 query41 48 45 46 45 query42 118 98 93 93 query43 527 470 483 470 query44 1191727 758 727 query45 210 172 172 172 query46 1083732 728 728 query47 1857178517681768 query48 365 292 287 287 query49 819 403 412 403 query50 783 387 390 387 query51 6813662666936626 query52 102 89 91 89 query53 356 290 292 290 query54 901 446 447 446 query55 76 77 74 74 query56 278 292 263 263 query57 1137106010421042 query58 248 245 254 245 query59 2920265427302654 query60 302 279 272 272 query61 99 105 96 96 query62 824 655 634 634 query63 319 283 286 283 query64 9137220616231623 query65 3170312130963096 query66 741 333 337 333 query67 15501 15031 15108 15031 query68 5317561 540 540 query69 654 433 366 366 query70 1112115810871087 query71 455 283 274 274 query72 7331596853025302 query73 769 322 330 322 query74 6182570656655665 query75 3682269726892689 query76 3156963 875 875 query77 640 319 292 292 query78 10358 914190959095 query79 1633539 529 529 query80 2441563 486 486 query81 595 227 227 227 query82 427 137 144 137 query83 305 169 165 165 query84 265 86 95 86 query85 1061309 300 300 query86 438 338 313 313 query87 3269309931473099 query88 3368238623642364 query89 471 391 412 391 query90 1637197 200 197 query91 137 100 102 100 query92 65 51 50 50 query93 1951535 525 525 query94 796 305 298 298 query95 415 332 323 323 query96 596 274 279 274 query97 3187304230303030 query98 228 196 195 195 query99 1568128112571257 Total cold run time: 278547 ms Total hot run time: 173437 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the