Re: [PR] [improve](nested_types)support nested type with agg replace_if_not_null [doris]

2024-07-25 Thread via GitHub


xiaokang merged PR #38304:
URL: https://github.com/apache/doris/pull/38304


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


amorynan commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249602266

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [chore](bash) Make sure that the `prof_prefix` defined in be.conf is not overwritten in start_be.sh [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38354:
URL: https://github.com/apache/doris/pull/38354#issuecomment-2249602257

   
   
   TPC-H: Total hot run time: 39590 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit ba33fa9344811d2e9f805f5d8970ab7abefa517f, 
data reload: false
   
   -- Round 1 --
   q1   17701   434542444244
   q2   2014191 181 181
   q3   10463   113510371037
   q4   10188   786 799 786
   q5   7541269926862686
   q6   219 136 136 136
   q7   954 593 597 593
   q8   9218205820862058
   q9   8875651165576511
   q10  8730378437393739
   q11  463 231 233 231
   q12  462 226 225 225
   q13  19158   301129822982
   q14  280 241 233 233
   q15  513 475 484 475
   q16  492 379 381 379
   q17  962 596 655 596
   q18  7973745973167316
   q19  8340149114291429
   q20  724 319 325 319
   q21  4869315640433156
   q22  353 278 288 278
   Total cold run time: 120492 ms
   Total hot run time: 39590 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4406422042134213
   q2   389 270 258 258
   q3   3180296829352935
   q4   2036170117171701
   q5   5582549055345490
   q6   231 131 132 131
   q7   2238185217811781
   q8   3304339134223391
   q9   8797894487578757
   q10  4049382838573828
   q11  606 499 487 487
   q12  813 633 644 633
   q13  17136   317531723172
   q14  324 285 302 285
   q15  539 482 489 482
   q16  506 444 437 437
   q17  1790152115161516
   q18  8109796679047904
   q19  1734144615161446
   q20  2170187018791870
   q21  7877481449364814
   q22  586 524 506 506
   Total cold run time: 76402 ms
   Total hot run time: 56037 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](test) fix unstable refresh catalog/db ut [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38352:
URL: https://github.com/apache/doris/pull/38352#issuecomment-2249605874

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](test) fix unstable refresh catalog/db ut [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38352:
URL: https://github.com/apache/doris/pull/38352#issuecomment-2249605928

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature][external catalog/lakesoul] LakeSoul Catalog support Filter Pushdown & Cdc data handling & S3 data access [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #37979:
URL: https://github.com/apache/doris/pull/37979#issuecomment-2249605561

   
   
   TPC-DS: Total hot run time: 174357 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 2360a214572d1e6ddf4e2ba28b1868d1f32eaf7a, 
data reload: false
   
   query1   908 383 364 364
   query2   6432196519601960
   query3   6647204 217 204
   query4   28431   17398   17458   17398
   query5   3755480 471 471
   query6   263 180 150 150
   query7   4584289 278 278
   query8   244 190 191 190
   query9   8428241223852385
   query10  430 293 274 274
   query11  12717   10163   10125   10125
   query12  120 86  85  85
   query13  1649378 369 369
   query14  10507   770182927701
   query15  232 166 170 166
   query16  7601465 444 444
   query17  1599588 542 542
   query18  1812290 283 283
   query19  210 158 155 155
   query20  90  86  83  83
   query21  217 134 133 133
   query22  4256409139543954
   query23  34599   33671   33496   33496
   query24  10937   297929102910
   query25  618 407 385 385
   query26  701 153 152 152
   query27  2271276 276 276
   query28  6166208820612061
   query29  922 662 638 638
   query30  260 162 154 154
   query31  987 774 772 772
   query32  97  53  63  53
   query33  715 342 333 333
   query34  925 520 513 513
   query35  885 781 754 754
   query36  1152959 929 929
   query37  138 85  86  85
   query38  2925276827612761
   query39  871 811 788 788
   query40  201 119 121 119
   query41  50  42  43  42
   query42  117 103 105 103
   query43  525 488 492 488
   query44  1193745 748 745
   query45  208 174 174 174
   query46  1087729 742 729
   query47  1852176717791767
   query48  360 294 287 287
   query49  831 406 411 406
   query50  788 386 406 386
   query51  6797668667096686
   query52  102 92  91  91
   query53  369 294 301 294
   query54  916 447 448 447
   query55  78  73  75  73
   query56  281 263 263 263
   query57  1187104611021046
   query58  255 251 256 251
   query59  3051270327502703
   query60  315 282 283 282
   query61  97  94  94  94
   query62  773 681 631 631
   query63  320 285 295 285
   query64  9183224074042240
   query65  3182308831103088
   query66  696 333 324 324
   query67  15507   15016   14882   14882
   query68  4705558 595 558
   query69  633 450 355 355
   query70  1191109210291029
   query71  420 282 286 282
   query72  8610521756435217
   query73  761 327 329 327
   query74  6070568757175687
   query75  3474268926892689
   query76  3439983 910 910
   query77  635 326 326 326
   query78  11577   949389598959
   query79  6444533 520 520
   query80  1541476 473 473
   query81  610 229 219 219
   query82  816 142 136 136
   query83  336 167 168 167
   query84  276 85  89  85
   query85  1358314 343 314
   query86  458 310 292 292
   query87  3310313231273127
   query88  4600237323722372
   query89  498 393 386 386
   query90  1942198 192 192
   query91  136 99  101 99
   query92  59  49  50  49
   query93  4820521 515 515
   query94  1309305 295 295
   query95  410 329 318 318
   query96  611 280 278 278
   query97  3211302630273026
   query98  211 211 195 195
   query99  1590126012691260
   Total cold run time: 292683 ms
   Total hot run time: 174357 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the

Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]

2024-07-25 Thread via GitHub


SWJTU-ZhangLei commented on PR #38243:
URL: https://github.com/apache/doris/pull/38243#issuecomment-2249608295

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature][external catalog/lakesoul] LakeSoul Catalog support Filter Pushdown & Cdc data handling & S3 data access [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #37979:
URL: https://github.com/apache/doris/pull/37979#issuecomment-2249614294

   
   
   ClickBench: Total hot run time: 31.01 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 2360a214572d1e6ddf4e2ba28b1868d1f32eaf7a, 
data reload: false
   
   query1   0.040.030.03
   query2   0.080.040.04
   query3   0.230.050.06
   query4   1.680.080.08
   query5   0.490.490.48
   query6   1.130.730.73
   query7   0.020.010.01
   query8   0.060.050.05
   query9   0.550.480.49
   query10  0.540.540.53
   query11  0.150.120.11
   query12  0.150.120.12
   query13  0.590.580.58
   query14  0.750.790.77
   query15  0.870.820.83
   query16  0.360.360.40
   query17  0.991.000.99
   query18  0.230.220.21
   query19  1.911.781.75
   query20  0.010.020.01
   query21  15.54   0.830.67
   query22  4.786.072.13
   query23  18.28   1.371.33
   query24  2.180.220.23
   query25  0.140.090.08
   query26  0.300.210.20
   query27  0.460.230.23
   query28  13.24   1.021.00
   query29  12.64   3.303.34
   query30  0.240.060.05
   query31  2.890.410.40
   query32  3.230.480.47
   query33  2.852.952.93
   query34  16.87   4.304.32
   query35  4.424.444.49
   query36  0.650.490.46
   query37  0.190.160.16
   query38  0.160.150.15
   query39  0.050.040.03
   query40  0.150.120.12
   query41  0.100.050.05
   query42  0.060.050.05
   query43  0.050.050.04
   Total cold run time: 110.3 s
   Total hot run time: 31.01 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249611394

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249612844

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [chore](bash) Make sure that the `prof_prefix` defined in be.conf is not overwritten in start_be.sh [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38354:
URL: https://github.com/apache/doris/pull/38354#issuecomment-2249619865

   
   
   TPC-DS: Total hot run time: 173725 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit ba33fa9344811d2e9f805f5d8970ab7abefa517f, 
data reload: false
   
   query1   911 380 365 365
   query2   6425198819181918
   query3   6646207 215 207
   query4   28367   17504   17620   17504
   query5   3597464 474 464
   query6   257 176 171 171
   query7   4583285 291 285
   query8   235 190 201 190
   query9   8557237923842379
   query10  446 288 279 279
   query11  11777   10065   10173   10065
   query12  117 84  94  84
   query13  1641372 379 372
   query14  10157   759970907090
   query15  223 171 171 171
   query16  7771471 464 464
   query17  1607591 544 544
   query18  1955286 280 280
   query19  199 156 159 156
   query20  95  84  81  81
   query21  214 123 126 123
   query22  4507412339703970
   query23  34339   33972   33709   33709
   query24  10640   290029112900
   query25  594 387 389 387
   query26  707 155 149 149
   query27  2233274 289 274
   query28  6018208120572057
   query29  906 653 649 649
   query30  261 155 157 155
   query31  965 801 759 759
   query32  96  52  53  52
   query33  642 334 332 332
   query34  900 487 510 487
   query35  918 746 741 741
   query36  1126990 988 988
   query37  145 85  83  83
   query38  2918290328262826
   query39  887 894 871 871
   query40  206 128 121 121
   query41  49  45  46  45
   query42  114 99  103 99
   query43  530 489 498 489
   query44  1092724 726 724
   query45  207 174 176 174
   query46  1097727 714 714
   query47  1852178518041785
   query48  361 293 284 284
   query49  843 400 405 400
   query50  786 386 384 384
   query51  6874671367206713
   query52  100 94  89  89
   query53  355 308 283 283
   query54  911 448 448 448
   query55  73  73  72  72
   query56  282 280 269 269
   query57  1153104310391039
   query58  247 252 267 252
   query59  2979281626892689
   query60  313 265 277 265
   query61  95  97  95  95
   query62  826 626 633 626
   query63  321 290 286 286
   query64  9297220816831683
   query65  3180310331413103
   query66  750 329 330 329
   query67  15400   14826   14865   14826
   query68  7033545 543 543
   query69  685 463 371 371
   query70  1194114110731073
   query71  488 278 272 272
   query72  6951537357635373
   query73  812 323 326 323
   query74  6265569056815681
   query75  4001270627132706
   query76  4119929 1004929
   query77  702 316 319 316
   query78  9806899598238995
   query79  5042527 527 527
   query80  1109481 476 476
   query81  571 221 229 221
   query82  792 139 135 135
   query83  208 178 175 175
   query84  272 91  85  85
   query85  1287315 302 302
   query86  347 311 318 311
   query87  3245309430983094
   query88  4685236323682363
   query89  498 383 378 378
   query90  1493191 190 190
   query91  126 163 100 100
   query92  59  49  48  48
   query93  5312521 503 503
   query94  591 294 285 285
   query95  397 315 324 315
   query96  606 277 275 275
   query97  3167309730263026
   query98  231 198 200 198
   query99  1556127012691269
   Total cold run time: 288525 ms
   Total hot run time: 173725 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the

Re: [PR] [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled [doris]

2024-07-25 Thread via GitHub


eldenmoon merged PR #37900:
URL: https://github.com/apache/doris/pull/37900


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled (#37900)

2024-07-25 Thread eldenmoon
This is an automated email from the ASF dual-hosted git repository.

eldenmoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 1b3d1678c23 [Refactor](ShortCircuit) Conjuncts outof key columns's 
order should be handled (#37900)
1b3d1678c23 is described below

commit 1b3d1678c23af03fcdd5ee78614a4bffbeec2dad
Author: lihangyu <15605149...@163.com>
AuthorDate: Thu Jul 25 15:14:10 2024 +0800

[Refactor](ShortCircuit) Conjuncts outof key columns's order should be 
handled (#37900)

…e handled
---
 .../main/java/org/apache/doris/qe/PointQueryExecutor.java  | 14 +-
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java 
b/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java
index 2ce047c9ddc..5aa07246570 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/PointQueryExecutor.java
@@ -47,6 +47,7 @@ import org.apache.doris.thrift.TStatusCode;
 import com.google.common.base.Preconditions;
 import com.google.common.base.Strings;
 import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
 import org.apache.thrift.TDeserializer;
@@ -57,6 +58,7 @@ import java.util.Collections;
 import java.util.HashSet;
 import java.util.Iterator;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
 import java.util.concurrent.ExecutionException;
 import java.util.concurrent.Future;
@@ -177,16 +179,18 @@ public class PointQueryExecutor implements CoordInterface 
{
 void addKeyTuples(
 InternalService.PTabletKeyLookupRequest.Builder requestBuilder) {
 // TODO handle IN predicates
+Map columnExpr = Maps.newHashMap();
 KeyTuple.Builder kBuilder = KeyTuple.newBuilder();
 for (Expr expr : shortCircuitQueryContext.scanNode.getConjuncts()) {
 BinaryPredicate predicate = (BinaryPredicate) expr;
 Expr left = predicate.getChild(0);
 Expr right = predicate.getChild(1);
-// ignore delete sign conjuncts only collect key conjuncts
-if (left instanceof SlotRef && ((SlotRef) 
left).getColumnName().equalsIgnoreCase(Column.DELETE_SIGN)) {
-continue;
-}
-kBuilder.addKeyColumnRep(right.getStringValue());
+SlotRef columnSlot = left.unwrapSlotRef();
+columnExpr.put(columnSlot.getColumnName(), right);
+}
+// add key tuple in keys order
+for (Column column : 
shortCircuitQueryContext.scanNode.getOlapTable().getBaseSchemaKeyColumns()) {
+
kBuilder.addKeyColumnRep(columnExpr.get(column.getName()).getStringValue());
 }
 requestBuilder.addKeyTuples(kBuilder);
 }


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Refactor](ShortCircuit) Conjuncts outof key columns's order should be handled [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #37900:
URL: https://github.com/apache/doris/pull/37900#issuecomment-2249621197

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [draft] [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38305:
URL: https://github.com/apache/doris/pull/38305#issuecomment-2249623305

   
   
   TPC-H: Total hot run time: 39794 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 0f6ea131249f387b333078f31d062d5c6a48c642, 
data reload: false
   
   -- Round 1 --
   q1   17630   433043014301
   q2   2005185 183 183
   q3   10473   121210921092
   q4   10181   830 769 769
   q5   7601267927052679
   q6   225 138 136 136
   q7   948 591 589 589
   q8   9213206120932061
   q9   8933657065596559
   q10  8926378537053705
   q11  451 239 235 235
   q12  451 233 222 222
   q13  17904   297529612961
   q14  277 236 243 236
   q15  526 478 482 478
   q16  503 397 383 383
   q17  969 671 680 671
   q18  8077753173197319
   q19  8204143114261426
   q20  706 326 319 319
   q21  4850319232863192
   q22  344 278 281 278
   Total cold run time: 119397 ms
   Total hot run time: 39794 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4387425742604257
   q2   382 275 254 254
   q3   3054292430142924
   q4   1944170917291709
   q5   5629556454735473
   q6   219 131 138 131
   q7   2282183518511835
   q8   3249341934043404
   q9   8817883688438836
   q10  4158368838733688
   q11  603 524 495 495
   q12  831 643 633 633
   q13  17223   314531833145
   q14  310 285 282 282
   q15  527 474 511 474
   q16  497 440 428 428
   q17  1805151815311518
   q18  8190796678917891
   q19  1761149214461446
   q20  2934191618791879
   q21  5155469348454693
   q22  574 519 509 509
   Total cold run time: 74531 ms
   Total hot run time: 55904 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](compaction) fix missing flush column during compaction [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38356:
URL: https://github.com/apache/doris/pull/38356#issuecomment-2249623339

   
   
   TPC-H: Total hot run time: 39630 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 903c6282d8858f886bc39855c3fa6b8447ef3474, 
data reload: false
   
   -- Round 1 --
   q1   17619   429142684268
   q2   2029192 185 185
   q3   10468   114711191119
   q4   10188   812 699 699
   q5   7517270426442644
   q6   218 136 141 136
   q7   943 596 594 594
   q8   9231203320652033
   q9   8756652265316522
   q10  8810375637583756
   q11  468 232 240 232
   q12  461 218 219 218
   q13  18747   298329712971
   q14  283 233 245 233
   q15  516 468 470 468
   q16  480 377 376 376
   q17  961 661 783 661
   q18  8002747473747374
   q19  7935137413881374
   q20  702 322 326 322
   q21  4883317332633173
   q22  335 278 272 272
   Total cold run time: 119552 ms
   Total hot run time: 39630 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4441425842064206
   q2   366 283 269 269
   q3   3075291629452916
   q4   1977175418201754
   q5   5608555954695469
   q6   222 135 140 135
   q7   2256191318091809
   q8   3230345333993399
   q9   8741886188448844
   q10  4177374337853743
   q11  595 499 501 499
   q12  812 641 642 641
   q13  15907   317132313171
   q14  318 284 289 284
   q15  534 498 482 482
   q16  525 433 446 433
   q17  1817154314961496
   q18  8200811378357835
   q19  1729139814641398
   q20  2273187519011875
   q21  10605   490946154615
   q22  606 506 503 503
   Total cold run time: 78014 ms
   Total hot run time: 55776 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]

2024-07-25 Thread via GitHub


amorynan commented on PR #38150:
URL: https://github.com/apache/doris/pull/38150#issuecomment-2249624954

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](bash) Make sure that the `prof_prefix` defined in `JEMALLOC_CONF` is not overwritten in start_be.sh [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38354:
URL: https://github.com/apache/doris/pull/38354#issuecomment-2249628693

   
   
   ClickBench: Total hot run time: 30.18 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit ba33fa9344811d2e9f805f5d8970ab7abefa517f, 
data reload: false
   
   query1   0.040.040.04
   query2   0.070.040.03
   query3   0.220.050.05
   query4   1.670.070.07
   query5   0.490.490.48
   query6   1.120.720.73
   query7   0.020.010.02
   query8   0.050.040.05
   query9   0.550.480.49
   query10  0.550.540.54
   query11  0.150.110.12
   query12  0.150.120.12
   query13  0.580.580.58
   query14  0.760.790.81
   query15  0.840.810.82
   query16  0.370.370.36
   query17  0.951.050.97
   query18  0.220.210.22
   query19  1.931.801.77
   query20  0.010.010.01
   query21  15.40   0.800.66
   query22  3.967.311.44
   query23  18.29   1.461.28
   query24  2.140.220.22
   query25  0.160.090.08
   query26  0.300.210.21
   query27  0.450.230.23
   query28  13.31   1.021.00
   query29  12.60   3.323.28
   query30  0.270.060.06
   query31  2.860.400.40
   query32  3.270.490.47
   query33  2.852.882.91
   query34  17.15   4.334.33
   query35  4.474.424.40
   query36  0.650.460.46
   query37  0.180.150.15
   query38  0.160.150.15
   query39  0.050.030.04
   query40  0.160.110.12
   query41  0.100.060.06
   query42  0.060.050.05
   query43  0.040.050.04
   Total cold run time: 109.62 s
   Total hot run time: 30.18 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](test) fix unstable refresh catalog/db ut [doris]

2024-07-25 Thread via GitHub


morningman merged PR #38352:
URL: https://github.com/apache/doris/pull/38352


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [fix](test) fix unstable refresh catalog/db ut (#38352)

2024-07-25 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 18e6af5ed7f [fix](test) fix unstable refresh catalog/db ut (#38352)
18e6af5ed7f is described below

commit 18e6af5ed7f094b6e10b9ee710c184900c22e334
Author: Mingyu Chen 
AuthorDate: Thu Jul 25 15:19:44 2024 +0800

[fix](test) fix unstable refresh catalog/db ut (#38352)

Followup #38244
fix some unstable fe ut after set `use_meta_cache` as true
---
 .../src/test/java/org/apache/doris/catalog/RefreshDbTest.java  | 3 +++
 .../test/java/org/apache/doris/datasource/RefreshCatalogTest.java  | 7 +--
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java 
b/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java
index fd920e9632c..fd7a6b1a24b 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/catalog/RefreshDbTest.java
@@ -91,6 +91,9 @@ public class RefreshDbTest extends TestWithFeService {
 }
 long l3 = db1.getLastUpdateTime();
 Assertions.assertTrue(l3 == l2);
+// when use_meta_cache is true, the table will be recreated after 
refresh.
+// so we need to get table again
+table = db1.getTable("tbl11").get();
 Assertions.assertFalse(table.isObjectCreated());
 test1.getDbNullable("db1").getTables();
 Assertions.assertFalse(table.isObjectCreated());
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java 
b/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java
index 87906261b24..34177f49382 100644
--- 
a/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java
+++ 
b/fe/fe-core/src/test/java/org/apache/doris/datasource/RefreshCatalogTest.java
@@ -145,9 +145,12 @@ public class RefreshCatalogTest extends TestWithFeService {
 // not triggered init method
 long l3 = test2.getLastUpdateTime();
 Assertions.assertTrue(l3 == l2);
-// Assertions.assertFalse(table.isObjectCreated());
+// when use_meta_cache is true, the table will be recreated after 
refresh.
+// so we need to get table again
+table = (TestExternalTable) 
test2.getDbNullable("db1").getTable("tbl11").get();
+Assertions.assertFalse(table.isObjectCreated());
 test2.getDbNullable("db1").getTables();
-// Assertions.assertFalse(table.isObjectCreated());
+Assertions.assertFalse(table.isObjectCreated());
 try {
 DdlExecutor.execute(Env.getCurrentEnv(), refreshCatalogStmt);
 } catch (Exception e) {


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249631012

   TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21466) 
Line Coverage: 29.48% (66546/225741)
Region Coverage: 28.95% (34299/118466)
Branch Coverage: 24.83% (17622/70960)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/dcf18ba45f3f802fc50dba752dffe83fb6227428_dcf18ba45f3f802fc50dba752dffe83fb6227428/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](JsonReader) support sepecify `$.` as root column in json path [doris]

2024-07-25 Thread via GitHub


xiaokang commented on code in PR #38213:
URL: https://github.com/apache/doris/pull/38213#discussion_r1690951663


##
be/src/vec/exec/format/json/new_json_reader.cpp:
##
@@ -1657,7 +1657,19 @@ Status 
NewJsonReader::_simdjson_write_columns_by_jsonpath(
 return st;
 }
 }
-if (i >= _parsed_jsonpaths.size() || st.is()) {
+if (i < _parsed_jsonpaths.size() && 
JsonFunctions::is_root_path(_parsed_jsonpaths[i])) {
+// Indicate that the jsonpath is "$.", read the full root json 
object, insert the original doc directly

Review Comment:
   Does this feature, that load all fields in JSON to variant column, match the 
need for users? I think usually users only need some fields in JSON, not all 
fields. 



##
regression-test/data/load_p0/stream_load/test_json_load.out:
##
@@ -250,3 +250,9 @@ testk2_value
 
 -- !select29 --
 10 \N
+
+-- !select30 --
+12345  {"k1":12345,"k2":"1","k3":11,"k4":[1]}  
{"k1":12345,"k2":"1","k3":11,"k4":[1]}  11
+12346  {"k1":12346,"k2":"2","k4":[2]}  
{"k1":12346,"k2":"2","k4":[2]}  \N
+12347  {"k1":12347,"k3":"3","k5":{"k51":1024,"":[1]}}  
{"k1":12347,"k3":"3","k5":{"k51":1024,"":[1]}}  3
+12347  {"k1":12347,"k3":"3","k4":[2]}  
{"k1":12347,"k3":"3","k4":[2]}  3

Review Comment:
   unstable output since k1 is the same for row 3 and 4.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](cloud) ensure error log upload to S3 before returning load result [doris]

2024-07-25 Thread via GitHub


sollhui closed pull request #38353: [fix](cloud) ensure error log upload to S3 
before returning load result
URL: https://github.com/apache/doris/pull/38353


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38150:
URL: https://github.com/apache/doris/pull/38150#issuecomment-2249635976

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](JsonReader) support sepecify `$.` as root column in json path [doris]

2024-07-25 Thread via GitHub


eldenmoon commented on code in PR #38213:
URL: https://github.com/apache/doris/pull/38213#discussion_r1690956423


##
be/src/vec/exec/format/json/new_json_reader.cpp:
##
@@ -1657,7 +1657,19 @@ Status 
NewJsonReader::_simdjson_write_columns_by_jsonpath(
 return st;
 }
 }
-if (i >= _parsed_jsonpaths.size() || st.is()) {
+if (i < _parsed_jsonpaths.size() && 
JsonFunctions::is_root_path(_parsed_jsonpaths[i])) {
+// Indicate that the jsonpath is "$.", read the full root json 
object, insert the original doc directly

Review Comment:
   some user need this feature, map all columns to a variant column



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249639340

   
   
   TPC-H: Total hot run time: 49757 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, 
data reload: false
   
   -- Round 1 --
   q1   17667   438343064306
   q2   2073155 144 144
   q3   10271   191121811911
   q4   10126   126113131261
   q5   8487395839393939
   q6   232 146 123 123
   q7   2051158315931583
   q8   9311275827042704
   q9   10582   10484   10228   10228
   q10  8658353434943494
   q11  427 234 251 234
   q12  471 304 295 295
   q13  18323   393240213932
   q14  351 318 318 318
   q15  509 466 466 466
   q16  677 578 563 563
   q17  1126973 974 973
   q18  7278697968286828
   q19  1793163916111611
   q20  530 307 314 307
   q21  4370410240944094
   q22  531 443 456 443
   Total cold run time: 115844 ms
   Total hot run time: 49757 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4296433042584258
   q2   322 230 221 221
   q3   4146413741104110
   q4   2759273427332733
   q5   7142712570807080
   q6   236 121 119 119
   q7   3239282628482826
   q8   4377449545124495
   q9   16805   16708   16692   16692
   q10  4220425542474247
   q11  796 681 697 681
   q12  1027859 837 837
   q13  6948372537043704
   q14  460 433 432 432
   q15  500 452 460 452
   q16  744 682 665 665
   q17  3871391237993799
   q18  8775869188068691
   q19  1714169916961696
   q20  2364209021132090
   q21  8641843185008431
   q22  1075980 1013980
   Total cold run time: 84457 ms
   Total hot run time: 79239 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]

2024-07-25 Thread via GitHub


liugddx commented on PR #38215:
URL: https://github.com/apache/doris/pull/38215#issuecomment-2249635287

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](function) support split_by_regexp function [doris]

2024-07-25 Thread via GitHub


zhangstar333 commented on PR #38259:
URL: https://github.com/apache/doris/pull/38259#issuecomment-2249638751

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [draft] [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38305:
URL: https://github.com/apache/doris/pull/38305#issuecomment-2249641731

   
   
   TPC-DS: Total hot run time: 174710 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 0f6ea131249f387b333078f31d062d5c6a48c642, 
data reload: false
   
   query1   897 371 370 370
   query2   6434206018861886
   query3   6655204 211 204
   query4   28284   17352   17410   17352
   query5   3755500 460 460
   query6   267 184 173 173
   query7   4572281 281 281
   query8   240 191 191 191
   query9   8489240423702370
   query10  435 285 260 260
   query11  11173   10041   10152   10041
   query12  111 85  97  85
   query13  1639371 361 361
   query14  8579809283418092
   query15  234 169 170 169
   query16  7185422 433 422
   query17  1596560 553 553
   query18  1140288 295 288
   query19  198 156 158 156
   query20  92  82  82  82
   query21  203 126 128 126
   query22  4551417341234123
   query23  34176   33859   33904   33859
   query24  10996   287029422870
   query25  609 414 393 393
   query26  704 153 150 150
   query27  2230274 283 274
   query28  7100214221222122
   query29  899 667 625 625
   query30  281 157 156 156
   query31  1001819 787 787
   query32  94  55  54  54
   query33  758 344 352 344
   query34  950 512 507 507
   query35  888 786 787 786
   query36  1151980 976 976
   query37  166 83  78  78
   query38  2918278528092785
   query39  867 804 816 804
   query40  203 122 117 117
   query41  44  42  43  42
   query42  119 93  106 93
   query43  533 463 501 463
   query44  1177737 723 723
   query45  207 175 177 175
   query46  1120709 737 709
   query47  1872177618271776
   query48  367 287 289 287
   query49  849 405 402 402
   query50  780 385 383 383
   query51  6836664366026602
   query52  101 94  92  92
   query53  354 285 285 285
   query54  933 448 438 438
   query55  75  73  70  70
   query56  285 279 263 263
   query57  1102108610471047
   query58  251 269 270 269
   query59  3035263727342637
   query60  300 270 274 270
   query61  94  93  91  91
   query62  790 655 662 655
   query63  319 287 284 284
   query64  9151224216551655
   query65  3168310031103100
   query66  766 329 323 323
   query67  15631   15050   15113   15050
   query68  4570547 540 540
   query69  662 416 352 352
   query70  1209107711531077
   query71  457 279 276 276
   query72  8059607254505450
   query73  758 325 322 322
   query74  6047573557105710
   query75  3522268526852685
   query76  3215920 938 920
   query77  690 316 302 302
   query78  9593905189068906
   query79  2728522 520 520
   query80  2414520 467 467
   query81  605 222 219 219
   query82  1250137 134 134
   query83  285 168 167 167
   query84  273 91  92  91
   query85  1345309 301 301
   query86  447 316 325 316
   query87  3279311231453112
   query88  4105234723482347
   query89  497 392 381 381
   query90  1944190 186 186
   query91  126 101 100 100
   query92  60  52  50  50
   query93  3466522 516 516
   query94  1285309 281 281
   query95  404 316 314 314
   query96  603 270 273 270
   query97  3234312130273027
   query98  225 205 193 193
   query99  1553126912621262
   Total cold run time: 281889 ms
   Total hot run time: 174710 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the

Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]

2024-07-25 Thread via GitHub


SWJTU-ZhangLei commented on PR #38243:
URL: https://github.com/apache/doris/pull/38243#issuecomment-2249642741

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](compaction) fix missing flush column during compaction [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38356:
URL: https://github.com/apache/doris/pull/38356#issuecomment-2249641771

   
   
   TPC-DS: Total hot run time: 174035 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 903c6282d8858f886bc39855c3fa6b8447ef3474, 
data reload: false
   
   query1   914 373 363 363
   query2   6435197919301930
   query3   6635205 220 205
   query4   28284   17424   17360   17360
   query5   3694482 473 473
   query6   267 184 183 183
   query7   4590288 285 285
   query8   255 188 198 188
   query9   8485240823752375
   query10  442 302 272 272
   query11  12727   991899529918
   query12  113 85  86  85
   query13  1638378 371 371
   query14  10393   764376237623
   query15  214 166 176 166
   query16  7129454 448 448
   query17  1137559 521 521
   query18  1917275 276 275
   query19  195 143 149 143
   query20  89  80  81  80
   query21  204 135 127 127
   query22  4403409239793979
   query23  34121   33763   33619   33619
   query24  10013   293329062906
   query25  595 403 383 383
   query26  703 153 150 150
   query27  2253274 278 274
   query28  5962209020732073
   query29  903 632 627 627
   query30  246 149 148 148
   query31  983 767 766 766
   query32  98  53  62  53
   query33  631 334 330 330
   query34  903 476 506 476
   query35  856 756 740 740
   query36  1147997 990 990
   query37  144 82  81  81
   query38  2965289928662866
   query39  915 844 866 844
   query40  215 127 135 127
   query41  48  47  50  47
   query42  115 103 106 103
   query43  523 492 489 489
   query44  1068732 731 731
   query45  209 174 177 174
   query46  1086737 720 720
   query47  1829179417751775
   query48  387 290 293 290
   query49  874 399 420 399
   query50  774 389 387 387
   query51  6767662766326627
   query52  104 90  98  90
   query53  356 298 289 289
   query54  806 439 439 439
   query55  74  73  73  73
   query56  291 264 262 262
   query57  1131108710591059
   query58  241 239 291 239
   query59  2893276727462746
   query60  324 275 271 271
   query61  100 100 95  95
   query62  799 651 645 645
   query63  335 286 289 286
   query64  9133219316551655
   query65  3192310231043102
   query66  829 322 336 322
   query67  15516   15092   14970   14970
   query68  4522547 545 545
   query69  559 397 356 356
   query70  1222114611821146
   query71  387 272 272 272
   query72  7135545060905450
   query73  752 324 320 320
   query74  6310566456515651
   query75  3380266027252660
   query76  2137989 904 904
   query77  438 307 295 295
   query78  10847   10182   91339133
   query79  4547526 521 521
   query80  1061466 531 466
   query81  559 227 215 215
   query82  742 137 133 133
   query83  197 166 167 166
   query84  273 89  82  82
   query85  1147329 299 299
   query86  460 310 330 310
   query87  3254309730923092
   query88  4454235323772353
   query89  506 397 383 383
   query90  1821192 190 190
   query91  127 99  97  97
   query92  65  48  52  48
   query93  5247523 511 511
   query94  1166287 295 287
   query95  409 309 312 309
   query96  610 277 274 274
   query97  3208304730253025
   query98  223 195 188 188
   query99  1510124612591246
   Total cold run time: 282892 ms
   Total hot run time: 174035 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the 

Re: [PR] [Bug](dry-run) make dryrun result set collect multi batch's QueryStatistics [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38303:
URL: https://github.com/apache/doris/pull/38303#issuecomment-2249643934

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38215:
URL: https://github.com/apache/doris/pull/38215#issuecomment-2249645732

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](compaction) fix missing flush column during compaction [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38356:
URL: https://github.com/apache/doris/pull/38356#issuecomment-2249651268

   
   
   ClickBench: Total hot run time: 30.17 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 903c6282d8858f886bc39855c3fa6b8447ef3474, 
data reload: false
   
   query1   0.040.030.03
   query2   0.080.040.04
   query3   0.230.040.04
   query4   1.680.070.07
   query5   0.490.480.48
   query6   1.130.770.74
   query7   0.020.010.01
   query8   0.050.050.04
   query9   0.560.500.49
   query10  0.540.540.54
   query11  0.160.110.11
   query12  0.150.120.12
   query13  0.600.590.58
   query14  0.760.790.78
   query15  0.870.820.82
   query16  0.360.360.37
   query17  1.020.990.98
   query18  0.220.220.21
   query19  1.801.691.66
   query20  0.020.010.01
   query21  15.39   0.760.66
   query22  4.706.991.64
   query23  18.34   1.331.18
   query24  2.120.230.23
   query25  0.150.090.08
   query26  0.300.200.21
   query27  0.460.230.23
   query28  13.30   1.021.00
   query29  12.55   3.283.26
   query30  0.250.060.05
   query31  2.880.390.39
   query32  3.250.480.47
   query33  2.902.912.93
   query34  16.97   4.354.32
   query35  4.384.404.44
   query36  0.640.460.47
   query37  0.180.150.16
   query38  0.160.150.15
   query39  0.050.040.03
   query40  0.160.120.12
   query41  0.100.050.04
   query42  0.060.050.05
   query43  0.040.040.04
   Total cold run time: 110.11 s
   Total hot run time: 30.17 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [draft] [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38305:
URL: https://github.com/apache/doris/pull/38305#issuecomment-2249651278

   
   
   ClickBench: Total hot run time: 30.48 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 0f6ea131249f387b333078f31d062d5c6a48c642, 
data reload: false
   
   query1   0.050.040.04
   query2   0.080.040.04
   query3   0.220.050.04
   query4   1.680.070.08
   query5   0.480.470.48
   query6   1.120.720.72
   query7   0.020.010.02
   query8   0.050.040.05
   query9   0.540.500.49
   query10  0.550.540.55
   query11  0.140.110.11
   query12  0.150.120.13
   query13  0.600.580.58
   query14  0.750.780.80
   query15  0.860.810.81
   query16  0.360.380.37
   query17  1.041.031.06
   query18  0.230.220.22
   query19  1.901.721.81
   query20  0.010.010.00
   query21  15.40   0.790.66
   query22  4.407.401.78
   query23  18.35   1.401.25
   query24  2.150.220.24
   query25  0.150.090.09
   query26  0.310.220.21
   query27  0.450.230.23
   query28  13.20   1.011.01
   query29  12.59   3.303.25
   query30  0.240.060.05
   query31  2.880.390.38
   query32  3.260.470.47
   query33  2.982.922.88
   query34  17.15   4.354.37
   query35  4.474.434.40
   query36  0.650.460.47
   query37  0.180.160.15
   query38  0.160.150.14
   query39  0.040.040.04
   query40  0.150.120.13
   query41  0.090.050.05
   query42  0.060.050.05
   query43  0.050.050.04
   Total cold run time: 110.19 s
   Total hot run time: 30.48 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (cloud-merge) Supports online capacity expansion and contraction [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #37484:
URL: https://github.com/apache/doris/pull/37484#issuecomment-2249653939

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [enhance](RateLimit) Add bvar to monitor OSS's rate limit sleep time and failure time [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on code in PR #38294:
URL: https://github.com/apache/doris/pull/38294#discussion_r1690979539


##
cloud/src/recycler/s3_accessor.h:
##
@@ -17,10 +17,13 @@
 
 #pragma once
 
+#include 

Review Comment:
   warning: 'bvar/latency_recorder.h' file not found [clang-diagnostic-error]
   ```cpp
   #include 
^
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] (cloud-merge) Supports online capacity expansion and contraction [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #37484:
URL: https://github.com/apache/doris/pull/37484#issuecomment-2249653992

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on code in PR #38243:
URL: https://github.com/apache/doris/pull/38243#discussion_r1690977477


##
cloud/src/meta-service/txn_lazy_committer.h:
##
@@ -0,0 +1,65 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include 

Review Comment:
   warning: 'gen_cpp/cloud.pb.h' file not found [clang-diagnostic-error]
   ```cpp
   #include 
^
   ```
   



##
cloud/src/meta-service/txn_lazy_committer.cpp:
##
@@ -0,0 +1,208 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "txn_lazy_committer.h"
+
+#include "common/logging.h"
+#include "common/util.h"
+#include "meta-service/keys.h"
+#include "meta-service/meta_service_helper.h"
+
+namespace doris::cloud {
+
+extern void scan_tmp_rowset(
+const std::string& instance_id, int64_t txn_id, std::shared_ptr 
txn_kv,
+MetaServiceCode& code, std::string& msg, int64_t* db_id,
+std::vector>* 
tmp_rowsets_meta);
+
+extern void convert_tmp_rowsets(
+const std::string& instance_id, int64_t txn_id, std::shared_ptr 
txn_kv,
+MetaServiceCode& code, std::string& msg, int64_t db_id,
+std::vector>& 
tmp_rowsets_meta,
+std::unordered_map& tablet_ids);
+
+extern void make_committed_txn_visible(const std::string& instance_id, int64_t 
db_id,
+   int64_t txn_id, std::shared_ptr 
txn_kv,
+   MetaServiceCode& code, std::string& 
msg);
+
+TxnLazyCommitTask::TxnLazyCommitTask(const std::string& instance_id, int64_t 
txn_id,

Review Comment:
   warning: function 'TxnLazyCommitTask' exceeds recommended size/complexity 
thresholds [readability-function-size]
   ```cpp
   TxnLazyCommitTask::TxnLazyCommitTask(const std::string& instance_id, int64_t 
txn_id,
  ^
   ```
   
   Additional context
   
   **cloud/src/meta-service/txn_lazy_committer.cpp:41:** 130 lines including 
whitespace and comments (threshold 80)
   ```cpp
   TxnLazyCommitTask::TxnLazyCommitTask(const std::string& instance_id, int64_t 
txn_id,
  ^
   ```
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](group commit) Pick some group commit pr [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38320:
URL: https://github.com/apache/doris/pull/38320#issuecomment-2249654974

   TeamCity be ut coverage result:
Function Coverage: 36.44% (9254/25398) 
Line Coverage: 28.00% (75644/270191)
Region Coverage: 26.81% (38878/145023)
Branch Coverage: 23.54% (19731/83808)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/c65a8ccd4284eb7473106b27b3391dffe7878073_c65a8ccd4284eb7473106b27b3391dffe7878073/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]

2024-07-25 Thread via GitHub


liaoxin01 opened a new pull request, #38359:
URL: https://github.com/apache/doris/pull/38359

   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38359:
URL: https://github.com/apache/doris/pull/38359#issuecomment-2249657240

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]

2024-07-25 Thread via GitHub


liaoxin01 commented on PR #38359:
URL: https://github.com/apache/doris/pull/38359#issuecomment-2249657417

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38150:
URL: https://github.com/apache/doris/pull/38150#issuecomment-2249657999

   
   
   TPC-H: Total hot run time: 49917 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit cb1808a3924f62b57926e08509bb31647acff685, 
data reload: false
   
   -- Round 1 --
   q1   18138   444343294329
   q2   2074156 157 156
   q3   10298   187619051876
   q4   10102   125013011250
   q5   8420383039353830
   q6   239 125 150 125
   q7   2072165316191619
   q8   9594273227402732
   q9   12916   10304   10308   10304
   q10  8612352335563523
   q11  413 239 258 239
   q12  473 303 303 303
   q13  18360   396440203964
   q14  346 339 322 322
   q15  513 456 456 456
   q16  660 579 577 577
   q17  1138948 945 945
   q18  7306696070006960
   q19  1797166216021602
   q20  526 318 314 314
   q21  4404414240504050
   q22  530 441 449 441
   Total cold run time: 118931 ms
   Total hot run time: 49917 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4375432943034303
   q2   322 230 222 222
   q3   4221415041464146
   q4   2754273527452735
   q5   7202712971607129
   q6   243 125 123 123
   q7   3191278428052784
   q8   4381448544344434
   q9   16830   16761   16750   16750
   q10  4221427842594259
   q11  751 697 667 667
   q12  1032882 864 864
   q13  6709379137563756
   q14  446 424 425 424
   q15  512 454 459 454
   q16  729 681 674 674
   q17  3920391838703870
   q18  8805878387818781
   q19  1761166817061668
   q20  2382217220842084
   q21  8546840985248409
   q22  1046101010081008
   Total cold run time: 84379 ms
   Total hot run time: 79544 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38150:
URL: https://github.com/apache/doris/pull/38150#issuecomment-2249661323

   TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21466) 
Line Coverage: 29.48% (66552/225741)
Region Coverage: 28.96% (34303/118467)
Branch Coverage: 24.84% (17625/70960)
Coverage Report: 
http://coverage.selectdb-in.cc/coverage/cb1808a3924f62b57926e08509bb31647acff685_cb1808a3924f62b57926e08509bb31647acff685/report/index.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Variant) fix outof range exception after calling function element_at for variant type [doris]

2024-07-25 Thread via GitHub


eldenmoon merged PR #38318:
URL: https://github.com/apache/doris/pull/38318


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38215:
URL: https://github.com/apache/doris/pull/38215#issuecomment-2249666440

   
   
   TPC-H: Total hot run time: 40224 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit ec848b55520d1a4c70c338b096ea396162b3fed2, 
data reload: false
   
   -- Round 1 --
   q1   18849   444044114411
   q2   2799198 201 198
   q3   12542   118711531153
   q4   10592   829 712 712
   q5   7590268926712671
   q6   229 141 137 137
   q7   953 598 593 593
   q8   9219207820972078
   q9   8620658865916588
   q10  8805377837583758
   q11  448 232 233 232
   q12  401 220 220 220
   q13  17749   298129722972
   q14  271 241 237 237
   q15  524 479 487 479
   q16  484 379 374 374
   q17  961 722 672 672
   q18  8119758174657465
   q19  8546146614931466
   q20  769 328 320 320
   q21  4885320233103202
   q22  347 288 286 286
   Total cold run time: 123702 ms
   Total hot run time: 40224 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4426425642414241
   q2   377 271 270 270
   q3   2999273628012736
   q4   1910158515701570
   q5   5293534053445340
   q6   219 134 135 134
   q7   2116173017521730
   q8   3252333833083308
   q9   8431841484498414
   q10  3882365137343651
   q11  565 510 482 482
   q12  746 603 591 591
   q13  17409   301129612961
   q14  306 264 266 264
   q15  525 477 468 468
   q16  466 425 404 404
   q17  1749148914721472
   q18  7582740273547354
   q19  1662166115991599
   q20  2020178817891788
   q21  4836464845884588
   q22  580 488 502 488
   Total cold run time: 71351 ms
   Total hot run time: 53853 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249662215

   
   
   TPC-DS: Total hot run time: 203935 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, 
data reload: false
   
   query1   930 424 382 382
   query2   6528277226392639
   query3   6922210 206 206
   query4   19794   18058   17930   17930
   query5   19714   649464796479
   query6   284 220 226 220
   query7   4162300 303 300
   query8   439 478 427 427
   query9   3135268126142614
   query10  409 317 307 307
   query11  11311   10719   10775   10719
   query12  121 81  76  76
   query13  5604698 712 698
   query14  17701   13041   13298   13041
   query15  364 236 245 236
   query16  6486295 271 271
   query17  16911463897 897
   query18  2289416 406 406
   query19  215 149 151 149
   query20  80  77  80  77
   query21  187 100 96  96
   query22  5352507550295029
   query23  32416   31909   31903   31903
   query24  7139645664946456
   query25  517 449 430 430
   query26  528 164 155 155
   query27  1905298 298 298
   query28  6176237423412341
   query29  3012286126352635
   query30  248 167 172 167
   query31  887 731 720 720
   query32  71  62  62  62
   query33  410 250 264 250
   query34  851 467 482 467
   query35  1103904 901 901
   query36  1222119711371137
   query37  89  60  62  60
   query38  3069295529752955
   query39  1374133213201320
   query40  206 106 90  90
   query41  46  45  45  45
   query42  83  82  80  80
   query43  812 630 785 630
   query44  1113713 703 703
   query45  248 235 231 231
   query46  1208938 973 938
   query47  1849172319201723
   query48  1001707 711 707
   query49  613 371 372 371
   query50  862 615 649 615
   query51  4675470246704670
   query52  103 81  98  81
   query53  442 346 316 316
   query54  2655243424132413
   query55  97  86  89  86
   query56  225 237 201 201
   query57  1242113911561139
   query58  219 193 210 193
   query59  4275385540733855
   query60  209 205 214 205
   query61  97  105 93  93
   query62  784 449 486 449
   query63  484 344 337 337
   query64  2554154914771477
   query65  3650352335723523
   query66  781 382 368 368
   query67  15598   16732   16067   16067
   query68  8288657 662 657
   query69  554 359 346 346
   query70  1553153513791379
   query71  374 304 327 304
   query72  6462347934743474
   query73  733 333 325 325
   query74  6300580258515802
   query75  4532366637053666
   query76  4638118111901181
   query77  593 268 257 257
   query78  12615   11661   12033   11661
   query79  9203658 659 658
   query80  1919400 409 400
   query81  504 236 232 232
   query82  1689101 96  96
   query83  173 133 138 133
   query84  261 71  69  69
   query85  1118324 336 324
   query86  324 297 298 297
   query87  3263306030493049
   query88  5229233823462338
   query89  387 290 285 285
   query90  1721206 210 206
   query91  171 154 138 138
   query92  59  54  56  54
   query93  5298542 613 542
   query94  673 209 211 209
   query95  1092106710471047
   query96  636 338 333 333
   query97  6515637062826282
   query98  193 173 177 173
   query99  2922956 974 956
   Total cold run time: 310858 ms
   Total hot run time: 203935 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [Fix](Variant) fix outof range exception after calling function element_at for variant type [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38318:
URL: https://github.com/apache/doris/pull/38318#issuecomment-2249664531

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated: [Fix](Variant) fix outof range exception after calling function element_at for variant type (#38318)

2024-07-25 Thread eldenmoon
This is an automated email from the ASF dual-hosted git repository.

eldenmoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 20c4cadcbbe [Fix](Variant) fix outof range exception after calling 
function element_at for variant type (#38318)
20c4cadcbbe is described below

commit 20c4cadcbbeb966c32423a01d0f8dda1e5176ac3
Author: lihangyu <15605149...@163.com>
AuthorDate: Thu Jul 25 15:40:03 2024 +0800

[Fix](Variant) fix outof range exception after calling function element_at 
for variant type (#38318)

…nt_at for variant type

src subcolumns empty but src row count may not be 0, so need to padding
rows to at least src.size() and finlize at last

```
[CANCELLED][E6] Parameter out of bound in IColumnStr::insert_range_from 
me !
```
---
 be/src/vec/functions/function_variant_element.cpp |  10 +-
 regression-test/data/variant_p0/rqg/rqg3.out  | 130 ++
 regression-test/suites/variant_p0/rqg/rqg3.sql|  43 +++
 3 files changed, 182 insertions(+), 1 deletion(-)

diff --git a/be/src/vec/functions/function_variant_element.cpp 
b/be/src/vec/functions/function_variant_element.cpp
index 84ddc3b8046..faa56ccb1e3 100644
--- a/be/src/vec/functions/function_variant_element.cpp
+++ b/be/src/vec/functions/function_variant_element.cpp
@@ -28,6 +28,7 @@
 #include "common/status.h"
 #include "exprs/json_functions.h"
 #include "simdjson.h"
+#include "util/defer_op.h"
 #include "vec/columns/column.h"
 #include "vec/columns/column_nullable.h"
 #include "vec/columns/column_object.h"
@@ -102,8 +103,11 @@ private:
 static Status get_element_column(const ColumnObject& src, const ColumnPtr& 
index_column,
  ColumnPtr* result) {
 std::string field_name = index_column->get_data_at(0).to_string();
+Defer finalize([&]() { (*result)->assume_mutable()->finalize(); });
 if (src.empty()) {
 *result = ColumnObject::create(true);
+// src subcolumns empty but src row count may not be 0
+(*result)->assume_mutable()->insert_many_defaults(src.size());
 return Status::OK();
 }
 if (src.is_scalar_variant() &&
@@ -135,8 +139,10 @@ private:
 PathInData path(field_name);
 ColumnObject::Subcolumns subcolumns = 
mutable_ptr->get_subcolumns();
 const auto* node = subcolumns.find_exact(path);
-auto result_col = ColumnObject::create(true, false /*should not 
create root*/);
+MutableColumnPtr result_col;
 if (node != nullptr) {
+// Create without root, since root will be added
+result_col = ColumnObject::create(true, false /*should not 
create root*/);
 std::vector nodes;
 PathsInData paths;
 ColumnObject::Subcolumns::get_leaves_of_node(node, nodes, 
paths);
@@ -162,6 +168,8 @@ private:
 auto container = 
ColumnObject::create(std::move(new_subcolumns), true);
 result_col->insert_range_from(*container, 0, 
container->size());
 } else {
+// Create with root, otherwise the root type maybe type Nothing
+result_col = ColumnObject::create(true);
 result_col->insert_many_defaults(src.size());
 }
 *result = result_col->get_ptr();
diff --git a/regression-test/data/variant_p0/rqg/rqg3.out 
b/regression-test/data/variant_p0/rqg/rqg3.out
new file mode 100644
index 000..d39b0e288dc
--- /dev/null
+++ b/regression-test/data/variant_p0/rqg/rqg3.out
@@ -0,0 +1,130 @@
+-- This file is automatically generated. You should know what you did if you 
want to edit this
+-- !rqg3 --
+0
+
+-- !rqg3_2 --
+0
+
+-- !rqg3_3 --
+0
+
+-- !rqg3_4 --
+0
+
+-- !rqg3_5 --
+0
+
+-- !rqg3_6 --
+0
+
+-- !rqg3_7 --
+0
+
+-- !rqg3_8 --
+0
+
+-- !rqg3_9 --
+0
+
+-- !rqg3_10 --
+0
+
+-- !rqg3_11 --
+10
+
+-- !rqg3_12 --
+10
+
+-- !rqg3_13 --
+10
+
+-- !rqg3_14 --
+10
+
+-- !rqg3_15 --
+10
+
+-- !rqg3_16 --
+10
+
+-- !rqg3_17 --
+10
+
+-- !rqg3_18 --
+10
+
+-- !rqg3_19 --
+21
+
+-- !rqg3_20 --
+21
+
+-- !rqg3_21 --
+21
+
+-- !rqg3_22 --
+21
+
+-- !rqg3_23 --
+21
+
+-- !rqg3_24 --
+21
+
+-- !rqg3_25 --
+25
+
+-- !rqg3_26 --
+25
+
+-- !rqg3_27 --
+25
+
+-- !rqg3_28 --
+25
+
+-- !rqg3_29 --
+25
+
+-- !rqg3_30 --
+25
+
+-- !rqg3_31 --
+6
+
+-- !rqg3_32 --
+6
+
+-- !rqg3_33 --
+6
+
+-- !rqg3_34 --
+6
+
+-- !rqg3_35 --
+6
+
+-- !rqg3_36 --
+6
+
+-- !rqg3_37 --
+7
+
+-- !rqg3_38 --
+7
+
+-- !rqg3_39 --
+7
+
+-- !rqg3_40 --
+7
+
+-- !rqg3_41 --
+7
+
+-- !rqg3_42 --
+7
+
+-- !rqg3_43 --
+0  \N  c   \N  \N  \N
+
diff --git a/regression-test/suites/variant_p0/rqg/rqg3.sql 
b/regression-test/suites/variant_p0/rqg/rqg3.sql
new file mode 100644
index 000..d7fde8f1a1c
--- /dev/null
+++ b/regression-te

Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38359:
URL: https://github.com/apache/doris/pull/38359#issuecomment-2249669519

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improve](columns)replace logFatal with exception and del some useless code in columns [doris]

2024-07-25 Thread via GitHub


eldenmoon commented on code in PR #38035:
URL: https://github.com/apache/doris/pull/38035#discussion_r1690991296


##
be/src/common/exception.h:
##
@@ -139,3 +139,48 @@ inline const std::string& Exception::to_string() const {
 }  
  \
 }  
  \
 } while (0);
+
+#define HANDLE_EXCEPTION_IF_CATCH_EXCEPTION(stmt, handle_exception)
  \
+do {   
  \
+try {  
  \
+doris::enable_thread_catch_bad_alloc++;
  \
+Defer defer {[&]() { doris::enable_thread_catch_bad_alloc--; }};   
  \
+{  
  \
+Status _status_ = (stmt);  
  \
+if (UNLIKELY(!_status_.ok())) {
  \
+return _status_;   
  \

Review Comment:
   call handle_exception() after `!_status_.ok()`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improve](columns)replace logFatal with exception and del some useless code in columns [doris]

2024-07-25 Thread via GitHub


eldenmoon commented on code in PR #38035:
URL: https://github.com/apache/doris/pull/38035#discussion_r1690991296


##
be/src/common/exception.h:
##
@@ -139,3 +139,48 @@ inline const std::string& Exception::to_string() const {
 }  
  \
 }  
  \
 } while (0);
+
+#define HANDLE_EXCEPTION_IF_CATCH_EXCEPTION(stmt, handle_exception)
  \
+do {   
  \
+try {  
  \
+doris::enable_thread_catch_bad_alloc++;
  \
+Defer defer {[&]() { doris::enable_thread_catch_bad_alloc--; }};   
  \
+{  
  \
+Status _status_ = (stmt);  
  \
+if (UNLIKELY(!_status_.ok())) {
  \
+return _status_;   
  \

Review Comment:
   call handle_exception() after `!_status_.ok()`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249671431

   
   
   ClickBench: Total hot run time: 30.46 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428, 
data reload: false
   
   query1   0.030.020.03
   query2   0.070.030.02
   query3   0.250.050.04
   query4   1.770.070.07
   query5   0.540.530.52
   query6   1.340.620.61
   query7   0.020.010.01
   query8   0.030.030.03
   query9   0.520.480.49
   query10  0.530.530.54
   query11  0.120.090.09
   query12  0.110.080.09
   query13  0.620.610.61
   query14  0.770.780.81
   query15  0.790.760.76
   query16  0.360.360.37
   query17  1.021.010.98
   query18  0.230.250.25
   query19  1.881.851.84
   query20  0.020.010.01
   query21  15.46   0.570.56
   query22  2.002.131.40
   query23  17.27   1.030.79
   query24  4.242.981.07
   query25  0.370.100.06
   query26  0.640.150.16
   query27  0.040.040.03
   query28  6.880.770.73
   query29  12.60   2.282.34
   query30  0.610.530.53
   query31  2.830.390.38
   query32  3.350.490.50
   query33  3.113.063.06
   query34  15.23   4.784.78
   query35  4.844.844.82
   query36  1.051.011.02
   query37  0.060.040.05
   query38  0.030.020.02
   query39  0.020.010.01
   query40  0.160.130.14
   query41  0.060.010.02
   query42  0.020.020.01
   query43  0.030.010.01
   Total cold run time: 101.92 s
   Total hot run time: 30.46 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](mysql)fix mysql row buffer open_dynamic_mode make _pos pointer out range of _buf [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38149:
URL: https://github.com/apache/doris/pull/38149#issuecomment-2249677118

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit dcf18ba45f3f802fc50dba752dffe83fb6227428 with 
default session variables
   Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:  58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:  31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:   21.4 seconds inserted 1000 Rows, about 467K 
ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38150:
URL: https://github.com/apache/doris/pull/38150#issuecomment-2249681154

   
   
   TPC-DS: Total hot run time: 203766 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit cb1808a3924f62b57926e08509bb31647acff685, 
data reload: false
   
   query1   940 424 380 380
   query2   6550272028272720
   query3   6930219 218 218
   query4   20233   18161   17912   17912
   query5   19727   655966466559
   query6   290 213 223 213
   query7   4160308 325 308
   query8   411 443 442 442
   query9   3186272226542654
   query10  420 304 284 284
   query11  11293   10754   10712   10712
   query12  123 84  73  73
   query13  5602723 701 701
   query14  17634   13373   13444   13373
   query15  360 238 242 238
   query16  6484287 261 261
   query17  17141441872 872
   query18  2327436 415 415
   query19  213 146 149 146
   query20  83  79  80  79
   query21  193 99  94  94
   query22  5228501549844984
   query23  32643   32120   32126   32120
   query24  6938647564746474
   query25  523 436 439 436
   query26  534 165 160 160
   query27  1884308 307 307
   query28  6177238423632363
   query29  2785275526962696
   query30  241 166 175 166
   query31  937 732 764 732
   query32  69  67  63  63
   query33  411 256 262 256
   query34  847 500 494 494
   query35  1125934 934 934
   query36  1332111610801080
   query37  89  65  60  60
   query38  3111290729582907
   query39  1364132113391321
   query40  201 90  97  90
   query41  48  45  44  44
   query42  97  89  86  86
   query43  713 706 658 658
   query44  1149717 727 717
   query45  244 236 242 236
   query46  1221975 966 966
   query47  1817183917411741
   query48  1024719 719 719
   query49  643 387 378 378
   query50  862 613 589 589
   query51  4729471446614661
   query52  96  87  91  87
   query53  445 325 318 318
   query54  2639249424752475
   query55  90  83  84  83
   query56  248 230 221 221
   query57  1248110611321106
   query58  209 215 208 208
   query59  4446413038983898
   query60  225 231 212 212
   query61  99  93  102 93
   query62  835 464 444 444
   query63  492 344 344 344
   query64  2507153814011401
   query65  3672356235733562
   query66  830 392 389 389
   query67  17329   16153   15017   15017
   query68  8609645 659 645
   query69  568 356 357 356
   query70  1707138814351388
   query71  394 303 328 303
   query72  6604344934973449
   query73  737 328 324 324
   query74  6356586359705863
   query75  4978373537433735
   query76  5026116111501150
   query77  756 272 259 259
   query78  12583   11632   11684   11632
   query79  7157633 634 633
   query80  1202416 412 412
   query81  501 237 232 232
   query82  1496101 99  99
   query83  178 140 134 134
   query84  258 73  70  70
   query85  1021331 335 331
   query86  342 291 297 291
   query87  3240302429812981
   query88  5076237323622362
   query89  353 285 315 285
   query90  1880207 210 207
   query91  176 140 141 140
   query92  58  54  53  53
   query93  3047580 539 539
   query94  729 216 213 213
   query95  1138106810561056
   query96  645 335 331 331
   query97  6557641763656365
   query98  187 175 168 168
   query99  2863810 860 810
   Total cold run time: 309691 ms
   Total hot run time: 203766 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t

Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38215:
URL: https://github.com/apache/doris/pull/38215#issuecomment-2249686411

   
   
   TPC-DS: Total hot run time: 172709 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit ec848b55520d1a4c70c338b096ea396162b3fed2, 
data reload: false
   
   query1   926 371 359 359
   query2   6448193518461846
   query3   6651210 211 210
   query4   28433   17694   17401   17401
   query5   4178480 480 480
   query6   266 169 165 165
   query7   4598293 284 284
   query8   259 204 200 200
   query9   8487242523832383
   query10  441 277 269 269
   query11  10593   999010030   9990
   query12  134 92  83  83
   query13  1640387 369 369
   query14  9442732476107324
   query15  217 166 163 163
   query16  7788466 440 440
   query17  1250546 528 528
   query18  1907271 291 271
   query19  194 146 146 146
   query20  89  83  81  81
   query21  205 136 130 130
   query22  4410404740414041
   query23  33935   33416   33180   33180
   query24  12099   288327822782
   query25  668 372 396 372
   query26  1822149 150 149
   query27  2873267 271 267
   query28  7545200819961996
   query29  1136616 613 613
   query30  289 152 147 147
   query31  979 755 751 751
   query32  94  51  53  51
   query33  773 327 334 327
   query34  908 469 496 469
   query35  843 739 722 722
   query36  938 944 938
   query37  210 78  77  77
   query38  2842272627482726
   query39  858 825 820 820
   query40  266 122 116 116
   query41  51  46  46  46
   query42  126 108 110 108
   query43  508 473 476 473
   query44  1255776 724 724
   query45  207 172 174 172
   query46  1100748 723 723
   query47  1879178717921787
   query48  372 297 284 284
   query49  1191421 413 413
   query50  800 392 399 392
   query51  6792666266396639
   query52  109 94  94  94
   query53  352 287 281 281
   query54  1027447 447 447
   query55  79  73  73  73
   query56  302 265 265 265
   query57  1177104910721049
   query58  258 250 250 250
   query59  2787283828212821
   query60  312 275 307 275
   query61  97  94  95  94
   query62  822 663 646 646
   query63  321 290 283 283
   query64  10483   221016421642
   query65  3454310631683106
   query66  1353346 323 323
   query67  15330   15155   14889   14889
   query68  4556544 560 544
   query69  483 339 352 339
   query70  1203108210881082
   query71  400 349 271 271
   query72  7008545861235458
   query73  760 326 325 325
   query74  6089559256655592
   query75  3430269327022693
   query76  2765989 917 917
   query77  489 305 298 298
   query78  10863   902596539025
   query79  2595530 521 521
   query80  2126470 464 464
   query81  586 225 227 225
   query82  826 134 133 133
   query83  301 170 172 170
   query84  273 89  88  88
   query85  2155321 301 301
   query86  387 325 286 286
   query87  3309307130913071
   query88  4303236723652365
   query89  474 369 380 369
   query90  1616196 192 192
   query91  132 100 103 100
   query92  64  50  51  50
   query93  2885518 509 509
   query94  692 283 292 283
   query95  403 316 321 316
   query96  601 274 271 271
   query97  3226303630373036
   query98  209 201 195 195
   query99  1598125512741255
   Total cold run time: 287158 ms
   Total hot run time: 172709 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the

Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38150:
URL: https://github.com/apache/doris/pull/38150#issuecomment-2249691493

   
   
   ClickBench: Total hot run time: 30.47 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit cb1808a3924f62b57926e08509bb31647acff685, 
data reload: false
   
   query1   0.020.030.02
   query2   0.070.030.03
   query3   0.250.050.04
   query4   1.800.070.07
   query5   0.540.520.52
   query6   1.230.880.61
   query7   0.020.010.01
   query8   0.030.020.02
   query9   0.530.480.48
   query10  0.540.540.54
   query11  0.120.090.08
   query12  0.120.090.10
   query13  0.620.620.60
   query14  0.790.790.79
   query15  0.760.750.76
   query16  0.370.360.38
   query17  1.001.031.02
   query18  0.230.240.25
   query19  1.851.851.85
   query20  0.010.020.01
   query21  15.48   0.560.56
   query22  2.282.161.73
   query23  17.11   1.130.95
   query24  7.970.720.54
   query25  0.380.090.07
   query26  0.720.160.14
   query27  0.040.040.05
   query28  5.970.730.75
   query29  12.68   2.292.31
   query30  0.550.530.52
   query31  2.820.380.38
   query32  3.380.490.49
   query33  3.073.043.07
   query34  15.26   4.794.76
   query35  4.834.834.85
   query36  1.051.011.01
   query37  0.060.040.05
   query38  0.040.020.02
   query39  0.030.020.02
   query40  0.160.140.14
   query41  0.070.020.01
   query42  0.020.020.01
   query43  0.020.020.02
   Total cold run time: 104.89 s
   Total hot run time: 30.47 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]

2024-07-25 Thread via GitHub


Mryange opened a new pull request, #38360:
URL: https://github.com/apache/doris/pull/38360

   ## Proposed changes
   
   The previous code only considered the x86 architecture, and 
_mm_movemask_epi8 does not have a corresponding instruction in ARM. According 
to the article below, we need to abstract the overall logic.
   
[link](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/porting-x86-vector-bitmask-optimizations-to-arm-neon?CommentId=af187ac6-ae00-4e4d-bbf0-e142187aa92e)
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38360:
URL: https://github.com/apache/doris/pull/38360#issuecomment-2249695441

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]

2024-07-25 Thread via GitHub


Mryange commented on PR #38360:
URL: https://github.com/apache/doris/pull/38360#issuecomment-2249695589

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [bugfix](arrow) Fix time zone issues and accuracy issues [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38215:
URL: https://github.com/apache/doris/pull/38215#issuecomment-2249696750

   
   
   ClickBench: Total hot run time: 30.68 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit ec848b55520d1a4c70c338b096ea396162b3fed2, 
data reload: false
   
   query1   0.040.030.03
   query2   0.100.040.03
   query3   0.220.050.05
   query4   1.680.080.07
   query5   0.500.490.49
   query6   1.140.740.73
   query7   0.020.010.01
   query8   0.050.040.05
   query9   0.550.490.49
   query10  0.540.550.55
   query11  0.160.100.11
   query12  0.150.130.12
   query13  0.590.590.59
   query14  0.760.790.77
   query15  0.850.810.84
   query16  0.360.350.37
   query17  0.951.000.94
   query18  0.220.200.21
   query19  1.791.821.73
   query20  0.010.000.01
   query21  15.39   0.800.64
   query22  4.147.191.94
   query23  18.29   1.391.30
   query24  2.100.230.23
   query25  0.160.090.09
   query26  0.300.210.21
   query27  0.460.230.23
   query28  13.23   1.021.01
   query29  12.60   3.283.25
   query30  0.250.060.06
   query31  2.880.390.39
   query32  3.310.480.46
   query33  2.842.972.92
   query34  16.81   4.424.41
   query35  4.494.394.43
   query36  0.660.470.49
   query37  0.190.160.16
   query38  0.160.140.14
   query39  0.040.030.03
   query40  0.150.120.12
   query41  0.090.050.06
   query42  0.060.050.04
   query43  0.040.050.04
   Total cold run time: 109.32 s
   Total hot run time: 30.68 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](func)fix array_with_const with larger than max_array_size [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38150:
URL: https://github.com/apache/doris/pull/38150#issuecomment-2249697525

   
   Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   ```
   Load test result on commit cb1808a3924f62b57926e08509bb31647acff685 with 
default session variables
   Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
   Stream load orc:  58 seconds loaded 1101869774 Bytes, about 18 MB/s
   Stream load parquet:  31 seconds loaded 861443392 Bytes, about 26 MB/s
   Insert into select:   21.3 seconds inserted 1000 Rows, about 469K 
ops/s
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]

2024-07-25 Thread via GitHub


LiBinfeng-01 opened a new pull request, #38361:
URL: https://github.com/apache/doris/pull/38361

   cherry-pick: #38135 
   
   explain shape plan select * from table2 where Null; explain shape plan 
select * from table2 where false; in this case, null literal can be regard as 
same as false literal
   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38361:
URL: https://github.com/apache/doris/pull/38361#issuecomment-2249702599

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]

2024-07-25 Thread via GitHub


LiBinfeng-01 commented on PR #38361:
URL: https://github.com/apache/doris/pull/38361#issuecomment-2249702724

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [I] [Bug] be必须要关闭swap才能运行,我这边是在自己的笔记本上跑,无法提供大内存 [doris]

2024-07-25 Thread via GitHub


xyismy commented on issue #37767:
URL: https://github.com/apache/doris/issues/37767#issuecomment-2249706124

   本地测试可以自行修改 bin/start_be.sh 脚本,找到下面代码注释它( doris 版本不同,代码不同,可以自己找对应的 swap 
检查代码),启动 be 时就不会检查 swap 了
   
 if [[ "$(swapon -s | wc -l)" -gt 1 ]]; then
 echo "Please disable swap memory before installation."
 exit 1
 fi


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38360:
URL: https://github.com/apache/doris/pull/38360#issuecomment-2249707377

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]

2024-07-25 Thread via GitHub


LiBinfeng-01 opened a new pull request, #38362:
URL: https://github.com/apache/doris/pull/38362

   cherry-pick: #38135 
   
   explain shape plan select * from table2 where Null; explain shape plan 
select * from table2 where false; in this case, null literal can be regard as 
same as false literal
   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38362:
URL: https://github.com/apache/doris/pull/38362#issuecomment-2249713617

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [opt](Nereids) add where Null rule to create empty relation as where false (#38135) [doris]

2024-07-25 Thread via GitHub


LiBinfeng-01 commented on PR #38362:
URL: https://github.com/apache/doris/pull/38362#issuecomment-2249713718

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](function) support split_by_regexp function [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38259:
URL: https://github.com/apache/doris/pull/38259#issuecomment-2249716123

   
   
   TPC-H: Total hot run time: 39752 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit daafe3c0b29f3119dd33734cf1b2b9efb44ded91, 
data reload: false
   
   -- Round 1 --
   q1   18187   469442514251
   q2   2028193 194 193
   q3   10534   121611031103
   q4   10222   830 839 830
   q5   7519265826652658
   q6   220 139 141 139
   q7   968 596 625 596
   q8   9222210021102100
   q9   8744653265316531
   q10  8910380838043804
   q11  476 243 244 243
   q12  404 238 234 234
   q13  17770   296429802964
   q14  271 239 233 233
   q15  531 474 484 474
   q16  488 406 390 390
   q17  980 651 771 651
   q18  7978761673167316
   q19  5715134313931343
   q20  698 316 331 316
   q21  4925310332353103
   q22  355 281 280 280
   Total cold run time: 117145 ms
   Total hot run time: 39752 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4369423642754236
   q2   364 272 266 266
   q3   3006288629212886
   q4   2016167116571657
   q5   571154835483
   q6   224 134 134 134
   q7   2180186218321832
   q8   3293340334553403
   q9   8796877489208774
   q10  4055386437673767
   q11  620 498 493 493
   q12  849 656 638 638
   q13  15956   325631633163
   q14  310 277 305 277
   q15  550 505 521 505
   q16  495 425 431 425
   q17  1831154315091509
   q18  7991784379517843
   q19  1722152414991499
   q20  2101188918781878
   q21  5046509847714771
   q22  615 489 507 489
   Total cold run time: 72100 ms
   Total hot run time: 55928 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](restore) Reset next version for restored partitions [doris]

2024-07-25 Thread via GitHub


w41ter merged PR #38343:
URL: https://github.com/apache/doris/pull/38343


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch branch-2.1 updated: [fix](restore) Reset next version for restored partitions (#38343)

2024-07-25 Thread w41ter
This is an automated email from the ASF dual-hosted git repository.

w41ter pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.1 by this push:
 new 3d6689d32cc [fix](restore) Reset next version for restored partitions 
(#38343)
3d6689d32cc is described below

commit 3d6689d32cc84cd96f5c2e76d4378d1586a1344e
Author: walter 
AuthorDate: Thu Jul 25 16:07:54 2024 +0800

[fix](restore) Reset next version for restored partitions (#38343)

Cherry-pick #38321
---
 .../java/org/apache/doris/backup/RestoreJob.java   |  7 --
 .../java/org/apache/doris/catalog/OlapTable.java   |  4 
 .../org/apache/doris/backup/RestoreJobTest.java| 27 ++
 3 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java 
b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
index 2e20a9fa223..57696bc5a84 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
@@ -82,6 +82,7 @@ import org.apache.doris.thrift.TStorageMedium;
 import org.apache.doris.thrift.TStorageType;
 import org.apache.doris.thrift.TTaskType;
 
+import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Joiner;
 import com.google.common.base.Preconditions;
 import com.google.common.collect.ArrayListMultimap;
@@ -1112,8 +1113,9 @@ public class RestoreJob extends AbstractJob {
 
 // reset remote partition.
 // reset all id in remote partition, but DO NOT modify any exist catalog 
objects.
-private Partition resetPartitionForRestore(OlapTable localTbl, OlapTable 
remoteTbl, String partName,
-   ReplicaAllocation replicaAlloc) 
{
+@VisibleForTesting
+protected Partition resetPartitionForRestore(OlapTable localTbl, OlapTable 
remoteTbl, String partName,
+ ReplicaAllocation 
replicaAlloc) {
 Preconditions.checkState(localTbl.getPartition(partName) == null);
 Partition remotePart = remoteTbl.getPartition(partName);
 Preconditions.checkNotNull(remotePart);
@@ -1143,6 +1145,7 @@ public class RestoreJob extends AbstractJob {
 
 // save version info for creating replicas
 long visibleVersion = remotePart.getVisibleVersion();
+remotePart.setNextVersion(visibleVersion + 1);
 LOG.info("reset partition {} for restore, visible version: {}, old 
partition id: {}",
 newPartId, visibleVersion, oldPartId);
 
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java 
b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
index de93324891a..8a4a9fee95f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
@@ -348,6 +348,10 @@ public class OlapTable extends Table implements 
MTMVRelatedTableIf {
 }
 }
 
+public void setPartitionInfo(PartitionInfo info) {
+partitionInfo = info;
+}
+
 public boolean hasMaterializedIndex(String indexName) {
 return indexNameToId.containsKey(indexName);
 }
diff --git 
a/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java 
b/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java
index ff60a6e8b90..a9e0a981a12 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/backup/RestoreJobTest.java
@@ -24,10 +24,13 @@ import 
org.apache.doris.backup.BackupJobInfo.BackupPartitionInfo;
 import org.apache.doris.backup.BackupJobInfo.BackupTabletInfo;
 import org.apache.doris.catalog.Database;
 import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.HashDistributionInfo;
 import org.apache.doris.catalog.MaterializedIndex;
 import org.apache.doris.catalog.MaterializedIndex.IndexExtState;
 import org.apache.doris.catalog.OlapTable;
 import org.apache.doris.catalog.Partition;
+import org.apache.doris.catalog.PartitionInfo;
+import org.apache.doris.catalog.PartitionType;
 import org.apache.doris.catalog.ReplicaAllocation;
 import org.apache.doris.catalog.Resource;
 import org.apache.doris.catalog.Table;
@@ -51,6 +54,7 @@ import mockit.Injectable;
 import mockit.Mock;
 import mockit.MockUp;
 import mockit.Mocked;
+import org.junit.Assert;
 import org.junit.Before;
 import org.junit.Test;
 
@@ -276,4 +280,27 @@ public class RestoreJobTest {
 System.out.println("tbl signature: " + 
tbl.getSignature(BackupHandler.SIGNATURE_VERSION, partNames));
 }
 
+@Test
+public void testResetPartitionVisibleAndNextVersionForRestore() throws 
Exception {
+long visibleVersion = 1234;
+long remotePartId = 123;
+String partName = "p20240723";
+

[PR] [regression](cluster key) move cluster key case to a new directory [doris]

2024-07-25 Thread via GitHub


mymeiyi opened a new pull request, #38363:
URL: https://github.com/apache/doris/pull/38363

   move cluster key case from `unique_with_mow_p0` directory to 
`unique_with_mow_p0_c` directory, because we will add more case like 
`unique_with_mow_p0`, now the directory is nested and confusing.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [regression](cluster key) move cluster key case to a new directory [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38363:
URL: https://github.com/apache/doris/pull/38363#issuecomment-2249718807

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [regression](cluster key) move cluster key case to a new directory [doris]

2024-07-25 Thread via GitHub


mymeiyi commented on PR #38363:
URL: https://github.com/apache/doris/pull/38363#issuecomment-2249723923

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [improve](columns)replace logFatal with exception and del some useless code in columns [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38035:
URL: https://github.com/apache/doris/pull/38035#issuecomment-2249729658

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Improvement](bloom filter) Forbid small bloom filter [doris]

2024-07-25 Thread via GitHub


Gabriel39 merged PR #38349:
URL: https://github.com/apache/doris/pull/38349


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



(doris) branch master updated (20c4cadcbbe -> 73a710fb0bd)

2024-07-25 Thread gabriellee
This is an automated email from the ASF dual-hosted git repository.

gabriellee pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from 20c4cadcbbe [Fix](Variant) fix outof range exception after calling 
function element_at for variant type (#38318)
 add 73a710fb0bd [Improvement](bloom filter) Forbid small bloom filter 
(#38349)

No new revisions were added by this update.

Summary of changes:
 be/src/exprs/bloom_filter_func.h | 20 
 be/src/exprs/runtime_filter.cpp  |  3 +++
 be/src/exprs/runtime_filter.h|  1 +
 .../java/org/apache/doris/qe/SessionVariable.java|  3 ++-
 gensrc/thrift/PaloInternalService.thrift |  4 +++-
 5 files changed, 25 insertions(+), 6 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]

2024-07-25 Thread via GitHub


eldenmoon opened a new pull request, #38364:
URL: https://github.com/apache/doris/pull/38364

   …subcolumns`
   
   ## Proposed changes
   
   Issue Number: close #xxx
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [fix](auth)fix show routine load db is null [doris]

2024-07-25 Thread via GitHub


zddr opened a new pull request, #38365:
URL: https://github.com/apache/doris/pull/38365

   cause by: https://github.com/apache/doris/pull/27861
   
   when label contains dbName, will loss
   ```
   show routine load for xxx.yyy
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38364:
URL: https://github.com/apache/doris/pull/38364#issuecomment-2249737823

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](group commit) Pick some group commit pr [doris]

2024-07-25 Thread via GitHub


mymeiyi commented on PR #38320:
URL: https://github.com/apache/doris/pull/38320#issuecomment-2249736944

   run feut


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](auth)fix show routine load db is null [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38365:
URL: https://github.com/apache/doris/pull/38365#issuecomment-2249738726

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](function) support split_by_regexp function [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38259:
URL: https://github.com/apache/doris/pull/38259#issuecomment-2249739868

   
   
   TPC-DS: Total hot run time: 173451 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit daafe3c0b29f3119dd33734cf1b2b9efb44ded91, 
data reload: false
   
   query1   908 371 360 360
   query2   6443201619071907
   query3   6654205 217 205
   query4   28082   17471   17639   17471
   query5   3685477 478 477
   query6   264 157 155 155
   query7   4594290 285 285
   query8   233 192 198 192
   query9   8599240923782378
   query10  458 294 284 284
   query11  11472   10025   99949994
   query12  117 81  83  81
   query13  1635403 371 371
   query14  10444   768175417541
   query15  225 168 168 168
   query16  7789474 469 469
   query17  1600546 514 514
   query18  1968275 267 267
   query19  188 146 146 146
   query20  85  98  78  78
   query21  204 133 129 129
   query22  4330419340584058
   query23  34378   33802   33534   33534
   query24  11026   289728552855
   query25  605 377 397 377
   query26  1108148 151 148
   query27  2370280 283 280
   query28  6430208620552055
   query29  881 692 629 629
   query30  259 159 158 158
   query31  981 772 737 737
   query32  96  52  57  52
   query33  763 317 337 317
   query34  947 482 486 482
   query35  859 763 759 759
   query36  1142989 981 981
   query37  140 84  105 84
   query38  2951285828292829
   query39  933 858 821 821
   query40  201 118 120 118
   query41  44  45  44  44
   query42  117 99  97  97
   query43  537 490 496 490
   query44  1271727 725 725
   query45  205 171 174 171
   query46  1080715 701 701
   query47  1839174817741748
   query48  369 289 289 289
   query49  830 402 415 402
   query50  773 397 392 392
   query51  6787676565436543
   query52  101 95  95  95
   query53  354 296 294 294
   query54  923 453 445 445
   query55  74  73  75  73
   query56  313 269 263 263
   query57  1128104010541040
   query58  262 247 297 247
   query59  2990266725792579
   query60  297 278 272 272
   query61  96  93  96  93
   query62  804 654 640 640
   query63  317 303 292 292
   query64  9219222116761676
   query65  3168312331643123
   query66  745 333 334 333
   query67  15385   15051   14763   14763
   query68  4547539 536 536
   query69  488 334 334 334
   query70  1203115611601156
   query71  427 287 271 271
   query72  7401572155715571
   query73  766 326 321 321
   query74  6318564657245646
   query75  3629267426752674
   query76  2730971 895 895
   query77  613 310 316 310
   query78  9603901589448944
   query79  2799516 521 516
   query80  1651483 473 473
   query81  582 217 226 217
   query82  907 142 139 139
   query83  325 169 175 169
   query84  277 89  89  89
   query85  1794383 307 307
   query86  484 325 303 303
   query87  3276307031083070
   query88  3812235323672353
   query89  485 393 392 392
   query90  1787195 195 195
   query91  129 102 100 100
   query92  61  52  51  51
   query93  3073516 501 501
   query94  1146283 290 283
   query95  409 315 327 315
   query96  597 270 274 270
   query97  3264302630913026
   query98  222 190 196 190
   query99  1744125812791258
   Total cold run time: 282624 ms
   Total hot run time: 173451 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the 

Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38359:
URL: https://github.com/apache/doris/pull/38359#issuecomment-2249742905

   
   
   TPC-H: Total hot run time: 40110 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 55208846a86b386c1969cde484bd7e26c33f23e8, 
data reload: false
   
   -- Round 1 --
   q1   18223   542444514451
   q2   2828194 191 191
   q3   11108   129610311031
   q4   11173   755 895 755
   q5   7806276227442744
   q6   227 140 138 138
   q7   988 601 589 589
   q8   9208208621242086
   q9   8688657866026578
   q10  8603378638493786
   q11  472 231 235 231
   q12  396 222 225 222
   q13  17780   299529552955
   q14  284 247 237 237
   q15  532 473 486 473
   q16  504 386 397 386
   q17  978 613 628 613
   q18  8115755574717471
   q19  7026141013981398
   q20  676 327 330 327
   q21  4975316932213169
   q22  337 279 281 279
   Total cold run time: 120927 ms
   Total hot run time: 40110 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4388440842254225
   q2   381 267 273 267
   q3   3025278327532753
   q4   1939166416001600
   q5   5332536053215321
   q6   222 127 131 127
   q7   2142174517271727
   q8   3191335833083308
   q9   8397837184018371
   q10  3911368536653665
   q11  586 490 504 490
   q12  769 610 569 569
   q13  17620   297830492978
   q14  326 273 271 271
   q15  512 468 467 467
   q16  481 412 427 412
   q17  1778147614721472
   q18  7781745074567450
   q19  1646138215071382
   q20  1993179617821782
   q21  4925475947164716
   q22  564 493 476 476
   Total cold run time: 71909 ms
   Total hot run time: 53829 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38243:
URL: https://github.com/apache/doris/pull/38243#issuecomment-2249743123

   
   
   TPC-H: Total hot run time: 39710 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit bf6aad6e7d1b3229f6fdc1b41de15dc9b7d01bfc, 
data reload: false
   
   -- Round 1 --
   q1   18903   633542784278
   q2   2011188 197 188
   q3   10567   112910771077
   q4   10335   730 832 730
   q5   7563271126732673
   q6   224 137 136 136
   q7   950 597 584 584
   q8   9216206820492049
   q9   8626652665306526
   q10  8716375637583756
   q11  483 238 245 238
   q12  411 226 224 224
   q13  17774   298229882982
   q14  284 237 263 237
   q15  522 486 493 486
   q16  511 392 400 392
   q17  986 696 695 695
   q18  8084744273017301
   q19  4948144214161416
   q20  642 329 317 317
   q21  4837315432013154
   q22  344 273 271 271
   Total cold run time: 116937 ms
   Total hot run time: 39710 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4377420342014201
   q2   360 268 262 262
   q3   3047290129562901
   q4   1993165417201654
   q5   5585553154895489
   q6   220 145 129 129
   q7   2223188418281828
   q8   3276339933853385
   q9   8726884088168816
   q10  4142379839203798
   q11  606 510 523 510
   q12  811 659 657 657
   q13  15727   323031883188
   q14  322 291 295 291
   q15  556 484 503 484
   q16  500 440 422 422
   q17  1845155015201520
   q18  8033795577097709
   q19  1778154415341534
   q20  2240188018791879
   q21  8261489148574857
   q22  609 541 519 519
   Total cold run time: 75237 ms
   Total hot run time: 56033 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Feature](function) support split_by_regexp function [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38259:
URL: https://github.com/apache/doris/pull/38259#issuecomment-2249750848

   
   
   ClickBench: Total hot run time: 30.96 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit daafe3c0b29f3119dd33734cf1b2b9efb44ded91, 
data reload: false
   
   query1   0.050.040.04
   query2   0.090.040.04
   query3   0.220.060.05
   query4   1.680.080.07
   query5   0.500.480.48
   query6   1.120.730.73
   query7   0.020.010.02
   query8   0.050.040.04
   query9   0.550.490.50
   query10  0.550.540.54
   query11  0.150.120.12
   query12  0.160.130.12
   query13  0.610.590.59
   query14  0.770.770.81
   query15  0.850.830.80
   query16  0.350.360.34
   query17  0.970.970.97
   query18  0.230.220.21
   query19  1.811.681.74
   query20  0.010.010.01
   query21  15.41   0.780.66
   query22  4.207.262.12
   query23  18.26   1.371.31
   query24  2.110.240.21
   query25  0.160.090.08
   query26  0.280.200.21
   query27  0.460.230.23
   query28  13.35   1.011.00
   query29  12.63   3.553.48
   query30  0.260.070.06
   query31  2.850.390.39
   query32  3.280.470.46
   query33  2.942.882.94
   query34  17.09   4.344.33
   query35  4.404.404.40
   query36  0.650.460.50
   query37  0.180.150.16
   query38  0.160.150.15
   query39  0.040.040.04
   query40  0.140.110.13
   query41  0.100.050.05
   query42  0.060.050.06
   query43  0.050.040.04
   Total cold run time: 109.8 s
   Total hot run time: 30.96 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [feature](http action)Add http action to show nested inverted index file [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on code in PR #38272:
URL: https://github.com/apache/doris/pull/38272#discussion_r1691046960


##
be/src/olap/rowset/beta_rowset.cpp:
##
@@ -701,4 +702,127 @@ Status BetaRowset::calc_file_crc(uint32_t* crc_value, 
int64_t* file_count) {
 return Status::OK();
 }
 
+Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value,

Review Comment:
   warning: function 'show_nested_index_file' exceeds recommended 
size/complexity thresholds [readability-function-size]
   ```cpp
   Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value,
  ^
   ```
   
   Additional context
   
   **be/src/olap/rowset/beta_rowset.cpp:704:** 120 lines including whitespace 
and comments (threshold 80)
   ```cpp
   Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value,
  ^
   ```
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38364:
URL: https://github.com/apache/doris/pull/38364#issuecomment-2249750710

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[I] [Bug] (mtmv) Unable to find a suitable base table for partitioning when there are both valid and invalid expressions [doris]

2024-07-25 Thread via GitHub


liutang123 opened a new issue, #38366:
URL: https://github.com/apache/doris/issues/38366

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no 
similar issues.
   
   
   ### Version
   
   2.1
   
   ### What's Wrong?
   
   reproduce:
   ```
   CREATE TABLE `test_t` (
 `id` VARCHAR(36) NOT NULL COMMENT 'id',
 `created_time` DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ''
   ) ENGINE=OLAP
   DUPLICATE KEY(`id`)
   COMMENT ''
   PARTITION BY RANGE(`created_time`)
   (PARTITION P_2024071713 VALUES [('2024-07-17 13:00:00'), ('2024-07-17 
14:00:00')),
   PARTITION P_2024071714 VALUES [('2024-07-17 14:00:00'), ('2024-07-17 
15:00:00')))
   DISTRIBUTED BY HASH(`id`) BUCKETS AUTO
   PROPERTIES (
   "replication_allocation" = "tag.location.default: 1"
   );
   CREATE MATERIALIZED VIEW test_mtmv
   BUILD IMMEDIATE REFRESH AUTO ON MANUAL
   partition by(date_trunc(created_time_minute, 'day'))
   DISTRIBUTED BY RANDOM BUCKETS 3
   PROPERTIES ('replication_num' = '1') 
   AS 
   select  id 
   ,date_trunc(created_time, 'minute') as created_time_minute
   ,min(created_time) as start_time
   ,if(count(id) > 0, 1, 0) as status
   from test_t 
   group by id, date_trunc(created_time, 'minute');
   ```
   An error occurs:
   ```
   ERROR 1105 (HY000): errCode = 2, detailMessage = Unable to find a suitable 
base table for partitioning, the fail reason is can't not find valid partition 
track column, because column to check use invalid implicit expression, invalid 
expression is min(created_time#12)
   ```
   
   ### What You Expected?
   
   `test_t` should be selected as base table.
   `date_trunc(created_time, 'minute')` should pass the check and 
`min(created_time)` should be ignore.
   
   ### How to Reproduce?
   
Refer to the description above.
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [Fix](Variant) implement resize and set num_rows right in `apply_for_… [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38364:
URL: https://github.com/apache/doris/pull/38364#issuecomment-2249753344

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[PR] [Fix] fix arrow read timestamp bug [doris-flink-connector]

2024-07-25 Thread via GitHub


JNSimba opened a new pull request, #446:
URL: https://github.com/apache/doris-flink-connector/pull/446

   # Proposed changes
   
   Currently, when reading arrow, since the be layer hard-codes all datetimev2 
scales to 6, arrow returns TimeStampMicroVector, which results in scale parsing 
errors when arrow is displayed. Refer to 
https://github.com/apache/doris/issues/38174
   
   This PR removes the strong consistency restriction of the type and is 
compatible with the logic of the scale repaired above
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [refine](bits) refine bytes_mask_to_bits_mask code [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38360:
URL: https://github.com/apache/doris/pull/38360#issuecomment-2249757100

   
   
   TPC-H: Total hot run time: 40187 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 6635a3866fd28147784237653e3621b85a570a71, 
data reload: false
   
   -- Round 1 --
   q1   18191   501844814481
   q2   2488225 194 194
   q3   11465   1194989 989
   q4   11179   745 856 745
   q5   7700275227392739
   q6   228 139 145 139
   q7   969 624 621 621
   q8   9243207721002077
   q9   8719661166536611
   q10  8774375037953750
   q11  477 236 241 236
   q12  396 232 231 231
   q13  17811   297530272975
   q14  281 234 247 234
   q15  535 479 491 479
   q16  482 392 391 391
   q17  978 624 737 624
   q18  8141747174317431
   q19  6977139513771377
   q20  692 330 340 330
   q21  4935324332603243
   q22  351 298 290 290
   Total cold run time: 121012 ms
   Total hot run time: 40187 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   4479429842794279
   q2   379 262 268 262
   q3   3020271927712719
   q4   1900160916171609
   q5   5282532753555327
   q6   217 133 133 133
   q7   2074165516951655
   q8   3212335733113311
   q9   8409844383618361
   q10  3881366536703665
   q11  617 494 497 494
   q12  766 637 611 611
   q13  17540   302430053005
   q14  305 269 276 269
   q15  510 485 474 474
   q16  484 428 431 428
   q17  1776149014791479
   q18  7596750272597259
   q19  1668161916141614
   q20  1983176917641764
   q21  4972472248164722
   q22  586 520 505 505
   Total cold run time: 71656 ms
   Total hot run time: 53945 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](inverted index) disable range query in StringTypeInvertedIndexReader [doris]

2024-07-25 Thread via GitHub


github-actions[bot] commented on PR #38218:
URL: https://github.com/apache/doris/pull/38218#issuecomment-2249759777

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix] (mtmv) Choose a valid partition column when there are both valid and invalid expressions [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38367:
URL: https://github.com/apache/doris/pull/38367#issuecomment-2249761648

   Thank you for your contribution to Apache Doris.
   Don't know what should be done next? See [How to process your 
PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
   
   Since 2024-03-18, the Document has been moved to 
[doris-website](https://github.com/apache/doris-website).
   See [Doris 
Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix] (mtmv) Choose a valid partition column when there are both valid and invalid expressions [doris]

2024-07-25 Thread via GitHub


liutang123 commented on PR #38367:
URL: https://github.com/apache/doris/pull/38367#issuecomment-2249762242

   Will add regression tests later


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



Re: [PR] [fix](cloud) Ensure that the eror log has been uploaded to S3 before returning an error URL to the user [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38359:
URL: https://github.com/apache/doris/pull/38359#issuecomment-2249766067

   
   
   TPC-DS: Total hot run time: 173066 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 55208846a86b386c1969cde484bd7e26c33f23e8, 
data reload: false
   
   query1   928 385 361 361
   query2   6445188518151815
   query3   6664211 214 211
   query4   27519   17329   17346   17329
   query5   4241479 469 469
   query6   265 185 159 159
   query7   4587291 280 280
   query8   235 197 198 197
   query9   8391241923892389
   query10  441 294 275 275
   query11  11812   10146   10044   10044
   query12  145 83  89  83
   query13  1643381 369 369
   query14  10273   788576977697
   query15  215 167 165 165
   query16  7630470 505 470
   query17  1612553 551 551
   query18  1526287 283 283
   query19  208 152 160 152
   query20  93  82  83  82
   query21  210 134 127 127
   query22  4190412139803980
   query23  33830   33304   33169   33169
   query24  11970   285728692857
   query25  660 390 395 390
   query26  1752151 150 150
   query27  2765272 283 272
   query28  7404202920212021
   query29  1048666 634 634
   query30  288 151 153 151
   query31  970 736 741 736
   query32  101 54  54  54
   query33  785 356 337 337
   query34  895 474 484 474
   query35  867 740 737 737
   query36  1095922 922 922
   query37  258 79  81  79
   query38  2849278927582758
   query39  854 813 855 813
   query40  284 122 121 121
   query41  48  46  48  46
   query42  127 97  104 97
   query43  498 468 458 458
   query44  1306728 736 728
   query45  214 179 179 179
   query46  1097780 744 744
   query47  1861179017971790
   query48  369 293 298 293
   query49  1272409 408 408
   query50  780 395 390 390
   query51  6839659666946596
   query52  101 96  90  90
   query53  363 289 290 289
   query54  976 442 448 442
   query55  77  72  75  72
   query56  281 271 266 266
   query57  1209107110611061
   query58  267 256 240 240
   query59  2791257125502550
   query60  306 278 283 278
   query61  96  97  97  97
   query62  838 663 667 663
   query63  320 301 290 290
   query64  10496   220816861686
   query65  3193310830963096
   query66  1292364 329 329
   query67  15629   15077   14830   14830
   query68  8511554 563 554
   query69  691 460 357 357
   query70  1218115311511151
   query71  527 284 270 270
   query72  8920576454765476
   query73  1306325 322 322
   query74  6161576857245724
   query75  4592267127432671
   query76  4987910 946 910
   query77  745 306 297 297
   query78  9592923389338933
   query79  8160521 519 519
   query80  1022481 485 481
   query81  599 222 218 218
   query82  292 133 130 130
   query83  300 168 160 160
   query84  272 92  84  84
   query85  1034309 346 309
   query86  355 313 339 313
   query87  3339310631203106
   query88  4790237223732372
   query89  522 389 376 376
   query90  2074197 192 192
   query91  129 104 100 100
   query92  64  51  50  50
   query93  5845517 510 510
   query94  1338283 258 258
   query95  410 319 312 312
   query96  607 271 271 271
   query97  3216307730383038
   query98  223 209 188 188
   query99  1552124312971243
   Total cold run time: 303987 ms
   Total hot run time: 173066 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the

Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]

2024-07-25 Thread via GitHub


doris-robot commented on PR #38243:
URL: https://github.com/apache/doris/pull/38243#issuecomment-2249765436

   
   
   TPC-DS: Total hot run time: 173437 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit bf6aad6e7d1b3229f6fdc1b41de15dc9b7d01bfc, 
data reload: false
   
   query1   906 365 362 362
   query2   6420205019471947
   query3   6633206 227 206
   query4   26326   17597   17202   17202
   query5   3582485 478 478
   query6   286 176 179 176
   query7   4585297 286 286
   query8   250 193 191 191
   query9   8529241723852385
   query10  438 291 276 276
   query11  12568   10058   99399939
   query12  120 91  84  84
   query13  1654372 370 370
   query14  10167   740676717406
   query15  208 161 164 161
   query16  7673462 443 443
   query17  1378532 543 532
   query18  1933281 273 273
   query19  199 146 162 146
   query20  94  87  79  79
   query21  209 128 123 123
   query22  4407400840784008
   query23  34386   33495   33531   33495
   query24  11288   301429792979
   query25  593 384 448 384
   query26  774 157 158 157
   query27  2210285 274 274
   query28  6128208720732073
   query29  917 632 638 632
   query30  260 160 150 150
   query31  979 783 777 777
   query32  100 56  54  54
   query33  767 331 347 331
   query34  933 488 487 487
   query35  867 749 754 749
   query36  11351006957 957
   query37  140 81  82  81
   query38  2968290328092809
   query39  901 874 829 829
   query40  202 123 116 116
   query41  48  45  46  45
   query42  118 98  93  93
   query43  527 470 483 470
   query44  1191727 758 727
   query45  210 172 172 172
   query46  1083732 728 728
   query47  1857178517681768
   query48  365 292 287 287
   query49  819 403 412 403
   query50  783 387 390 387
   query51  6813662666936626
   query52  102 89  91  89
   query53  356 290 292 290
   query54  901 446 447 446
   query55  76  77  74  74
   query56  278 292 263 263
   query57  1137106010421042
   query58  248 245 254 245
   query59  2920265427302654
   query60  302 279 272 272
   query61  99  105 96  96
   query62  824 655 634 634
   query63  319 283 286 283
   query64  9137220616231623
   query65  3170312130963096
   query66  741 333 337 333
   query67  15501   15031   15108   15031
   query68  5317561 540 540
   query69  654 433 366 366
   query70  1112115810871087
   query71  455 283 274 274
   query72  7331596853025302
   query73  769 322 330 322
   query74  6182570656655665
   query75  3682269726892689
   query76  3156963 875 875
   query77  640 319 292 292
   query78  10358   914190959095
   query79  1633539 529 529
   query80  2441563 486 486
   query81  595 227 227 227
   query82  427 137 144 137
   query83  305 169 165 165
   query84  265 86  95  86
   query85  1061309 300 300
   query86  438 338 313 313
   query87  3269309931473099
   query88  3368238623642364
   query89  471 391 412 391
   query90  1637197 200 197
   query91  137 100 102 100
   query92  65  51  50  50
   query93  1951535 525 525
   query94  796 305 298 298
   query95  415 332 323 323
   query96  596 274 279 274
   query97  3187304230303030
   query98  228 196 195 195
   query99  1568128112571257
   Total cold run time: 278547 ms
   Total hot run time: 173437 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the 

  1   2   3   4   5   6   7   >