Re: [PR] [opt](mtmv)Enable mv rewrite default cherrypick 21 [doris]
seawinde commented on PR #36949: URL: https://github.com/apache/doris/pull/36949#issuecomment-2210316676 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](ES Catalog)Support control scroll level by config [doris]
qidaye commented on PR #37180: URL: https://github.com/apache/doris/pull/37180#issuecomment-2210319397 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](nereids) Default set enable_nereids_distribute_planner=true [doris]
doris-robot commented on PR #37219: URL: https://github.com/apache/doris/pull/37219#issuecomment-2210319749 TPC-H: Total hot run time: 40637 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a2b6bdc87e867fbb7b9b0575bba803b8acd8e464, data reload: false -- Round 1 -- q1 17873 449844364436 q2 2668204 198 198 q3 12242 115611781156 q4 10698 777 733 733 q5 8067270826522652 q6 229 141 142 141 q7 950 601 621 601 q8 9224208921392089 q9 9191702569906990 q10 8939378037383738 q11 450 246 244 244 q12 414 235 227 227 q13 17766 295129932951 q14 277 220 227 220 q15 530 492 474 474 q16 516 376 386 376 q17 981 781 765 765 q18 8006751874217421 q19 6971153613871387 q20 662 335 343 335 q21 5129315038713150 q22 415 353 356 353 Total cold run time: 122198 ms Total hot run time: 40637 ms - Round 2, with runtime_filter_mode=off - q1 4416423442874234 q2 375 283 275 275 q3 3005282727292729 q4 1913156915851569 q5 5260525852595258 q6 223 137 138 137 q7 2152172117171717 q8 3332338833403340 q9 8873885288718852 q10 3906371536653665 q11 591 494 505 494 q12 801 604 612 604 q13 16559 295630022956 q14 284 266 279 266 q15 524 496 478 478 q16 480 411 430 411 q17 1791149714751475 q18 7742748272797279 q19 1718166116341634 q20 2097190418761876 q21 4915485546964696 q22 668 568 570 568 Total cold run time: 71625 ms Total hot run time: 54513 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](alter) support add partitions in batch [doris]
xy720 commented on PR #37114: URL: https://github.com/apache/doris/pull/37114#issuecomment-2210323835 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](ES Catalog)Support control scroll level by config [doris]
qidaye commented on PR #37180: URL: https://github.com/apache/doris/pull/37180#issuecomment-2210324582 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](mtmv) Materialized view partition track supports date_trunc and optimize the fail reason (#35562) [doris]
morrySnow merged PR #36947: URL: https://github.com/apache/doris/pull/36947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](ES Catalog)Support control scroll level by config [doris]
github-actions[bot] commented on PR #37180: URL: https://github.com/apache/doris/pull/37180#issuecomment-2210325607 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [opt](mtmv) Materialized view partition track supports date_trunc and optimize the fail reason (#35562) (#36947)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 8a0d05d9b01 [opt](mtmv) Materialized view partition track supports
date_trunc and optimize the fail reason (#35562) (#36947)
8a0d05d9b01 is described below
commit 8a0d05d9b01760fab1e2e7d6af5f479272289275
Author: seawinde <149132972+seawi...@users.noreply.github.com>
AuthorDate: Fri Jul 5 15:12:43 2024 +0800
[opt](mtmv) Materialized view partition track supports date_trunc and
optimize the fail reason (#35562) (#36947)
cherry pick from master #35562
commitId: 43d0f191
---
.../org/apache/doris/mtmv/MTMVPartitionInfo.java | 6 -
.../mv/AsyncMaterializationContext.java| 6 +-
.../exploration/mv/MaterializedViewUtils.java | 221 --
.../mv/rollup/AggFunctionRollUpHandler.java| 5 +-
.../rules/expression/ExpressionNormalization.java | 2 +
.../rules/expression/rules/MergeDateTrunc.java | 78 +
.../org/apache/doris/nereids/trees/TreeNode.java | 4 +-
.../trees/expressions/literal/Interval.java| 37 ++-
.../commands/info/MTMVPartitionDefinition.java | 107 ---
.../exploration/mv/MaterializedViewUtilsTest.java | 172 ---
.../expression/ExpressionRewriteTestHelper.java| 6 +
.../rules/expression/MergeDateTruncTest.java | 57
.../data/mtmv_p0/test_rollup_partition_mtmv.out| 22 ++
.../mtmv_p0/test_rollup_partition_mtmv.groovy | 325 +
14 files changed, 900 insertions(+), 148 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionInfo.java
b/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionInfo.java
index 7ca1b7e3e63..ff4060f334a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionInfo.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionInfo.java
@@ -21,8 +21,6 @@ import org.apache.doris.analysis.Expr;
import org.apache.doris.catalog.Column;
import org.apache.doris.common.AnalysisException;
-import com.google.common.collect.ImmutableSet;
-import com.google.common.collect.ImmutableSortedSet;
import com.google.gson.annotations.SerializedName;
import java.util.List;
@@ -38,10 +36,6 @@ public class MTMVPartitionInfo {
SELF_MANAGE
}
-public static final ImmutableSet MTMV_PARTITION_FUNCTIONS = new
ImmutableSortedSet.Builder(
-String.CASE_INSENSITIVE_ORDER).add("date_trunc")
-.build();
-
@SerializedName("pt")
private MTMVPartitionType partitionType;
@SerializedName("rt")
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AsyncMaterializationContext.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AsyncMaterializationContext.java
index 9776673de79..d830e9d41cb 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AsyncMaterializationContext.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/AsyncMaterializationContext.java
@@ -104,9 +104,9 @@ public class AsyncMaterializationContext extends
MaterializationContext {
return Optional.empty();
}
RelationId relationId = null;
-List logicalOlapScan =
this.getScanPlan().collectFirst(LogicalOlapScan.class::isInstance);
-if (!logicalOlapScan.isEmpty()) {
-relationId = logicalOlapScan.get(0).getRelationId();
+Optional logicalOlapScan =
this.getScanPlan().collectFirst(LogicalOlapScan.class::isInstance);
+if (logicalOlapScan.isPresent()) {
+relationId = logicalOlapScan.get().getRelationId();
}
return Optional.of(Pair.of(relationId,
normalizeStatisticsColumnExpression(mtmvCache.getStatistics(;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewUtils.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewUtils.java
index 53f775e2177..5c6a666fdb4 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewUtils.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewUtils.java
@@ -27,12 +27,18 @@ import org.apache.doris.mtmv.MTMVRelatedTableIf;
import org.apache.doris.nereids.CascadesContext;
import org.apache.doris.nereids.memo.Group;
import org.apache.doris.nereids.memo.StructInfoMap;
+import org.apache.doris.nereids.rules.expression.ExpressionNormalization;
+import org.apache.doris.nereids.rules.expression.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.Alias;
import org.apache.doris.nereids.trees.expressions.ExprId;
import org.apache.doris.nereids.trees.expressions.Expression;
i
Re: [PR] [feature](ES Catalog)Support control scroll level by config [doris]
github-actions[bot] commented on PR #37180: URL: https://github.com/apache/doris/pull/37180#issuecomment-2210328273 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Feature] FQDN FE Cluster start ERROR [doris]
xcsxcsxcs closed issue #35887: [Feature] FQDN FE Cluster start ERROR URL: https://github.com/apache/doris/issues/35887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Feature] FQDN FE Cluster start ERROR [doris]
xcsxcsxcs commented on issue #35887: URL: https://github.com/apache/doris/issues/35887#issuecomment-2210326237 set hostname -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](fe) Add ATTN logs about partition range [doris]
w41ter opened a new pull request, #37338: URL: https://github.com/apache/doris/pull/37338 The CCR syncer relies on the value of the partition range to determine the mapping relationship between the upstream partition and the downstream partition. This PR records this behavior in the corresponding code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Add ATTN logs about partition range [doris]
doris-robot commented on PR #37338: URL: https://github.com/apache/doris/pull/37338#issuecomment-2210329220 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Add ATTN logs about partition range [doris]
w41ter commented on PR #37338: URL: https://github.com/apache/doris/pull/37338#issuecomment-2210329310 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cloud) add inverted index storage format V2 [doris]
doris-robot commented on PR #37336: URL: https://github.com/apache/doris/pull/37336#issuecomment-2210329720 TPC-H: Total hot run time: 40157 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c2857620166a27588a97d13e220280f116dd6282, data reload: false -- Round 1 -- q1 17613 445342944294 q2 2006201 193 193 q3 10455 124710661066 q4 10191 723 857 723 q5 7480268426192619 q6 234 139 141 139 q7 969 607 621 607 q8 9241207420842074 q9 9019646164766461 q10 8979375537503750 q11 448 244 244 244 q12 494 236 234 234 q13 18856 298029962980 q14 269 226 223 223 q15 527 484 483 483 q16 522 386 380 380 q17 988 709 714 709 q18 7997748275207482 q19 5864153815601538 q20 659 342 368 342 q21 5108327340333273 q22 402 346 343 343 Total cold run time: 118321 ms Total hot run time: 40157 ms - Round 2, with runtime_filter_mode=off - q1 4418429142584258 q2 368 287 276 276 q3 3100294128952895 q4 2024175017861750 q5 5534556354545454 q6 229 135 132 132 q7 2286191918611861 q8 3316347334393439 q9 8796882788458827 q10 4148375638363756 q11 615 505 494 494 q12 795 642 702 642 q13 17180 325132303230 q14 305 260 297 260 q15 523 483 510 483 q16 497 439 433 433 q17 1835153515331533 q18 8177791878157815 q19 1814165317031653 q20 2951192318681868 q21 6075490848924892 q22 656 560 594 560 Total cold run time: 75642 ms Total hot run time: 56511 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[fix](memory safe) Fix block-reader memory alloc failed make … [doris]
doris-robot commented on PR #36979: URL: https://github.com/apache/doris/pull/36979#issuecomment-2210329718 TPC-H: Total hot run time: 40492 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9573edd2a1b76d4b323462f6665ab940336761fe, data reload: false -- Round 1 -- q1 17611 432042734273 q2 2009192 190 190 q3 10465 116211151115 q4 10191 788 756 756 q5 7506268326242624 q6 224 138 139 138 q7 950 611 628 611 q8 9228209520942094 q9 9101647465006474 q10 8862371337003700 q11 462 236 243 236 q12 511 236 234 234 q13 18872 295029882950 q14 260 223 224 223 q15 520 473 486 473 q16 475 370 369 369 q17 962 597 632 597 q18 8123742774727427 q19 3852147515161475 q20 673 332 332 332 q21 4967406338623862 q22 402 339 340 339 Total cold run time: 116226 ms Total hot run time: 40492 ms - Round 2, with runtime_filter_mode=off - q1 4368421243734212 q2 374 260 264 260 q3 3017284428762844 q4 2024170416731673 q5 5678550254415441 q6 219 136 136 136 q7 2204188718841884 q8 3267342633643364 q9 8697872688148726 q10 4047389337873787 q11 599 505 499 499 q12 814 663 638 638 q13 16128 313431933134 q14 289 283 285 283 q15 530 493 497 493 q16 498 423 427 423 q17 1815152915211521 q18 8154793578467846 q19 1786162714521452 q20 2153192818311831 q21 9175497449044904 q22 603 586 565 565 Total cold run time: 76439 ms Total hot run time: 55916 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](uncaught exception) Fix uncaught exception from load_index [doris]
doris-robot commented on PR #36335: URL: https://github.com/apache/doris/pull/36335#issuecomment-2210329915 TPC-H: Total hot run time: 39672 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 3e3edb1597c352be6e6448454ff267a0cd74e498, data reload: false -- Round 1 -- q1 17598 436842854285 q2 2022195 196 195 q3 10458 125411201120 q4 10188 779 785 779 q5 7488268326362636 q6 223 139 139 139 q7 956 613 594 594 q8 9238209420542054 q9 8914647365206473 q10 8920372037023702 q11 457 240 234 234 q12 513 224 234 224 q13 18782 298029872980 q14 281 227 223 223 q15 534 475 482 475 q16 508 371 374 371 q17 985 613 663 613 q18 8011746074157415 q19 7737145014021402 q20 693 319 326 319 q21 5085309739503097 q22 394 344 342 342 Total cold run time: 119985 ms Total hot run time: 39672 ms - Round 2, with runtime_filter_mode=off - q1 4391420642154206 q2 369 275 264 264 q3 3068292429582924 q4 1977175917281728 q5 5526548853915391 q6 256 136 133 133 q7 2266186018731860 q8 3264343434153415 q9 8757881687788778 q10 4128374938723749 q11 575 499 490 490 q12 808 657 652 652 q13 17235 320031243124 q14 306 284 271 271 q15 538 487 489 487 q16 498 425 418 418 q17 1803153715231523 q18 8130808377497749 q19 1776176017181718 q20 2911184118251825 q21 9420471149384711 q22 643 533 541 533 Total cold run time: 78645 ms Total hot run time: 55949 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the sql of AddPartitionRecord [doris]
doris-robot commented on PR #37339: URL: https://github.com/apache/doris/pull/37339#issuecomment-2210332601 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](fe) Fix the sql of AddPartitionRecord [doris]
w41ter opened a new pull request, #37339: URL: https://github.com/apache/doris/pull/37339 Cherry-pick #37295. The range field is accidentally compared to DUMMY_ITEM. It was introduced by #35461. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the sql of AddPartitionRecord [doris]
w41ter commented on PR #37339: URL: https://github.com/apache/doris/pull/37339#issuecomment-2210332722 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Fix](planner)fix bug of char(255) toSql [doris]
feiniaofeiafei opened a new pull request, #37340: URL: https://github.com/apache/doris/pull/37340 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](planner)fix bug of char(255) toSql [doris]
doris-robot commented on PR #37340: URL: https://github.com/apache/doris/pull/37340#issuecomment-2210332935 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](nereids) Default set enable_nereids_distribute_planner=true [doris]
doris-robot commented on PR #37219: URL: https://github.com/apache/doris/pull/37219#issuecomment-2210334324 TPC-DS: Total hot run time: 173733 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a2b6bdc87e867fbb7b9b0575bba803b8acd8e464, data reload: false query1 930 375 374 374 query2 6475250725172507 query3 6683205 218 205 query4 27914 17486 17261 17261 query5 4170495 505 495 query6 286 184 159 159 query7 4593305 302 302 query8 316 283 295 283 query9 8655247124332433 query10 602 290 276 276 query11 10464 999510128 9995 query12 140 92 82 82 query13 1653373 380 373 query14 10459 822267356735 query15 238 189 203 189 query16 7850341 312 312 query17 1737570 542 542 query18 2025290 288 288 query19 214 161 160 160 query20 90 87 84 84 query21 216 128 127 127 query22 4545414141264126 query23 33745 33194 33105 33105 query24 12067 301130473011 query25 664 393 393 393 query26 1677164 162 162 query27 2934323 339 323 query28 7612211521002100 query29 989 645 641 641 query30 278 149 146 146 query31 1152937 962 937 query32 101 55 56 55 query33 775 320 332 320 query34 880 492 483 483 query35 759 647 623 623 query36 1056974 922 922 query37 148 79 77 77 query38 2865270627672706 query39 895 800 798 798 query40 271 130 128 128 query41 54 50 53 50 query42 123 101 101 101 query43 606 574 578 574 query44 1188734 726 726 query45 204 165 169 165 query46 1124753 752 752 query47 1888177017891770 query48 367 298 314 298 query49 1193428 438 428 query50 783 394 405 394 query51 6905688968846884 query52 107 94 101 94 query53 370 298 305 298 query54 1000469 461 461 query55 77 73 78 73 query56 309 293 290 290 query57 1163104510491045 query58 261 246 370 246 query59 3453317032983170 query60 316 277 275 275 query61 101 88 115 88 query62 638 445 451 445 query63 322 292 299 292 query64 9913225817691769 query65 3369309931153099 query66 1439343 334 334 query67 15727 15081 14972 14972 query68 6956557 555 555 query69 723 450 342 342 query70 1204112411231123 query71 551 285 282 282 query72 8863565959015659 query73 796 323 324 323 query74 6046562655435543 query75 4783267026972670 query76 4860100610071006 query77 761 312 312 312 query78 9540883290208832 query79 8824516 514 514 query80 1019470 465 465 query81 589 228 230 228 query82 751 106 105 105 query83 358 171 168 168 query84 265 85 84 84 query85 1407291 287 287 query86 464 374 344 344 query87 3296308730813081 query88 4694251524332433 query89 535 383 390 383 query90 2055189 189 189 query91 129 102 97 97 query92 67 47 49 47 query93 7074510 502 502 query94 1329213 207 207 query95 406 320 317 317 query96 605 268 272 268 query97 3204301730063006 query98 221 198 188 188 query99 1180842 875 842 Total cold run time: 305678 ms Total hot run time: 173733 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Improvement]Log be thread num [doris]
doris-robot commented on PR #37289: URL: https://github.com/apache/doris/pull/37289#issuecomment-2210335387 TPC-H: Total hot run time: 40032 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 24323711f20c9b33c1a58ef36b6a77fb11af1ef2, data reload: false -- Round 1 -- q1 17638 441743254325 q2 2022191 183 183 q3 10464 120810551055 q4 10203 850 822 822 q5 7503276226312631 q6 225 135 135 135 q7 971 587 600 587 q8 9230209320622062 q9 8984651265076507 q10 9016372437283724 q11 470 241 237 237 q12 428 228 225 225 q13 17771 295429882954 q14 275 227 236 227 q15 532 477 482 477 q16 511 380 376 376 q17 981 613 697 613 q18 8125758675057505 q19 5434146315541463 q20 674 326 348 326 q21 4957325740393257 q22 401 341 348 341 Total cold run time: 116815 ms Total hot run time: 40032 ms - Round 2, with runtime_filter_mode=off - q1 4445420842134208 q2 367 256 254 254 q3 2999288328872883 q4 1997165917471659 q5 5608550854325432 q6 226 130 129 129 q7 2224188419031884 q8 3266341134433411 q9 8671872388078723 q10 4063390038063806 q11 596 534 514 514 q12 830 644 665 644 q13 17031 316331543154 q14 298 287 279 279 q15 537 497 481 481 q16 511 448 436 436 q17 1826153015171517 q18 8105781180677811 q19 1782167915711571 q20 2202187118211821 q21 5138478949734789 q22 637 535 566 535 Total cold run time: 73359 ms Total hot run time: 55941 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](fe) Fix the sql of AddPartitionRecord [doris]
w41ter opened a new pull request, #37341: URL: https://github.com/apache/doris/pull/37341 Cherry-pick #37295 The range field is accidentally compared to DUMMY_ITEM. It was introduced by #35461. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the sql of AddPartitionRecord [doris]
doris-robot commented on PR #37341: URL: https://github.com/apache/doris/pull/37341#issuecomment-2210336248 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the sql of AddPartitionRecord [doris]
w41ter commented on PR #37341: URL: https://github.com/apache/doris/pull/37341#issuecomment-2210336333 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](auth)fix fe [doris]
zddr opened a new pull request, #37342: URL: https://github.com/apache/doris/pull/37342 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth)fix fe [doris]
doris-robot commented on PR #37342: URL: https://github.com/apache/doris/pull/37342#issuecomment-2210340729 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](nereids) Default set enable_nereids_distribute_planner=true [doris]
doris-robot commented on PR #37219: URL: https://github.com/apache/doris/pull/37219#issuecomment-2210341343 ClickBench: Total hot run time: 31.21 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a2b6bdc87e867fbb7b9b0575bba803b8acd8e464, data reload: false query1 0.040.030.03 query2 0.070.040.04 query3 0.230.060.05 query4 1.660.070.07 query5 0.500.490.49 query6 1.130.730.72 query7 0.020.020.02 query8 0.050.040.05 query9 0.530.510.49 query10 0.550.540.55 query11 0.150.120.12 query12 0.150.120.12 query13 0.590.590.58 query14 0.770.760.78 query15 0.850.810.81 query16 0.370.380.36 query17 0.981.041.02 query18 0.230.250.25 query19 1.801.661.69 query20 0.010.010.01 query21 15.39 0.740.64 query22 4.247.032.40 query23 18.32 1.371.30 query24 2.140.230.21 query25 0.160.080.08 query26 0.300.220.20 query27 0.460.240.23 query28 13.21 1.021.00 query29 12.59 3.383.34 query30 0.260.060.05 query31 2.920.390.38 query32 3.260.480.47 query33 2.862.922.93 query34 17.16 4.354.44 query35 4.444.404.49 query36 0.650.470.47 query37 0.180.170.16 query38 0.150.150.16 query39 0.040.040.03 query40 0.160.120.12 query41 0.090.050.05 query42 0.060.050.04 query43 0.050.040.04 Total cold run time: 109.77 s Total hot run time: 31.21 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cloud) add inverted index storage format V2 [doris]
doris-robot commented on PR #37336: URL: https://github.com/apache/doris/pull/37336#issuecomment-2210343961 TPC-DS: Total hot run time: 173399 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c2857620166a27588a97d13e220280f116dd6282, data reload: false query1 914 393 371 371 query2 6484257424242424 query3 6645205 216 205 query4 22071 17599 17240 17240 query5 3792497 508 497 query6 263 177 175 175 query7 4589307 309 307 query8 339 302 307 302 query9 8644246924442444 query10 583 314 282 282 query11 10448 997599719971 query12 119 89 85 85 query13 1639371 374 371 query14 9831785667356735 query15 242 183 191 183 query16 7716310 309 309 query17 1698546 522 522 query18 1863286 272 272 query19 200 150 154 150 query20 91 82 89 82 query21 210 133 131 131 query22 4319401339863986 query23 33799 33481 33597 33481 query24 11140 292128992899 query25 652 393 391 391 query26 1219181 163 163 query27 2315325 327 325 query28 6566217221692169 query29 901 642 644 642 query30 284 153 148 148 query31 975 763 757 757 query32 93 56 56 56 query33 779 317 301 301 query34 1014491 498 491 query35 728 637 634 634 query36 1115998 949 949 query37 151 86 89 86 query38 2973285128242824 query39 923 845 841 841 query40 220 138 135 135 query41 54 55 52 52 query42 111 102 103 102 query43 620 594 548 548 query44 1194725 722 722 query45 201 166 162 162 query46 1112739 722 722 query47 1826175617431743 query48 386 311 301 301 query49 880 421 429 421 query50 781 399 407 399 query51 6874678068356780 query52 111 96 101 96 query53 372 298 300 298 query54 947 473 505 473 query55 73 71 75 71 query56 304 274 276 274 query57 1118103810681038 query58 261 273 289 273 query59 3506346932473247 query60 307 280 280 280 query61 98 96 94 94 query62 597 429 437 429 query63 333 305 312 305 query64 8855235317941794 query65 3249317031803170 query66 749 331 335 331 query67 15435 15029 14798 14798 query68 6191544 558 544 query69 653 445 384 384 query70 1226118211511151 query71 496 300 285 285 query72 7610587153025302 query73 832 324 322 322 query74 6256552454285428 query75 4057269026682668 query76 4383936 944 936 query77 670 315 326 315 query78 9565898010885 8980 query79 3571514 520 514 query80 1091493 484 484 query81 554 222 217 217 query82 404 113 111 111 query83 192 170 167 167 query84 272 84 88 84 query85 1360321 312 312 query86 464 339 321 321 query87 3316314230593059 query88 3955245124372437 query89 483 401 408 401 query90 1841190 190 190 query91 132 104 103 103 query92 61 53 52 52 query93 1317511 500 500 query94 1103213 207 207 query95 410 317 324 317 query96 586 267 270 267 query97 3194305930603059 query98 234 206 204 204 query99 1225853 819 819 Total cold run time: 276605 ms Total hot run time: 173399 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](uncaught exception) Fix uncaught exception from load_index [doris]
doris-robot commented on PR #36335: URL: https://github.com/apache/doris/pull/36335#issuecomment-2210344621 TPC-DS: Total hot run time: 170976 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 3e3edb1597c352be6e6448454ff267a0cd74e498, data reload: false query1 937 371 367 367 query2 6395264624732473 query3 6645203 215 203 query4 28412 17260 17268 17260 query5 3790479 478 478 query6 263 175 168 168 query7 4599305 310 305 query8 324 314 311 311 query9 8488246624482448 query10 589 308 286 286 query11 11598 10136 99679967 query12 126 88 94 88 query13 1640398 382 382 query14 10123 690975686909 query15 256 181 187 181 query16 7389329 323 323 query17 1341552 544 544 query18 1777292 284 284 query19 205 155 156 155 query20 94 86 83 83 query21 220 139 124 124 query22 4328402140134013 query23 34025 33740 33673 33673 query24 10908 289629772896 query25 627 426 407 407 query26 727 172 165 165 query27 2301332 337 332 query28 5924221021662166 query29 926 661 687 661 query30 255 154 159 154 query31 955 741 757 741 query32 108 58 67 58 query33 684 331 326 326 query34 918 512 498 498 query35 788 640 634 634 query36 1143976 941 941 query37 141 81 84 81 query38 2943282528842825 query39 874 818 795 795 query40 212 130 127 127 query41 58 54 53 53 query42 117 98 106 98 query43 602 569 561 561 query44 1062735 726 726 query45 200 174 163 163 query46 1073724 723 723 query47 1847175417801754 query48 373 309 304 304 query49 938 411 417 411 query50 767 390 388 388 query51 6867686967506750 query52 100 93 95 93 query53 365 295 300 295 query54 894 457 452 452 query55 72 72 73 72 query56 305 280 272 272 query57 1147104710701047 query58 255 253 258 253 query59 3651332333753323 query60 299 278 301 278 query61 94 94 93 93 query62 604 455 439 439 query63 326 293 292 292 query64 8591230818051805 query65 3146313431203120 query66 803 324 327 324 query67 15768 15007 14849 14849 query68 8626555 564 555 query69 771 494 350 350 query70 1382105211351052 query71 542 276 280 276 query72 9606508627962796 query73 2240327 329 327 query74 5847549254485448 query75 5472264226742642 query76 52141042892 892 query77 800 359 311 311 query78 10572 890588088808 query79 8720520 525 520 query80 1093467 480 467 query81 585 223 217 217 query82 473 104 105 104 query83 341 168 162 162 query84 274 88 86 86 query85 981 304 304 304 query86 352 336 309 309 query87 3294309330853085 query88 4373249124542454 query89 531 383 381 381 query90 2087195 195 195 query91 137 108 106 106 query92 63 51 52 51 query93 5941515 502 502 query94 1362220 215 215 query95 422 324 330 324 query96 631 275 271 271 query97 3212298030072980 query98 219 195 198 195 query99 1132844 821 821 Total cold run time: 301617 ms Total hot run time: 170976 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [PR] Revert "[fix](memory safe) Fix block-reader memory alloc failed make … [doris]
doris-robot commented on PR #36979: URL: https://github.com/apache/doris/pull/36979#issuecomment-2210344697 TPC-DS: Total hot run time: 174459 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 9573edd2a1b76d4b323462f6665ab940336761fe, data reload: false query1 908 374 377 374 query2 6462243024082408 query3 6636203 210 203 query4 28280 17198 17097 17097 query5 3664509 489 489 query6 261 171 172 171 query7 4602305 299 299 query8 323 295 305 295 query9 8493248724632463 query10 575 300 287 287 query11 11977 10076 99869986 query12 116 83 91 83 query13 1648375 384 375 query14 10289 786379577863 query15 268 193 192 192 query16 7667314 316 314 query17 1815574 551 551 query18 2029285 288 285 query19 207 161 157 157 query20 93 84 89 84 query21 220 134 132 132 query22 4406411140424042 query23 34134 34281 33615 33615 query24 10898 292929502929 query25 631 429 425 425 query26 1265168 162 162 query27 2892332 345 332 query28 7183221222112211 query29 919 685 684 684 query30 263 160 154 154 query31 987 822 771 771 query32 101 56 58 56 query33 797 332 324 324 query34 909 530 515 515 query35 792 664 733 664 query36 1155984 985 984 query37 152 82 86 82 query38 2878281326932693 query39 945 819 802 802 query40 206 128 126 126 query41 54 50 53 50 query42 131 103 97 97 query43 602 563 550 550 query44 1186743 730 730 query45 190 165 159 159 query46 1080735 758 735 query47 1835175317691753 query48 370 299 301 299 query49 853 414 426 414 query50 763 385 394 385 query51 6890680667236723 query52 108 92 94 92 query53 380 299 293 293 query54 868 458 454 454 query55 74 75 73 73 query56 296 294 274 274 query57 1154103210201020 query58 254 255 250 250 query59 3386325630803080 query60 302 283 280 280 query61 98 98 94 94 query62 577 440 423 423 query63 332 297 299 297 query64 8956224318461846 query65 3157313131113111 query66 698 331 338 331 query67 15702 15098 14859 14859 query68 8970560 573 560 query69 819 448 366 366 query70 1390115611281128 query71 513 282 279 279 query72 9357546455645464 query73 2277329 327 327 query74 5887541455045414 query75 5370270926442644 query76 5534955 930 930 query77 756 302 318 302 query78 10535 895288338833 query79 10788 536 545 536 query80 1249501 486 486 query81 589 220 223 220 query82 269 104 108 104 query83 302 171 166 166 query84 269 89 88 88 query85 1119322 311 311 query86 376 310 318 310 query87 3390306930653065 query88 5081245124362436 query89 507 400 392 392 query90 2155186 188 186 query91 136 104 106 104 query92 63 51 50 50 query93 6410523 514 514 query94 1519212 214 212 query95 407 320 319 319 query96 606 275 275 275 query97 3218304230343034 query98 220 208 195 195 query99 1126859 858 858 Total cold run time: 309549 ms Total hot run time: 174459 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [PR] [Improvement]Log be thread num [doris]
doris-robot commented on PR #37289: URL: https://github.com/apache/doris/pull/37289#issuecomment-2210350800 TPC-DS: Total hot run time: 174037 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 24323711f20c9b33c1a58ef36b6a77fb11af1ef2, data reload: false query1 927 372 368 368 query2 6459253324632463 query3 6637212 222 212 query4 28350 17561 17306 17306 query5 3786495 491 491 query6 268 184 163 163 query7 4584297 296 296 query8 357 305 291 291 query9 8454234023262326 query10 574 321 295 295 query11 11937 10025 10156 10025 query12 134 92 86 86 query13 1655384 384 384 query14 9575761478897614 query15 231 186 186 186 query16 7784314 318 314 query17 1671563 543 543 query18 1956280 287 280 query19 202 159 159 159 query20 90 88 85 85 query21 220 130 128 128 query22 4331416140654065 query23 34172 33451 33523 33451 query24 10883 289928492849 query25 628 423 430 423 query26 723 160 159 159 query27 2252331 336 331 query28 5926212521242124 query29 899 655 677 655 query30 268 167 156 156 query31 971 762 805 762 query32 102 58 58 58 query33 684 350 334 334 query34 920 495 505 495 query35 774 672 653 653 query36 1135980 989 980 query37 146 81 86 81 query38 2898275027512750 query39 892 815 808 808 query40 204 131 132 131 query41 53 52 53 52 query42 123 98 102 98 query43 628 565 579 565 query44 1147752 737 737 query45 202 160 169 160 query46 1094732 730 730 query47 1852177917961779 query48 386 305 403 305 query49 834 415 421 415 query50 751 387 386 386 query51 6918679667706770 query52 113 92 104 92 query53 379 301 290 290 query54 869 447 452 447 query55 75 73 74 73 query56 288 264 270 264 query57 1121103310401033 query58 255 254 261 254 query59 3657322331623162 query60 299 271 273 271 query61 95 94 94 94 query62 611 449 435 435 query63 329 292 291 291 query64 8587226117681768 query65 3149317331413141 query66 715 324 328 324 query67 15761 14968 14841 14841 query68 8915544 554 544 query69 740 464 386 386 query70 1421113610461046 query71 532 294 292 292 query72 9041548455495484 query73 2241326 321 321 query74 5821549755095497 query75 5678269126692669 query76 5113974 892 892 query77 816 310 302 302 query78 9735893488288828 query79 9385510 529 510 query80 1967495 475 475 query81 585 227 217 217 query82 262 105 107 105 query83 343 168 173 168 query84 275 88 85 85 query85 884 309 293 293 query86 357 306 303 303 query87 3357310731443107 query88 4709244724312431 query89 532 386 376 376 query90 2076192 190 190 query91 136 104 102 102 query92 61 50 53 50 query93 5882500 498 498 query94 1340207 209 207 query95 404 307 317 307 query96 600 270 271 270 query97 3159298629722972 query98 206 201 203 201 query99 1301852 863 852 Total cold run time: 302824 ms Total hot run time: 174037 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](cloud) add inverted index storage format V2 [doris]
doris-robot commented on PR #37336: URL: https://github.com/apache/doris/pull/37336#issuecomment-2210351745 ClickBench: Total hot run time: 30.44 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c2857620166a27588a97d13e220280f116dd6282, data reload: false query1 0.050.030.03 query2 0.080.040.04 query3 0.220.050.04 query4 1.680.070.10 query5 0.510.490.50 query6 1.140.730.74 query7 0.020.010.02 query8 0.050.040.05 query9 0.550.490.50 query10 0.540.530.53 query11 0.150.120.11 query12 0.140.120.12 query13 0.600.580.59 query14 0.780.770.78 query15 0.840.810.83 query16 0.360.360.35 query17 1.011.041.02 query18 0.220.250.24 query19 1.841.741.71 query20 0.010.010.01 query21 15.39 0.760.64 query22 3.908.311.68 query23 18.24 1.371.23 query24 2.080.230.21 query25 0.150.080.08 query26 0.300.210.21 query27 0.450.240.23 query28 13.30 1.000.99 query29 12.94 3.433.42 query30 0.250.060.05 query31 2.870.400.40 query32 3.250.470.47 query33 2.852.992.94 query34 17.15 4.334.33 query35 4.484.344.35 query36 0.660.460.46 query37 0.180.150.15 query38 0.150.150.14 query39 0.040.040.03 query40 0.150.120.12 query41 0.090.050.05 query42 0.060.050.04 query43 0.040.040.04 Total cold run time: 109.76 s Total hot run time: 30.44 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](docker)Docker be register bug and add fe new interface recovery [doris]
FreeOnePlus commented on PR #37335: URL: https://github.com/apache/doris/pull/37335#issuecomment-2210351755 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](uncaught exception) Fix uncaught exception from load_index [doris]
doris-robot commented on PR #36335: URL: https://github.com/apache/doris/pull/36335#issuecomment-2210352367 ClickBench: Total hot run time: 31.04 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 3e3edb1597c352be6e6448454ff267a0cd74e498, data reload: false query1 0.040.030.03 query2 0.070.040.04 query3 0.230.050.05 query4 1.670.080.08 query5 0.510.600.47 query6 1.140.720.73 query7 0.020.010.01 query8 0.050.040.05 query9 0.550.500.50 query10 0.540.540.55 query11 0.160.120.11 query12 0.150.120.12 query13 0.590.590.59 query14 0.750.780.76 query15 0.850.810.81 query16 0.340.370.36 query17 0.971.031.00 query18 0.240.230.23 query19 1.791.691.72 query20 0.010.010.01 query21 15.40 0.780.65 query22 4.476.652.26 query23 18.30 1.441.30 query24 2.440.220.21 query25 0.150.080.09 query26 0.310.210.21 query27 0.460.230.23 query28 13.33 1.010.99 query29 12.66 3.383.39 query30 0.260.060.06 query31 2.850.390.38 query32 3.280.490.47 query33 2.922.862.91 query34 17.09 4.424.38 query35 4.394.394.38 query36 0.670.450.48 query37 0.170.150.15 query38 0.160.150.15 query39 0.040.040.03 query40 0.160.120.12 query41 0.090.050.05 query42 0.060.060.05 query43 0.040.050.04 Total cold run time: 110.37 s Total hot run time: 31.04 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[fix](memory safe) Fix block-reader memory alloc failed make … [doris]
doris-robot commented on PR #36979: URL: https://github.com/apache/doris/pull/36979#issuecomment-2210352417 ClickBench: Total hot run time: 30.92 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 9573edd2a1b76d4b323462f6665ab940336761fe, data reload: false query1 0.040.030.04 query2 0.070.040.04 query3 0.220.040.05 query4 1.690.070.06 query5 0.510.480.48 query6 1.140.720.73 query7 0.020.010.01 query8 0.050.050.05 query9 0.560.490.49 query10 0.530.530.54 query11 0.150.110.11 query12 0.160.120.12 query13 0.590.590.59 query14 0.810.780.78 query15 0.840.810.81 query16 0.380.380.38 query17 0.971.020.98 query18 0.230.230.26 query19 1.801.751.72 query20 0.010.010.01 query21 15.39 0.760.67 query22 4.316.462.02 query23 18.35 1.451.37 query24 2.420.220.22 query25 0.160.090.08 query26 0.280.220.21 query27 0.450.230.23 query28 13.27 1.010.99 query29 12.59 3.423.38 query30 0.260.050.06 query31 2.870.390.39 query32 3.270.490.47 query33 2.942.972.91 query34 17.08 4.354.38 query35 4.404.424.39 query36 0.650.460.47 query37 0.170.140.14 query38 0.150.140.15 query39 0.040.040.03 query40 0.150.120.13 query41 0.090.040.04 query42 0.050.050.04 query43 0.040.040.04 Total cold run time: 110.15 s Total hot run time: 30.92 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth)fix fe can not restart when replay create row policy log [doris]
zddr commented on PR #37342: URL: https://github.com/apache/doris/pull/37342#issuecomment-2210354870 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](planner)fix bug of char(255) toSql [doris]
feiniaofeiafei commented on PR #37340: URL: https://github.com/apache/doris/pull/37340#issuecomment-2210355939 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement]Log be thread num [doris]
doris-robot commented on PR #37289: URL: https://github.com/apache/doris/pull/37289#issuecomment-2210358582 ClickBench: Total hot run time: 31.19 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 24323711f20c9b33c1a58ef36b6a77fb11af1ef2, data reload: false query1 0.040.030.04 query2 0.080.030.03 query3 0.220.060.05 query4 1.660.090.10 query5 0.490.490.48 query6 1.150.730.73 query7 0.020.010.02 query8 0.050.050.04 query9 0.550.490.49 query10 0.550.540.54 query11 0.160.110.12 query12 0.140.120.12 query13 0.580.580.58 query14 0.770.770.77 query15 0.840.820.80 query16 0.370.370.37 query17 1.041.041.03 query18 0.210.240.26 query19 1.871.721.86 query20 0.010.010.01 query21 15.40 0.750.64 query22 3.736.552.39 query23 18.27 1.361.28 query24 2.120.230.22 query25 0.150.090.09 query26 0.300.200.20 query27 0.450.230.22 query28 13.29 1.011.00 query29 12.58 3.353.35 query30 0.260.060.07 query31 2.850.380.39 query32 3.290.460.47 query33 2.832.932.91 query34 17.00 4.394.37 query35 4.414.364.42 query36 0.650.460.47 query37 0.180.160.15 query38 0.150.140.14 query39 0.040.030.04 query40 0.150.130.12 query41 0.090.040.04 query42 0.050.050.04 query43 0.050.040.05 Total cold run time: 109.09 s Total hot run time: 31.19 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Purge all unused Jemalloc dirty pages for all arenas when exceed limit [doris]
github-actions[bot] commented on PR #37164: URL: https://github.com/apache/doris/pull/37164#issuecomment-2210359872 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) Purge all unused Jemalloc dirty pages for all arenas when exceed limit [doris]
github-actions[bot] commented on PR #37164: URL: https://github.com/apache/doris/pull/37164#issuecomment-2210359930 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Add ATTN logs about partition range [doris]
doris-robot commented on PR #37338: URL: https://github.com/apache/doris/pull/37338#issuecomment-2210360030 TPC-H: Total hot run time: 40113 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d16bfa6a1cea37d27a0369633beb2e9ce56025f2, data reload: false -- Round 1 -- q1 17633 441743054305 q2 2019200 189 189 q3 10446 122211271127 q4 10197 829 872 829 q5 7499269826392639 q6 228 140 138 138 q7 964 612 607 607 q8 9227212720762076 q9 10505 655965026502 q10 9052373437513734 q11 448 251 243 243 q12 432 226 238 226 q13 18713 299029762976 q14 271 223 241 223 q15 507 481 494 481 q16 510 374 377 374 q17 965 633 686 633 q18 8204747475237474 q19 1706146915111469 q20 666 326 331 326 q21 5015319541053195 q22 406 350 347 347 Total cold run time: 115613 ms Total hot run time: 40113 ms - Round 2, with runtime_filter_mode=off - q1 4351424842984248 q2 369 269 271 269 q3 3046268227622682 q4 1976166616591659 q5 5513558455315531 q6 239 134 130 130 q7 2231187419061874 q8 3332343534353435 q9 8608865983978397 q10 3861367335833583 q11 604 476 464 464 q12 770 602 621 602 q13 15911 298829852985 q14 293 266 280 266 q15 524 471 471 471 q16 468 412 415 412 q17 1793148514741474 q18 7661752873627362 q19 1718150416241504 q20 2084176817411741 q21 4812483446804680 q22 612 558 547 547 Total cold run time: 70776 ms Total hot run time: 54316 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Optimize](Row store) change default value of row_column_page_size to 16KB [doris]
github-actions[bot] commented on PR #37145: URL: https://github.com/apache/doris/pull/37145#issuecomment-2210361064 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](ES Catalog)Support control scroll level by config [doris]
doris-robot commented on PR #37180: URL: https://github.com/apache/doris/pull/37180#issuecomment-2210362571 TPC-H: Total hot run time: 39950 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6f0581622845c800b39c09e46fe580309a2cb4d7, data reload: false -- Round 1 -- q1 17636 432242934293 q2 2016198 192 192 q3 10468 119811061106 q4 10200 925 778 778 q5 7478266426232623 q6 226 135 140 135 q7 955 598 611 598 q8 9221205520852055 q9 7068650765016501 q10 8826367737183677 q11 462 250 243 243 q12 443 231 236 231 q13 17773 299129572957 q14 273 236 237 236 q15 532 509 487 487 q16 517 379 383 379 q17 970 728 728 728 q18 8094754874107410 q19 6870153914161416 q20 665 336 342 336 q21 4894322639143226 q22 396 345 343 343 Total cold run time: 115983 ms Total hot run time: 39950 ms - Round 2, with runtime_filter_mode=off - q1 4394423642194219 q2 371 280 263 263 q3 2975288728842884 q4 2025169516421642 q5 5650550454405440 q6 221 133 134 133 q7 2186185818791858 q8 3292343434343434 q9 8695873288988732 q10 4060384437273727 q11 612 490 491 490 q12 858 616 635 616 q13 15836 316231973162 q14 318 298 285 285 q15 525 480 490 480 q16 485 436 433 433 q17 1832155915251525 q18 8122800078817881 q19 1782158815821582 q20 2147188018571857 q21 5845494348624862 q22 607 555 585 555 Total cold run time: 72838 ms Total hot run time: 56060 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](cloud) Extract SyncPoint to common cpp [doris]
platoneko commented on code in PR #37165:
URL: https://github.com/apache/doris/pull/37165#discussion_r1666454425
##
common_cpp/sync_point.h:
##
@@ -259,15 +259,12 @@ auto try_any_cast_ret(std::vector& any) {
# define TEST_INJECTION_POINT_RETURN_WITH_VOID(x, ...)
# define TEST_INJECTION_POINT_SINGLETON()
#else
-namespace doris::config {
-extern bool enable_injection_point;
-}
-# define TEST_INJECTION_POINT(x) if (doris::config::enable_injection_point) {
LOG_INFO("enter inject point {}", x); SYNC_POINT(x); }
-# define TEST_IDX_INJECTION_POINT(x, index) if
(doris::config::enable_injection_point) { LOG_INFO("enter inject point {}", x);
IDX_SYNC_POINT(x, index); }
-# define TEST_INJECTION_POINT_CALLBACK(x, ...) if
(doris::config::enable_injection_point) { LOG_INFO("enter inject point {}", x);
SYNC_POINT_CALLBACK(x, __VA_ARGS__); }
-# define TEST_INJECTION_POINT_SINGLETON() if
(doris::config::enable_injection_point) { LOG_INFO("enter inject point {}", x);
SYNC_POINT_SINGLETON(); }
-# define TEST_INJECTION_POINT_RETURN_WITH_VALUE(x, default_ret_val, ...) if
(doris::config::enable_injection_point) { LOG_INFO("enter inject point {}", x);
SYNC_POINT_RETURN_WITH_VALUE(x, default_ret_val, __VA_ARGS__); }
-# define TEST_INJECTION_POINT_RETURN_WITH_VOID(x, ...) if
(doris::config::enable_injection_point) { LOG_INFO("enter inject point {}", x);
SYNC_POINT_RETURN_WITH_VOID(x, __VA_ARGS__); }
+# define TEST_INJECTION_POINT(x) SYNC_POINT(x);
Review Comment:
Injection points will be enabled via http api
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](alter) support add partitions in batch [doris]
doris-robot commented on PR #37114: URL: https://github.com/apache/doris/pull/37114#issuecomment-2210370694 TPC-H: Total hot run time: 39627 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f2cabc8462c9673ad860c37e3dc8eaad7033f604, data reload: false -- Round 1 -- q1 17629 435642364236 q2 2019192 186 186 q3 10442 123811861186 q4 10189 834 722 722 q5 7573260825422542 q6 221 139 137 137 q7 954 590 601 590 q8 9232202720512027 q9 9051643364696433 q10 8977371937173717 q11 460 238 236 236 q12 508 235 241 235 q13 18616 301929762976 q14 277 226 235 226 q15 520 488 487 487 q16 516 375 371 371 q17 966 630 687 630 q18 8026753174507450 q19 6342135115161351 q20 667 329 325 325 q21 5055321939133219 q22 403 345 355 345 Total cold run time: 118643 ms Total hot run time: 39627 ms - Round 2, with runtime_filter_mode=off - q1 4342423742074207 q2 380 272 266 266 q3 2977291929142914 q4 2000172217271722 q5 5546547154495449 q6 237 132 130 130 q7 2216186018741860 q8 3267338734263387 q9 8652875188148751 q10 4118376237783762 q11 624 502 499 499 q12 839 662 633 633 q13 16368 316431803164 q14 313 274 277 274 q15 549 493 481 481 q16 497 434 427 427 q17 1806153415031503 q18 8116803878477847 q19 1788158616611586 q20 2137188518781878 q21 5087484546914691 q22 666 555 559 555 Total cold run time: 72525 ms Total hot run time: 55986 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](nereids) derive column stats for 'expr and A is not null' [doris]
englefly merged PR #37235: URL: https://github.com/apache/doris/pull/37235 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (fa389f467f7 -> 68522bc0967)
This is an automated email from the ASF dual-hosted git repository. englefly pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from fa389f467f7 [Fix](Timezone)The time zone should not be fixed to UTC+8. (#37294) add 68522bc0967 [fix](nereids) derive column stats for 'expr and A is not null' (#37235) No new revisions were added by this update. Summary of changes: .../doris/nereids/stats/FilterEstimation.java | 1 + .../org/apache/doris/statistics/Statistics.java| 18 + .../doris/nereids/stats/FilterEstimationTest.java | 35 - .../rf_prune/query64.out | 74 +-- .../nereids_tpcds_shape_sf100_p0/shape/query64.out | 84 +++--- 5 files changed, 132 insertions(+), 80 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Add ATTN logs about partition range [doris]
doris-robot commented on PR #37338: URL: https://github.com/apache/doris/pull/37338#issuecomment-2210375742 TPC-DS: Total hot run time: 172443 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d16bfa6a1cea37d27a0369633beb2e9ce56025f2, data reload: false query1 920 368 376 368 query2 6479263026572630 query3 6527201 213 201 query4 28383 17250 17133 17133 query5 4227485 476 476 query6 264 190 164 164 query7 4511289 310 289 query8 310 290 277 277 query9 8241244524442444 query10 587 316 282 282 query11 11575 10084 10052 10052 query12 128 88 87 87 query13 1644385 377 377 query14 10046 750376047503 query15 238 204 186 186 query16 7871344 313 313 query17 1727575 561 561 query18 1874296 292 292 query19 206 154 155 154 query20 91 84 88 84 query21 215 133 137 133 query22 4309408140124012 query23 34432 32677 32955 32677 query24 10115 277327742773 query25 596 370 378 370 query26 700 153 152 152 query27 2166318 315 315 query28 5846210421022102 query29 881 623 612 612 query30 278 144 145 144 query31 992 725 726 725 query32 95 52 52 52 query33 751 304 294 294 query34 890 489 479 479 query35 745 637 616 616 query36 1070929 907 907 query37 133 72 75 72 query38 2878272827282728 query39 867 823 804 804 query40 201 128 128 128 query41 55 53 52 52 query42 124 95 104 95 query43 587 564 566 564 query44 1159731 718 718 query45 188 163 157 157 query46 1080730 750 730 query47 1890175017981750 query48 376 302 293 293 query49 1037404 399 399 query50 773 400 393 393 query51 6871691767556755 query52 103 95 95 95 query53 363 300 292 292 query54 939 451 443 443 query55 74 74 71 71 query56 285 285 269 269 query57 1140104410301030 query58 299 269 246 246 query59 3403347732143214 query60 297 287 274 274 query61 95 98 98 98 query62 667 457 447 447 query63 320 295 285 285 query64 8608228417601760 query65 3157309030913090 query66 824 335 331 331 query67 15422 15082 14853 14853 query68 6186548 542 542 query69 681 450 350 350 query70 1190108711641087 query71 453 281 276 276 query72 8103557051515151 query73 774 326 326 326 query74 5959557654695469 query75 3571266526722665 query76 31981007978 978 query77 688 365 309 309 query78 9551890888128812 query79 2425513 517 513 query80 2214485 472 472 query81 576 221 219 219 query82 693 112 106 106 query83 273 174 173 173 query84 282 92 93 92 query85 1288329 337 329 query86 416 330 305 305 query87 3298312031083108 query88 3596246324332433 query89 502 392 378 378 query90 1848190 196 190 query91 134 103 104 103 query92 62 50 51 50 query93 4166505 509 505 query94 1160211 209 209 query95 415 322 319 319 query96 600 271 264 264 query97 3232299030422990 query98 281 199 200 199 query99 1201851 849 849 Total cold run time: 283161 ms Total hot run time: 172443 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feat](Nereids) push down predicates with multi columns through LogicalWindow and LogicalPartitionTopN [doris]
morrySnow merged PR #36984: URL: https://github.com/apache/doris/pull/36984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [feat](Nereids) push down predicates with multi columns through LogicalWindow and LogicalPartitionTopN (#36984)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new a90b3633414 [feat](Nereids) push down predicates with multi columns
through LogicalWindow and LogicalPartitionTopN (#36984)
a90b3633414 is described below
commit a90b363341464d31828aaa2053b15ca56be62514
Author: feiniaofeiafei <53502832+feiniaofeia...@users.noreply.github.com>
AuthorDate: Fri Jul 5 15:49:52 2024 +0800
[feat](Nereids) push down predicates with multi columns through
LogicalWindow and LogicalPartitionTopN (#36984)
cherry-pick #36828 to branch-2.0
The requirement for predicate pushdown through the window operator is
that the partition by slots of the window contains all slots in the
predicate. The original implementation of doris only allows predicate
pushdown with one slot. This PR relaxes this restriction and allows for
predicate pushdown with multiple slots. The same applies to the
predicate pushdown of the LogicalPartitionTopN operator. The following
sql is an example.
select
*
from
(
select
row_number() over(partition by id, value1 order by value1)
as num,
id,
value1
from
push_down_multi_column_predicate_through_window_t ) t
where
abs(id + value1)<4
and num <= 2;
Co-authored-by: feiniaofeiafei
---
.../PushdownFilterThroughPartitionTopN.java| 21 +--
.../rules/rewrite/PushdownFilterThroughWindow.java | 15 ++--
.../push_down_filter_through_window.out| 32
.../push_down_filter_through_window.groovy | 43 ++
4 files changed, 88 insertions(+), 23 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughPartitionTopN.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughPartitionTopN.java
index da31fb2ae3d..27061d7277e 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughPartitionTopN.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughPartitionTopN.java
@@ -29,6 +29,7 @@ import
org.apache.doris.nereids.trees.plans.logical.LogicalPartitionTopN;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSet.Builder;
+import java.util.HashSet;
import java.util.Set;
/**
@@ -62,19 +63,17 @@ public class PushdownFilterThroughPartitionTopN extends
OneRewriteRuleFactory {
// PushdownFilterThroughWindow
Builder bottomConjunctsBuilder =
ImmutableSet.builder();
Builder upperConjunctsBuilder = ImmutableSet.builder();
+Set partitionKeySlots = new HashSet<>();
+for (Expression partitionKey : partitionTopN.getPartitionKeys()) {
+if (partitionKey instanceof SlotReference) {
+partitionKeySlots.add((SlotReference) partitionKey);
+}
+}
for (Expression expr : filter.getConjuncts()) {
-boolean pushed = false;
Set exprInputSlots = expr.getInputSlots();
-for (Expression partitionKey :
partitionTopN.getPartitionKeys()) {
-if (partitionKey instanceof SlotReference
-&& exprInputSlots.size() == 1
-&&
partitionKey.getInputSlots().containsAll(exprInputSlots)) {
-bottomConjunctsBuilder.add(expr);
-pushed = true;
-break;
-}
-}
-if (!pushed) {
+if (partitionKeySlots.containsAll(exprInputSlots)) {
+bottomConjunctsBuilder.add(expr);
+} else {
upperConjunctsBuilder.add(expr);
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughWindow.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughWindow.java
index 1902ff50300..1dba171d29a 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughWindow.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PushdownFilterThroughWindow.java
@@ -64,18 +64,9 @@ public class PushdownFilterThroughWindow extends
OneRewriteRuleFactory {
Set bottomConjuncts = Sets.newHashSet();
Set upperConjuncts = Sets.newHashSet();
for (Expression expr : filter.getConjuncts()) {
-boolean pushed = false;
-for (Expression
Re: [PR] [fix](binlog) gc dropped partition records in disable binlog gc [doris]
github-actions[bot] commented on PR #37326: URL: https://github.com/apache/doris/pull/37326#issuecomment-2210378847 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](binlog) gc dropped partition records in disable binlog gc [doris]
github-actions[bot] commented on PR #37326: URL: https://github.com/apache/doris/pull/37326#issuecomment-2210378891 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [opt](memory) Purge all unused Jemalloc dirty pages for all arenas when exceed limit (#37164)
This is an automated email from the ASF dual-hosted git repository.
zouxinyi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new ba410b52f51 [opt](memory) Purge all unused Jemalloc dirty pages for
all arenas when exceed limit (#37164)
ba410b52f51 is described below
commit ba410b52f511c2a42c6e78ace3370c9be9be750c
Author: Xinyi Zou
AuthorDate: Fri Jul 5 15:52:33 2024 +0800
[opt](memory) Purge all unused Jemalloc dirty pages for all arenas when
exceed limit (#37164)
add BE conf je_dirty_pages_mem_limit_percent, default 5% * mem_limit,
purge all unused Jemalloc dirty pages for all arenas when exceed
je_dirty_pages_mem_limit and process exceed soft limit.
---
be/src/common/config.cpp | 1 +
be/src/common/config.h | 2 ++
be/src/common/daemon.cpp | 27 +++
be/src/util/mem_info.cpp | 6 ++
be/src/util/mem_info.h | 8
5 files changed, 32 insertions(+), 12 deletions(-)
diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp
index 61a460d4084..a2e9729199d 100644
--- a/be/src/common/config.cpp
+++ b/be/src/common/config.cpp
@@ -568,6 +568,7 @@ DEFINE_String(pprof_profile_dir, "${DORIS_HOME}/log");
// for jeprofile in jemalloc
DEFINE_mString(jeprofile_dir, "${DORIS_HOME}/log");
DEFINE_mBool(enable_je_purge_dirty_pages, "true");
+DEFINE_mString(je_dirty_pages_mem_limit_percent, "5%");
// to forward compatibility, will be removed later
DEFINE_mBool(enable_token_check, "true");
diff --git a/be/src/common/config.h b/be/src/common/config.h
index b30cd6265f3..e450f220427 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -619,6 +619,8 @@ DECLARE_String(pprof_profile_dir);
DECLARE_mString(jeprofile_dir);
// Purge all unused dirty pages for all arenas.
DECLARE_mBool(enable_je_purge_dirty_pages);
+// Purge all unused Jemalloc dirty pages for all arenas when exceed
je_dirty_pages_mem_limit and process exceed soft limit.
+DECLARE_mString(je_dirty_pages_mem_limit_percent);
// to forward compatibility, will be removed later
DECLARE_mBool(enable_token_check);
diff --git a/be/src/common/daemon.cpp b/be/src/common/daemon.cpp
index 44f21c47e79..757b1605688 100644
--- a/be/src/common/daemon.cpp
+++ b/be/src/common/daemon.cpp
@@ -212,26 +212,29 @@ void Daemon::memory_maintenance_thread() {
doris::PerfCounters::refresh_proc_status();
doris::MemInfo::refresh_proc_meminfo();
doris::GlobalMemoryArbitrator::reset_refresh_interval_memory_growth();
+
ExecEnv::GetInstance()->brpc_iobuf_block_memory_tracker()->set_consumption(
+butil::IOBuf::block_memory());
+// Refresh allocator memory metrics.
+#if !defined(ADDRESS_SANITIZER) && !defined(LEAK_SANITIZER) &&
!defined(THREAD_SANITIZER)
+doris::MemInfo::refresh_allocator_mem();
+#ifdef USE_JEMALLOC
+if (doris::MemInfo::je_dirty_pages_mem() >
doris::MemInfo::je_dirty_pages_mem_limit() &&
+GlobalMemoryArbitrator::is_exceed_soft_mem_limit()) {
+doris::MemInfo::notify_je_purge_dirty_pages();
+}
+#endif
+if (config::enable_system_metrics) {
+
DorisMetrics::instance()->system_metrics()->update_allocator_metrics();
+}
+#endif
// Update and print memory stat when the memory changes by 256M.
if (abs(last_print_proc_mem - PerfCounters::get_vm_rss()) > 268435456)
{
last_print_proc_mem = PerfCounters::get_vm_rss();
doris::MemTrackerLimiter::clean_tracker_limiter_group();
doris::MemTrackerLimiter::enable_print_log_process_usage();
-
// Refresh mem tracker each type counter.
doris::MemTrackerLimiter::refresh_global_counter();
-
-// Refresh allocator memory metrics.
-#if !defined(ADDRESS_SANITIZER) && !defined(LEAK_SANITIZER) &&
!defined(THREAD_SANITIZER)
-doris::MemInfo::refresh_allocator_mem();
-if (config::enable_system_metrics) {
-
DorisMetrics::instance()->system_metrics()->update_allocator_metrics();
-}
-#endif
-
-
ExecEnv::GetInstance()->brpc_iobuf_block_memory_tracker()->set_consumption(
-butil::IOBuf::block_memory());
LOG(INFO) << doris::GlobalMemoryArbitrator::
process_mem_log_str(); // print mem log when
memory state by 256M
}
diff --git a/be/src/util/mem_info.cpp b/be/src/util/mem_info.cpp
index 45e609d7100..57e48e947ff 100644
--- a/be/src/util/mem_info.cpp
+++ b/be/src/util/mem_info.cpp
@@ -52,6 +52,8 @@ std::atomic MemInfo::_s_mem_limit =
std::numeric_limits::max()
std::atomic MemInfo::_s_soft_mem_limit =
std::numeric_limits::max();
std::atomic MemInfo::_s_allocator_cache_mem = 0;
+std::atomic MemInfo::_s_je_dirty_pages_mem =
std::numeric_limits::min();
+std::atomic MemInfo::_s_je_dirty_pages_m
Re: [PR] [branch-2.1] Picks "[fix](regression) Fix p0 case test_modify_reorder_column #37256" [doris]
bobhan1 commented on PR #37332: URL: https://github.com/apache/doris/pull/37332#issuecomment-2210380438 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth)fix fe can not restart when replay create row policy log [doris]
doris-robot commented on PR #37342: URL: https://github.com/apache/doris/pull/37342#issuecomment-2210380505 TPC-H: Total hot run time: 39747 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 82407cf3fa8eaf7a96348736b8f81524803d3bdd, data reload: false -- Round 1 -- q1 18191 443543834383 q2 2519199 201 199 q3 11065 120110811081 q4 10857 813 791 791 q5 7758268226672667 q6 226 144 141 141 q7 968 616 633 616 q8 9598204420602044 q9 8765647064726470 q10 8816369636513651 q11 497 243 249 243 q12 395 248 236 236 q13 18839 298729972987 q14 268 230 226 226 q15 535 487 495 487 q16 521 373 380 373 q17 964 661 746 661 q18 8134740673497349 q19 7325147513081308 q20 702 324 324 324 q21 4976316638723166 q22 406 344 344 344 Total cold run time: 122325 ms Total hot run time: 39747 ms - Round 2, with runtime_filter_mode=off - q1 4398425142214221 q2 396 271 273 271 q3 2952277727212721 q4 1834159915811581 q5 5256526453865264 q6 217 130 130 130 q7 2076177117691769 q8 3201331733073307 q9 8341834883668348 q10 3851361136023602 q11 575 503 500 500 q12 756 610 613 610 q13 17396 298429782978 q14 291 273 261 261 q15 504 471 476 471 q16 473 409 424 409 q17 1768149314611461 q18 7645749573937393 q19 1687169315511551 q20 1971179817651765 q21 10141 477346524652 q22 618 552 579 552 Total cold run time: 76347 ms Total hot run time: 53817 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] set grpc channel's keepAliveTime and remove proxy on InterruptedExcep… [doris]
BiteThet commented on PR #37304: URL: https://github.com/apache/doris/pull/37304#issuecomment-2210378326 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](ES Catalog)Support control scroll level by config [doris]
doris-robot commented on PR #37180: URL: https://github.com/apache/doris/pull/37180#issuecomment-2210378514 TPC-DS: Total hot run time: 174541 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6f0581622845c800b39c09e46fe580309a2cb4d7, data reload: false query1 925 400 387 387 query2 6456263324212421 query3 6639207 217 207 query4 28168 17535 17397 17397 query5 3697481 490 481 query6 300 177 167 167 query7 4598304 294 294 query8 309 287 300 287 query9 8542246924472447 query10 565 305 274 274 query11 12707 10024 10080 10024 query12 120 85 91 85 query13 1653391 384 384 query14 10309 748678777486 query15 238 184 181 181 query16 7426328 311 311 query17 1349536 522 522 query18 1902287 280 280 query19 193 153 147 147 query20 100 87 84 84 query21 217 147 135 135 query22 392540933925 query23 34232 33814 33632 33632 query24 11091 288729042887 query25 600 419 378 378 query26 715 163 159 159 query27 2366323 339 323 query28 5969217721562156 query29 919 622 650 622 query30 269 159 163 159 query31 967 789 785 785 query32 98 55 54 54 query33 770 308 310 308 query34 952 509 508 508 query35 773 663 653 653 query36 1166995 967 967 query37 143 87 86 86 query38 2980285428112811 query39 955 861 848 848 query40 206 128 128 128 query41 51 51 51 51 query42 113 98 96 96 query43 606 548 582 548 query44 1247709 724 709 query45 188 170 163 163 query46 1086701 694 694 query47 1857173618201736 query48 377 297 301 297 query49 851 418 422 418 query50 768 388 388 388 query51 6987682068306820 query52 109 93 92 92 query53 362 288 291 288 query54 925 458 446 446 query55 74 72 74 72 query56 297 271 273 271 query57 1136105510661055 query58 271 257 261 257 query59 3586328434603284 query60 321 280 277 277 query61 105 93 98 93 query62 636 450 452 450 query63 333 292 294 292 query64 8596227917441744 query65 3321318831403140 query66 801 344 396 344 query67 15618 14967 14916 14916 query68 4823531 525 525 query69 696 427 349 349 query70 1199112711571127 query71 445 283 277 277 query72 8257581052975297 query73 777 322 324 322 query74 5896542655265426 query75 3547269927292699 query76 3076918 937 918 query77 645 319 311 311 query78 9979909010158 9090 query79 3728533 518 518 query80 1551483 490 483 query81 602 240 235 235 query82 1444116 110 110 query83 333 178 171 171 query84 280 95 91 91 query85 1336306 304 304 query86 469 336 314 314 query87 3284309731013097 query88 4150246624642464 query89 500 385 384 384 query90 1802201 190 190 query91 136 105 102 102 query92 58 49 51 49 query93 3231503 505 503 query94 1092215 230 215 query95 414 321 330 321 query96 609 269 268 268 query97 3200302630043004 query98 227 205 197 197 query99 1131871 849 849 Total cold run time: 285567 ms Total hot run time: 174541 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [opt](memory) Purge all unused Jemalloc dirty pages for all arenas when exceed limit [doris]
xinyiZzz merged PR #37164: URL: https://github.com/apache/doris/pull/37164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](memory) `max_sys_mem_available_low_water_mark_bytes`default value modified to 6.4G [doris]
xinyiZzz merged PR #37174: URL: https://github.com/apache/doris/pull/37174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [opt](memory) `max_sys_mem_available_low_water_mark_bytes`default value modified to 6.4G (#37174)
This is an automated email from the ASF dual-hosted git repository. zouxinyi pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 473c7ed5164 [opt](memory) `max_sys_mem_available_low_water_mark_bytes`default value modified to 6.4G (#37174) 473c7ed5164 is described below commit 473c7ed51641db57f310b3829e4b05c190cf0faa Author: Xinyi Zou AuthorDate: Fri Jul 5 15:53:02 2024 +0800 [opt](memory) `max_sys_mem_available_low_water_mark_bytes`default value modified to 6.4G (#37174) // The maximum low water mark of the system `/proc/meminfo/MemAvailable`, Unit byte, default 6.4G, // actual low water mark=min(6.4G, MemTotal * 10%), avoid wasting too much memory on machines // with large memory larger than 64G. // Turn up max. On machines with more than 64G memory, more memory buffers will be reserved for Full GC. // Turn down max. will use as much memory as possible. --- be/src/common/config.cpp | 7 +-- be/src/common/config.h | 8 2 files changed, 5 insertions(+), 10 deletions(-) diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp index a2e9729199d..a5677721326 100644 --- a/be/src/common/config.cpp +++ b/be/src/common/config.cpp @@ -116,12 +116,7 @@ DEFINE_mInt32(max_fill_rate, "2"); DEFINE_mInt32(double_resize_threshold, "23"); -// The maximum low water mark of the system `/proc/meminfo/MemAvailable`, Unit byte, default 1.6G, -// actual low water mark=min(1.6G, MemTotal * 10%), avoid wasting too much memory on machines -// with large memory larger than 16G. -// Turn up max. On machines with more than 16G memory, more memory buffers will be reserved for Full GC. -// Turn down max. will use as much memory as possible. -DEFINE_Int64(max_sys_mem_available_low_water_mark_bytes, "1717986918"); +DEFINE_Int64(max_sys_mem_available_low_water_mark_bytes, "6871947673"); // The size of the memory that gc wants to release each time, as a percentage of the mem limit. DEFINE_mString(process_minor_gc_size, "10%"); diff --git a/be/src/common/config.h b/be/src/common/config.h index e450f220427..0d424940cfc 100644 --- a/be/src/common/config.h +++ b/be/src/common/config.h @@ -158,10 +158,10 @@ DECLARE_mInt32(max_fill_rate); DECLARE_mInt32(double_resize_threshold); -// The maximum low water mark of the system `/proc/meminfo/MemAvailable`, Unit byte, default 1.6G, -// actual low water mark=min(1.6G, MemTotal * 10%), avoid wasting too much memory on machines -// with large memory larger than 16G. -// Turn up max. On machines with more than 16G memory, more memory buffers will be reserved for Full GC. +// The maximum low water mark of the system `/proc/meminfo/MemAvailable`, Unit byte, default 6.4G, +// actual low water mark=min(6.4G, MemTotal * 10%), avoid wasting too much memory on machines +// with large memory larger than 64G. +// Turn up max. On machines with more than 64G memory, more memory buffers will be reserved for Full GC. // Turn down max. will use as much memory as possible. DECLARE_Int64(max_sys_mem_available_low_water_mark_bytes); - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](planner)fix bug of char(255) toSql [doris]
doris-robot commented on PR #37340: URL: https://github.com/apache/doris/pull/37340#issuecomment-2210381008 TPC-H: Total hot run time: 40034 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 75be8a45b2753539fffdf48bb567740c46286dc6, data reload: false -- Round 1 -- q1 18229 543044224422 q2 3002202 201 201 q3 11363 117511761175 q4 11071 781 795 781 q5 7857270327442703 q6 229 140 143 140 q7 968 606 605 605 q8 9228205020332033 q9 8777643065156430 q10 8864367636993676 q11 465 240 243 240 q12 399 225 240 225 q13 17768 295629842956 q14 280 230 232 230 q15 523 486 510 486 q16 530 372 387 372 q17 964 687 682 682 q18 7950741274347412 q19 2718141213601360 q20 677 332 340 332 q21 4955400132343234 q22 389 344 339 339 Total cold run time: 117206 ms Total hot run time: 40034 ms - Round 2, with runtime_filter_mode=off - q1 4358427842214221 q2 356 270 272 270 q3 2926274227432742 q4 1843162115661566 q5 5195528952745274 q6 223 134 134 134 q7 2136174617391739 q8 3210331433193314 q9 8390833583998335 q10 3905367036643664 q11 573 490 480 480 q12 811 602 635 602 q13 16413 297829932978 q14 280 259 252 252 q15 520 467 476 467 q16 485 407 416 407 q17 1811147314621462 q18 7781741174887411 q19 4914154314851485 q20 1991177117651765 q21 4782474845404540 q22 610 555 511 511 Total cold run time: 73513 ms Total hot run time: 53619 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](suites) Allow change db name in get_ccr_body [doris]
github-actions[bot] commented on PR #37324: URL: https://github.com/apache/doris/pull/37324#issuecomment-2210381474 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](suites) Allow change db name in get_ccr_body [doris]
github-actions[bot] commented on PR #37324: URL: https://github.com/apache/doris/pull/37324#issuecomment-2210381532 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Add ATTN logs about partition range [doris]
doris-robot commented on PR #37338: URL: https://github.com/apache/doris/pull/37338#issuecomment-2210383807 ClickBench: Total hot run time: 31.02 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d16bfa6a1cea37d27a0369633beb2e9ce56025f2, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.230.050.05 query4 1.670.090.08 query5 0.500.470.49 query6 1.120.720.72 query7 0.020.010.01 query8 0.050.050.05 query9 0.560.480.49 query10 0.550.560.55 query11 0.160.110.12 query12 0.150.120.12 query13 0.600.580.58 query14 0.770.800.76 query15 0.850.810.82 query16 0.370.370.35 query17 1.060.950.98 query18 0.220.250.24 query19 1.761.711.65 query20 0.020.010.01 query21 15.41 0.740.64 query22 4.096.702.59 query23 18.32 1.351.20 query24 2.100.210.22 query25 0.160.080.08 query26 0.290.200.20 query27 0.460.240.24 query28 13.31 1.000.98 query29 12.61 3.343.33 query30 0.250.060.06 query31 2.870.390.39 query32 3.270.480.47 query33 2.862.842.92 query34 17.13 4.344.31 query35 4.374.434.40 query36 0.640.460.50 query37 0.180.150.15 query38 0.150.140.14 query39 0.050.030.04 query40 0.150.130.13 query41 0.090.040.05 query42 0.060.040.04 query43 0.040.040.03 Total cold run time: 109.64 s Total hot run time: 31.02 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](suites) Allow change db name in get_ccr_body [doris]
dataroaring merged PR #37324: URL: https://github.com/apache/doris/pull/37324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (473c7ed5164 -> 9c2fa630c3d)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 473c7ed5164 [opt](memory) `max_sys_mem_available_low_water_mark_bytes`default value modified to 6.4G (#37174) add 9c2fa630c3d [chore](suites) Allow change db name in get_ccr_body (#37324) No new revisions were added by this update. Summary of changes: .../groovy/org/apache/doris/regression/suite/Suite.groovy | 12 .../org/apache/doris/regression/suite/SuiteContext.groovy | 14 ++ 2 files changed, 18 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Add ATTN logs about partition range [doris]
github-actions[bot] commented on PR #37338: URL: https://github.com/apache/doris/pull/37338#issuecomment-2210384521 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](fe) Add ATTN logs about partition range [doris]
github-actions[bot] commented on PR #37338: URL: https://github.com/apache/doris/pull/37338#issuecomment-2210384572 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](binlog) gc dropped partition records in disable binlog gc [doris]
dataroaring merged PR #37326: URL: https://github.com/apache/doris/pull/37326 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (9c2fa630c3d -> f5c18ee9321)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 9c2fa630c3d [chore](suites) Allow change db name in get_ccr_body (#37324) add f5c18ee9321 [fix](binlog) gc dropped partition records in disable binlog gc (#37326) No new revisions were added by this update. Summary of changes: fe/fe-core/src/main/java/org/apache/doris/binlog/DBBinlog.java | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](alter) support add partitions in batch [doris]
doris-robot commented on PR #37114: URL: https://github.com/apache/doris/pull/37114#issuecomment-2210387078 TPC-DS: Total hot run time: 170831 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit f2cabc8462c9673ad860c37e3dc8eaad7033f604, data reload: false query1 933 367 362 362 query2 6458259524562456 query3 6625208 213 208 query4 27971 17591 17293 17293 query5 3692491 473 473 query6 258 163 164 163 query7 4592314 300 300 query8 324 325 301 301 query9 8573244224362436 query10 571 325 283 283 query11 11702 10156 10115 10115 query12 118 86 88 86 query13 1658376 372 372 query14 10100 679177106791 query15 224 184 192 184 query16 7749313 314 313 query17 1803554 535 535 query18 1863289 284 284 query19 203 158 149 149 query20 92 84 84 84 query21 213 132 130 130 query22 4450409541304095 query23 33806 33782 33620 33620 query24 11039 286128612861 query25 656 451 423 423 query26 1186156 157 156 query27 2436319 337 319 query28 7366215621412141 query29 920 664 655 655 query30 249 162 153 153 query31 947 799 758 758 query32 99 55 62 55 query33 773 317 310 310 query34 886 505 512 505 query35 746 669 656 656 query36 1127987 976 976 query37 158 82 84 82 query38 2994286228202820 query39 902 830 796 796 query40 212 129 132 129 query41 62 53 58 53 query42 122 98 105 98 query43 588 586 566 566 query44 1210718 724 718 query45 190 168 165 165 query46 1073707 730 707 query47 1900178418041784 query48 378 301 395 301 query49 847 405 424 405 query50 768 382 389 382 query51 6833683967466746 query52 103 91 92 91 query53 360 288 292 288 query54 869 438 443 438 query55 76 72 68 68 query56 320 274 278 274 query57 1130106110621061 query58 250 232 260 232 query59 3481351131773177 query60 295 283 292 283 query61 94 89 95 89 query62 611 436 434 434 query63 324 281 291 281 query64 8796226917241724 query65 3140314731003100 query66 824 324 327 324 query67 15972 14989 15270 14989 query68 7491568 542 542 query69 734 407 343 343 query70 1158105411381054 query71 531 284 280 280 query72 8619537327742774 query73 813 325 325 325 query74 5899547155945471 query75 5081263827052638 query76 4817957 1048957 query77 776 298 306 298 query78 9473909688368836 query79 7586518 516 516 query80 1035467 478 467 query81 577 216 236 216 query82 769 110 109 109 query83 332 167 165 165 query84 277 86 90 86 query85 1362314 297 297 query86 398 306 318 306 query87 3280306030573057 query88 4624247424432443 query89 485 378 381 378 query90 2010185 189 185 query91 134 104 101 101 query92 63 50 48 48 query93 5904508 508 508 query94 1268209 206 206 query95 410 326 325 325 query96 607 267 274 267 query97 3188300429872987 query98 218 211 194 194 query99 1163877 846 846 Total cold run time: 298372 ms Total hot run time: 170831 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [feature](ES Catalog)Support control scroll level by config [doris]
doris-robot commented on PR #37180: URL: https://github.com/apache/doris/pull/37180#issuecomment-2210386404 ClickBench: Total hot run time: 30.18 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6f0581622845c800b39c09e46fe580309a2cb4d7, data reload: false query1 0.040.030.03 query2 0.070.040.04 query3 0.220.050.05 query4 1.670.080.08 query5 0.490.480.48 query6 1.140.760.72 query7 0.020.010.02 query8 0.050.040.04 query9 0.550.490.50 query10 0.550.540.53 query11 0.160.110.11 query12 0.150.120.12 query13 0.590.580.58 query14 0.770.770.77 query15 0.850.820.83 query16 0.360.370.36 query17 0.990.970.99 query18 0.220.240.27 query19 1.811.701.75 query20 0.010.010.01 query21 15.44 0.750.65 query22 4.127.511.60 query23 18.26 1.311.23 query24 2.100.220.24 query25 0.160.090.08 query26 0.290.220.21 query27 0.460.250.24 query28 13.22 1.001.00 query29 12.61 3.303.29 query30 0.250.060.05 query31 2.860.400.39 query32 3.270.480.47 query33 2.922.852.86 query34 17.11 4.364.36 query35 4.444.384.47 query36 0.650.460.47 query37 0.190.150.16 query38 0.160.140.15 query39 0.040.030.04 query40 0.160.120.12 query41 0.090.040.05 query42 0.060.050.04 query43 0.040.040.03 Total cold run time: 109.61 s Total hot run time: 30.18 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](test) shorten the external p0 running time [doris]
morningman commented on PR #37320: URL: https://github.com/apache/doris/pull/37320#issuecomment-2210388044 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](function) opt aes_encrypt/decrypt function to handle const column [doris]
github-actions[bot] commented on PR #37194: URL: https://github.com/apache/doris/pull/37194#issuecomment-2210390606 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](function) opt aes_encrypt/decrypt function to handle const column [doris]
github-actions[bot] commented on PR #37194: URL: https://github.com/apache/doris/pull/37194#issuecomment-2210390654 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](jdbc catalog) Added support for Oracle Raw type [doris]
zy-kkk commented on PR #37078: URL: https://github.com/apache/doris/pull/37078#issuecomment-2210394977 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](alter) support add partitions in batch [doris]
doris-robot commented on PR #37114: URL: https://github.com/apache/doris/pull/37114#issuecomment-2210395431 ClickBench: Total hot run time: 30.82 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit f2cabc8462c9673ad860c37e3dc8eaad7033f604, data reload: false query1 0.040.040.03 query2 0.090.040.04 query3 0.230.050.05 query4 1.670.070.08 query5 0.510.470.49 query6 1.150.730.73 query7 0.020.020.01 query8 0.050.050.05 query9 0.550.510.50 query10 0.550.550.54 query11 0.150.110.12 query12 0.150.130.12 query13 0.590.580.58 query14 0.760.780.78 query15 0.840.800.83 query16 0.350.370.36 query17 1.011.011.03 query18 0.220.260.23 query19 1.791.831.78 query20 0.020.010.01 query21 15.38 0.720.65 query22 3.837.472.05 query23 18.26 1.361.25 query24 2.070.240.22 query25 0.160.080.09 query26 0.300.210.20 query27 0.460.230.23 query28 13.51 1.021.00 query29 12.86 3.373.34 query30 0.250.060.06 query31 2.870.390.39 query32 3.260.490.47 query33 2.892.862.93 query34 17.20 4.354.39 query35 4.404.384.48 query36 0.640.460.47 query37 0.180.150.15 query38 0.150.150.13 query39 0.050.030.03 query40 0.160.120.14 query41 0.090.050.05 query42 0.060.040.05 query43 0.050.040.05 Total cold run time: 109.82 s Total hot run time: 30.82 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth)fix fe can not restart when replay create row policy log [doris]
doris-robot commented on PR #37342: URL: https://github.com/apache/doris/pull/37342#issuecomment-2210397646 TPC-DS: Total hot run time: 172719 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 82407cf3fa8eaf7a96348736b8f81524803d3bdd, data reload: false query1 907 369 386 369 query2 6487258225342534 query3 6644201 209 201 query4 28641 17404 17289 17289 query5 4214475 477 475 query6 268 162 171 162 query7 4590299 293 293 query8 322 305 294 294 query9 8574246424342434 query10 618 307 280 280 query11 10970 995010019 9950 query12 147 86 86 86 query13 1659372 377 372 query14 10401 761782797617 query15 235 183 184 183 query16 7256328 313 313 query17 1801588 544 544 query18 1189283 284 283 query19 199 153 157 153 query20 93 82 83 82 query21 218 140 139 139 query22 4165406340014001 query23 33698 32933 33204 32933 query24 12044 291028382838 query25 671 389 387 387 query26 1679156 160 156 query27 2600337 337 337 query28 7090212720892089 query29 996 643 633 633 query30 290 151 147 147 query31 954 724 741 724 query32 98 54 55 54 query33 794 316 315 315 query34 921 497 501 497 query35 736 609 613 609 query36 1110951 950 950 query37 154 78 79 78 query38 2882271627222716 query39 876 814 805 805 query40 278 127 126 126 query41 56 54 53 53 query42 121 95 103 95 query43 616 602 565 565 query44 1238723 727 723 query45 190 167 164 164 query46 1076719 699 699 query47 1829175017621750 query48 372 302 293 293 query49 1202420 425 420 query50 776 400 398 398 query51 6931671966156615 query52 108 95 99 95 query53 356 297 292 292 query54 1120479 465 465 query55 76 76 76 76 query56 327 291 292 291 query57 1130104611691046 query58 260 251 247 247 query59 3467332031883188 query60 322 276 273 273 query61 97 107 96 96 query62 653 450 447 447 query63 326 290 295 290 query64 9941230017451745 query65 3175311331363113 query66 1271340 345 340 query67 15697 14815 14773 14773 query68 8587545 556 545 query69 776 431 368 368 query70 1109111611041104 query71 530 278 276 276 query72 8620521853245218 query73 838 327 326 326 query74 5879547755825477 query75 5188263426662634 query76 5002972 882 882 query77 794 296 297 296 query78 9698902888248824 query79 8758513 517 513 query80 1125492 486 486 query81 576 216 226 216 query82 786 114 103 103 query83 353 170 168 168 query84 279 85 133 85 query85 1397315 292 292 query86 395 331 304 304 query87 3365319531743174 query88 5227244524422442 query89 542 380 380 380 query90 2146187 186 186 query91 130 102 103 102 query92 61 50 49 49 query93 6806514 504 504 query94 1384205 207 205 query95 408 312 322 312 query96 616 313 270 270 query97 3186294630022946 query98 222 203 257 203 query99 1205853 829 829 Total cold run time: 306190 ms Total hot run time: 172719 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
[PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
XieJiann opened a new pull request, #37343: URL: https://github.com/apache/doris/pull/37343 ## Proposed changes This PR refines the LogicalJoin class by introducing robust input validation. Key improvements: * Implement precise checks for join input validity * Ensure consistency between input slots and output sets * Gracefully handle various join scenarios (left/right) These enhancements bolster query integrity and optimize join operations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
doris-robot commented on PR #37343: URL: https://github.com/apache/doris/pull/37343#issuecomment-2210398268 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](planner)fix bug of char(255) toSql [doris]
doris-robot commented on PR #37340: URL: https://github.com/apache/doris/pull/37340#issuecomment-2210398317 TPC-DS: Total hot run time: 172907 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 75be8a45b2753539fffdf48bb567740c46286dc6, data reload: false query1 919 359 365 359 query2 6483256124182418 query3 6651199 218 199 query4 28326 17457 17148 17148 query5 4226499 484 484 query6 271 156 157 156 query7 4582296 306 296 query8 332 315 286 286 query9 8590245924322432 query10 620 296 280 280 query11 10639 984498909844 query12 149 84 81 81 query13 1652377 381 377 query14 10274 756977767569 query15 234 188 184 184 query16 7222314 323 314 query17 1802552 535 535 query18 1751295 281 281 query19 198 149 156 149 query20 88 80 82 80 query21 209 136 124 124 query22 4260402840374028 query23 33630 33143 33130 33130 query24 12085 289128262826 query25 679 385 388 385 query26 1799161 158 158 query27 2796318 316 316 query28 7103208320822082 query29 1040638 632 632 query30 291 150 148 148 query31 983 761 749 749 query32 96 55 61 55 query33 785 320 309 309 query34 932 493 491 491 query35 739 668 623 623 query36 1059968 933 933 query37 246 77 78 77 query38 2864270527062705 query39 902 813 808 808 query40 281 133 126 126 query41 57 47 56 47 query42 119 101 99 99 query43 605 552 562 552 query44 1227740 725 725 query45 202 163 158 158 query46 1107722 743 722 query47 1846177717701770 query48 384 303 305 303 query49 1217430 428 428 query50 773 400 413 400 query51 6895677567756775 query52 110 100 97 97 query53 366 300 294 294 query54 1067470 460 460 query55 79 77 76 76 query56 315 296 284 284 query57 1221105311921053 query58 260 245 245 245 query59 3612325733513257 query60 299 278 272 272 query61 94 96 106 96 query62 653 448 450 448 query63 322 292 296 292 query64 9972229817651765 query65 3173310430583058 query66 1362368 332 332 query67 15885 14834 14894 14834 query68 8774539 554 539 query69 737 451 355 355 query70 1329113911231123 query71 537 282 278 278 query72 8868549154725472 query73 2243324 332 324 query74 6027552455295524 query75 5056260626922606 query76 5360933 921 921 query77 791 300 304 300 query78 9534897187698769 query79 9454522 515 515 query80 950 468 473 468 query81 577 214 222 214 query82 259 106 106 106 query83 337 165 164 164 query84 278 89 86 86 query85 993 350 298 298 query86 357 331 302 302 query87 3290311630543054 query88 5045245124792451 query89 507 373 393 373 query90 2043184 183 183 query91 137 105 102 102 query92 62 49 50 49 query93 6304511 507 507 query94 1451206 213 206 query95 397 321 321 321 query96 612 279 266 266 query97 3179302229962996 query98 224 208 199 199 query99 1140817 843 817 Total cold run time: 307862 ms Total hot run time: 172907 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
XieJiann commented on PR #37343: URL: https://github.com/apache/doris/pull/37343#issuecomment-2210398475 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](brpc) support set brpc_idle_timeout_sec [doris]
BiteThet commented on PR #37333: URL: https://github.com/apache/doris/pull/37333#issuecomment-2210398919 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] errors while publish version to all backends [doris]
353355756 commented on issue #35455: URL: https://github.com/apache/doris/issues/35455#issuecomment-2210399013 Has it been resolved? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] 2.1 fix star except to sql [doris]
doris-robot commented on PR #37344: URL: https://github.com/apache/doris/pull/37344#issuecomment-2210400965 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] 2.1 fix star except to sql [doris]
feiniaofeiafei commented on PR #37344: URL: https://github.com/apache/doris/pull/37344#issuecomment-2210401029 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](file cache) write segment data asynchronous and merge small cache files [doris]
AshinGau commented on PR #37190: URL: https://github.com/apache/doris/pull/37190#issuecomment-2210402576 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] 2.1 fix star except to sql [doris]
feiniaofeiafei opened a new pull request, #37344: URL: https://github.com/apache/doris/pull/37344 cherry-pick #37274 to branch-2.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](planner)fix bug of char(255) toSql [doris]
doris-robot commented on PR #37340: URL: https://github.com/apache/doris/pull/37340#issuecomment-2210407124 ClickBench: Total hot run time: 30.46 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 75be8a45b2753539fffdf48bb567740c46286dc6, data reload: false query1 0.040.030.03 query2 0.080.040.04 query3 0.220.060.06 query4 1.670.080.09 query5 0.500.490.49 query6 1.140.740.73 query7 0.020.010.01 query8 0.050.050.05 query9 0.560.490.48 query10 0.530.530.54 query11 0.150.110.11 query12 0.160.130.12 query13 0.600.590.58 query14 0.770.810.77 query15 0.860.840.84 query16 0.370.370.37 query17 1.001.001.05 query18 0.240.250.24 query19 1.791.681.67 query20 0.020.010.01 query21 15.40 0.770.64 query22 4.447.661.37 query23 18.27 1.371.39 query24 2.180.220.22 query25 0.150.080.08 query26 0.310.220.22 query27 0.470.240.23 query28 13.20 1.081.02 query29 12.64 3.403.37 query30 0.260.070.05 query31 2.870.410.39 query32 3.280.510.48 query33 2.972.972.95 query34 17.19 4.434.38 query35 4.414.444.46 query36 0.660.470.50 query37 0.180.150.15 query38 0.150.150.15 query39 0.050.030.03 query40 0.160.120.12 query41 0.100.040.05 query42 0.050.040.05 query43 0.050.040.04 Total cold run time: 110.21 s Total hot run time: 30.46 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](auth)fix fe can not restart when replay create row policy log [doris]
doris-robot commented on PR #37342: URL: https://github.com/apache/doris/pull/37342#issuecomment-2210406469 ClickBench: Total hot run time: 30.67 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 82407cf3fa8eaf7a96348736b8f81524803d3bdd, data reload: false query1 0.040.030.03 query2 0.080.050.04 query3 0.220.050.05 query4 1.680.110.08 query5 0.510.490.48 query6 1.160.730.71 query7 0.020.010.02 query8 0.050.040.04 query9 0.550.500.50 query10 0.530.550.54 query11 0.160.110.11 query12 0.150.130.13 query13 0.590.580.58 query14 0.760.780.76 query15 0.860.820.82 query16 0.370.370.36 query17 1.020.961.02 query18 0.220.250.23 query19 1.871.681.73 query20 0.020.010.01 query21 15.45 0.770.64 query22 4.107.511.95 query23 18.28 1.421.27 query24 2.090.220.22 query25 0.160.080.08 query26 0.290.200.20 query27 0.460.230.23 query28 13.29 1.021.00 query29 12.59 3.363.34 query30 0.250.050.05 query31 2.880.390.38 query32 3.270.480.47 query33 2.922.942.94 query34 17.04 4.354.34 query35 4.434.374.38 query36 0.660.480.47 query37 0.190.150.15 query38 0.150.150.15 query39 0.040.030.04 query40 0.150.130.13 query41 0.090.050.05 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 109.75 s Total hot run time: 30.67 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](brpc) support set brpc_idle_timeout_sec [doris]
github-actions[bot] commented on PR #37333: URL: https://github.com/apache/doris/pull/37333#issuecomment-2210408690 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](test)Fix index_change_on_new_column on cloud [doris]
qidaye opened a new pull request, #37345: URL: https://github.com/apache/doris/pull/37345 ## Proposed changes Fix index_change_on_new_column on cloud mode. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test)Fix index_change_on_new_column on cloud [doris]
doris-robot commented on PR #37345: URL: https://github.com/apache/doris/pull/37345#issuecomment-2210409465 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test)Fix index_change_on_new_column on cloud [doris]
qidaye commented on PR #37345: URL: https://github.com/apache/doris/pull/37345#issuecomment-2210409728 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](file cache) write segment data asynchronous and merge small cache files [doris]
github-actions[bot] commented on code in PR #37190:
URL: https://github.com/apache/doris/pull/37190#discussion_r1666488710
##
be/src/io/cache/block/block_lru_file_cache.cpp:
##
@@ -460,6 +461,156 @@ size_t LRUFileCache::try_release() {
return trash.size();
}
+std::pair LRUFileCache::try_merge() {
+std::unordered_map>,
HashCachedFileKey> merged_files;
+{
+std::lock_guard l(_mutex);
+for (auto& [key, segments] : _files) {
+std::vector> merged_blocks =
find_continuous_cells(segments);
+if (!merged_blocks.empty()) {
+merged_files[key] = merged_blocks;
+}
+}
+}
+if (!merged_files.empty()) {
+return merge_continuous_cells(merged_files);
+}
+return {0, 0};
+}
+
+std::vector> LRUFileCache::find_continuous_cells(
+const FileBlocksByOffset& segments) {
+std::vector> merged_blocks;
+std::vector continuous_blocks;
+size_t end_offset;
+size_t merged_size = 0;
+for (auto& [offset, cell] : segments) {
+if (cell.file_block->cache_type() != CacheType::NORMAL) {
+break;
+}
+if (cell.releasable()) {
+if (continuous_blocks.empty()) {
+continuous_blocks.push_back(offset);
+end_offset = offset + cell.size();
+merged_size = cell.size();
+} else if (offset == end_offset) {
+if (merged_size >= MAX_MERGED_SIZE) {
+if (continuous_blocks.size() > 1) {
+merged_blocks.push_back(continuous_blocks);
+}
+continuous_blocks.clear();
+merged_size = cell.size();
+} else {
+merged_size += cell.size();
+}
+continuous_blocks.push_back(offset);
+end_offset = offset + cell.size();
+} else {
+if (continuous_blocks.size() > 1) {
+merged_blocks.push_back(continuous_blocks);
+}
+continuous_blocks.clear();
+continuous_blocks.push_back(offset);
+end_offset = offset + cell.size();
+merged_size = cell.size();
+}
+}
+}
+if (continuous_blocks.size() > 1) {
+merged_blocks.push_back(continuous_blocks);
+}
+return merged_blocks;
+}
+
+std::pair LRUFileCache::merge_continuous_cells(
Review Comment:
warning: function 'merge_continuous_cells' exceeds recommended
size/complexity thresholds [readability-function-size]
```cpp
std::pair LRUFileCache::merge_continuous_cells(
^
```
Additional context
**be/src/io/cache/block/block_lru_file_cache.cpp:524:** 85 lines including
whitespace and comments (threshold 80)
```cpp
std::pair LRUFileCache::merge_continuous_cells(
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test)Fix index_change_on_new_column on cloud [doris]
github-actions[bot] commented on PR #37345: URL: https://github.com/apache/doris/pull/37345#issuecomment-2210443492 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test)Fix index_change_on_new_column on cloud [doris]
github-actions[bot] commented on PR #37345: URL: https://github.com/apache/doris/pull/37345#issuecomment-2210443548 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
