This is an automated email from the ASF dual-hosted git repository.
zclll pushed a change to branch tpc_preview4
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 1896ff0e9c2 fix query error
discard 4ffbe1686d4 opt continous window
discard 09e0c16fd77 remix shuffle hash
discard 4f27129c1ae [Chore](thirdparty) add crc32c-1.1.2 to thirdparty (#58462)
discard 812df29aa96 contine page
discard e4ab9730ffb fix
discard 5fdbf5de7d5 fix
discard fb07a5b37ad optimize trivial function deserialize_and_merge_vec
discard 08863e00213 eager code 1212: checkStats with one high ndv
discard 4f35e347960 [Improvement](external) Increase scanner concurrency
(#58073) (#58927)
discard a36f689bb04 opt scanner projection
discard 1836bd2333a [opt](agg) Adjust the parameters of
StreamingHtMinReductionEntry
discard 7adbb1ad805 Revert "use bitshuffle to decoding"
discard 4f6a8a756b5 eager-code
discard 5a54f831420 use bitshuffle to decoding
omit d2589b7200c plain lz4 null page encode
omit 9209c20f873 change the null map encode
add f5433371db9 [Improvement](external) Increase scanner concurrency
(#58073) (#58927)
add 51614b6475c [opt](agg) Adjust the parameters of
StreamingHtMinReductionEntry
add 2694f8fd8fc opt scanner projection
add d4f6f553467 eager-agg (#58969)
add 5b704598f10 shape-base
add 49eb9333c2c shape-1212-preview3
add 58c9010179b checkstats
add 6324ec3a2b9 eager code 1212: checkStats with one high ndv
add 30e1194c761 optimize trivial function deserialize_and_merge_vec
add 702c6f7ca7b fix
add 31041739878 fix
add 31d628c3fe5 eager_aggregation_mode does not control push down sum-if
add 047a3ba2f69 do not push agg if all joins are small
add 066b69e915b [Chore](thirdparty) add crc32c-1.1.2 to thirdparty (#58462)
add 5c69a293311 remix shuffle hash
add a3881f8b22b [opt](Arena)Release Arena memory earlier in pipeline
operators. (#59090)
add 45209447bad eager agg: 先检查eager_aggregation_mode, 后检查是否下压大join,
保证mode=1 时可以强制下压
add 54c1a37c2b7 [enhance](nereids_rules) Add rewrite rule
DecomposeRepeatWithPreAggregation (#59148)
add 1d6937553d4 opt continous window
add eff09a822f1 [fix](eager-agg) push through project
add b3aea4f57ff change the null map encode
add 3a2427d08b8 plain lz4 null page encode
add c777b258f66 use bitshuffle to decoding
add 53ac5c8d0b4 Revert "use bitshuffle to decoding"
add 92db0ca2146 contine page
add 362ad1e488e fix query error
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (1896ff0e9c2)
\
N -- N -- N refs/heads/tpc_preview4 (362ad1e488e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/pipeline/dependency.h | 6 +
be/src/pipeline/exec/aggregation_sink_operator.cpp | 46 +--
be/src/pipeline/exec/aggregation_sink_operator.h | 2 -
.../pipeline/exec/aggregation_source_operator.cpp | 13 +-
be/src/pipeline/exec/aggregation_source_operator.h | 1 -
be/src/pipeline/exec/analytic_sink_operator.cpp | 9 +-
be/src/pipeline/exec/analytic_sink_operator.h | 1 -
.../distinct_streaming_aggregation_operator.cpp | 4 +-
.../exec/distinct_streaming_aggregation_operator.h | 2 -
be/src/pipeline/exec/set_sink_operator.cpp | 2 +-
be/src/pipeline/exec/set_sink_operator.h | 1 -
.../exec/streaming_aggregation_operator.cpp | 2 +-
.../pipeline/exec/streaming_aggregation_operator.h | 1 -
.../org/apache/doris/analysis/VirtualSlotRef.java | 3 -
.../glue/translator/ExpressionTranslator.java | 6 -
.../glue/translator/PhysicalPlanTranslator.java | 27 +-
.../glue/translator/PlanTranslatorContext.java | 25 +-
.../doris/nereids/jobs/executor/Rewriter.java | 4 +-
.../org/apache/doris/nereids/rules/RuleType.java | 2 +-
.../nereids/rules/analysis/CheckAfterRewrite.java | 32 +-
.../nereids/rules/analysis/NormalizeRepeat.java | 51 ++-
.../mv/AbstractMaterializedViewAggregateRule.java | 44 +--
.../nereids/rules/exploration/mv/StructInfo.java | 52 +--
.../LogicalRepeatToPhysicalRepeat.java | 1 +
.../rewrite/DecomposeRepeatWithPreAggregation.java | 352 +++++++++++++++++++++
.../nereids/rules/rewrite/ExprIdRewriter.java | 28 +-
.../nereids/rules/rewrite/SetPreAggStatus.java | 35 +-
.../rewrite/eageraggregation/EagerAggRewriter.java | 25 +-
.../eageraggregation/PushDownAggContext.java | 21 +-
.../eageraggregation/PushdownSumIfAggregation.java | 8 +-
.../rewrite/eageraggregation/SumAggWriter.java | 21 +-
.../doris/nereids/stats/ExpressionEstimation.java | 12 +-
.../org/apache/doris/nereids/trees/TreeNode.java | 27 ++
.../nereids/trees/copier/ExpressionDeepCopier.java | 29 --
.../trees/copier/LogicalPlanDeepCopier.java | 4 +-
.../doris/nereids/trees/expressions/Alias.java | 4 +
.../trees/expressions/VirtualSlotReference.java | 170 ----------
.../expressions/functions/scalar/Grouping.java | 2 +-
.../expressions/functions/scalar/GroupingId.java | 2 +-
.../functions/scalar/GroupingScalarFunction.java | 2 +-
.../trees/expressions/functions/udf/JavaUdaf.java | 12 +-
.../trees/expressions/functions/udf/JavaUdf.java | 12 +-
.../trees/expressions/functions/udf/JavaUdtf.java | 12 +-
.../expressions/visitor/ExpressionVisitor.java | 5 -
.../doris/nereids/trees/plans/algebra/Repeat.java | 54 +---
.../nereids/trees/plans/logical/LogicalRepeat.java | 85 +++--
.../plans/physical/PhysicalHashAggregate.java | 10 -
.../trees/plans/physical/PhysicalRepeat.java | 28 +-
.../apache/doris/nereids/util/ExpressionUtils.java | 11 +-
.../org/apache/doris/planner/ResultFileSink.java | 27 --
.../properties/ChildOutputPropertyDeriverTest.java | 9 +
.../DecomposeRepeatWithPreAggregationTest.java | 99 ++++++
.../trees/copier/LogicalPlanDeepCopierTest.java | 4 +
.../decompose_repeat/decompose_repeat.out | 111 +++++++
.../data/shape_check/tpcds_sf1000/shape/query1.out | 2 +-
.../data/shape_check/tpcds_sf1000/shape/query2.out | 24 +-
.../data/shape_check/tpcds_sf1000/shape/query3.out | 19 +-
.../shape_check/tpcds_sf1000/shape/query30.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query32.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query47.out | 37 ++-
.../shape_check/tpcds_sf1000/shape/query53.out | 49 ++-
.../shape_check/tpcds_sf1000/shape/query55.out | 9 +-
.../shape_check/tpcds_sf1000/shape/query57.out | 37 ++-
.../shape_check/tpcds_sf1000/shape/query59.out | 9 +-
.../shape_check/tpcds_sf1000/shape/query63.out | 49 ++-
.../shape_check/tpcds_sf1000/shape/query72.out | 24 +-
.../shape_check/tpcds_sf1000/shape/query81.out | 2 +-
.../shape_check/tpcds_sf1000/shape/query89.out | 39 ++-
.../shape_check/tpcds_sf1000/shape/query92.out | 2 +-
.../decompose_repeat/decompose_repeat.groovy | 41 +++
70 files changed, 1130 insertions(+), 775 deletions(-)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/DecomposeRepeatWithPreAggregation.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/VirtualSlotReference.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/DecomposeRepeatWithPreAggregationTest.java
create mode 100644
regression-test/data/nereids_rules_p0/decompose_repeat/decompose_repeat.out
create mode 100644
regression-test/suites/nereids_rules_p0/decompose_repeat/decompose_repeat.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
