This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from 2913a111f60 [Bug](top-n) init query_ctx runtime predicate before
_build_pipelines #31895
new f5991d8b029 [docs](jdbc catalog) add sqlserver timestamp type mapping
doc (#31832)
new 2e94512e075 [fix](build)Fix index_tool build error (#31834)
new 538032a75c6 [fix](partition) add log when tablet partition id eq 0
(#31796)
new 28f0b7eb321 [Improvement](profile)Add tvf active_be_tasks() #31815
new ff8c679851d [fix](group commit) group commit may heap-use-after-free
if execute plan failed (#31839)
new f897db666c2 [docs](flink-connector) Fix Flink Doris Connector typo
(#31830)
new 2e4201e4066 [opt](session variable) max_msg_size_of_result_receiver
#31809
new d0847a89531 [feat](Nereids): add struct info map in group (#31800)
new b978365a7b0 [fix](cloud) Fix FE cannot work when restarting after
schema change (#31842)
new c7d86c062ad [fix](Nereids) json_object can't be odd parameters, need
even parameters (#31767)
new dc9de4b6b56 [Bug](load) fix wrong data in mv when routine load with
function mapping (#31787)
new 686cbffd8a7 [fix](multi-catalog) fix iceberg defalut credentials
proviers (#31704)
new 5c1c73e5bb4 [Fix](Nereids) fix missing comment when creating table
(#31869)
new 3716f8a171f [Bug](partition) fix npe when prune partition with not
exist partition column in mv #31860
new 1d9e9fc8843 [regression test] Test the unique model by modify a value
type from TINYINT to other type (#31682)
new d276610694e [deps](docker) update docker build env to add jdk17
(#31644)
new ad2f7fc3165 [bugfix](coordinator) should use fragment id not profile
fragment id to cancel fragment (#31852)
new 561709451c5 [fix](Nereids) fix group_concat(distinct) failed (#31873)
new 80c80b67196 [enhancement](Nereids): support more condition
Date/DateTime Literal (#31858)
new 54a8c83f6a1 [enhancement](cloud) add CloudTabletStatMgr to capture
stats in cloud mode (#31818)
new b3db6b2eae1 [improvement](jdbc catalog) opt get db2 schema list & xml
type mapping (#31856)
new 8084a0a74ae [typo](doc) Add a full backup case and modify sidebars
(#31801)
new 326196b93dd [Bug](fix) fix or and "<=>" cause coredump in query
(#31884)
new 0baa00af159 [type](doc) Standardize the name in the sql-dialect
document and modify the format (#31636)
new c83f600c05f [enhancement](nereids) allow reorder mark join (#30644)
new 4d259377f31 [enhancement](nereids) show nullaware semi join in plan
(#31738)
new 6d0c1ad6328 [fix](http stream) http stream support
memtable_on_sink_node header (#31866)
new 73ebeb24a3c [fix](mtmv) support null value in partition for updating
(#31843)
new 13766665149 [fix](money_format) fix money_format #31883
new c0b25869a9e [fix](function) fix date_format function execution error
on fe (#31645)
new 286685ff2fe [doc](apache-superset)add doc for apache superset
compatible (#31314)
new 2f012b834d9 [bug](meta) exit when get RollbackException in observer
(#31687)
new 44f41054d8c [cloud](point query) enable short circuit query in cloud
(#31897)
new 586f155d84e [enhance](mtmv) MTMV Use partial partition of base table
(#31632)
new 917b2011747 [chore](build) Using multithread to accelerate FE
compilation (#31855)
new db65114f77d [Fix](Nereids) fix hint cases with random result (#31865)
new 73f6d9532dc [feat](nereids) support null safe eq runtime filter (FE
part) (#31655)
new 677c9348a7d [fix] (doc)Fix invalid link on benchmark tool page #31920
The 38 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/http/action/http_stream.cpp | 4 +
be/src/index-tools/index_tool.cpp | 2 +-
be/src/olap/data_dir.cpp | 4 +-
be/src/pipeline/exec/scan_operator.cpp | 2 +-
be/src/runtime/group_commit_mgr.cpp | 20 +-
be/src/runtime/group_commit_mgr.h | 3 +-
be/src/runtime/runtime_query_statistics_mgr.cpp | 50 ++++
be/src/runtime/runtime_query_statistics_mgr.h | 5 +
be/src/vec/exec/scan/vmeta_scanner.cpp | 22 +-
be/src/vec/exec/scan/vmeta_scanner.h | 2 +
be/src/vec/exec/scan/vscan_node.cpp | 2 +-
be/src/vec/functions/function.cpp | 2 +-
be/src/vec/functions/function_string.h | 30 ++-
build.sh | 8 +-
docker/compilation/Dockerfile | 21 +-
.../en/community/developer-guide/benchmark-tool.md | 2 +-
.../ecosystem/bi-integration/apache-superset.md | 53 ++++
.../install/source-install/compilation-general.md | 33 ++-
docs/en/docs/lakehouse/multi-catalog/jdbc.md | 41 ++--
docs/en/docs/lakehouse/sql-dialect.md | 44 ++--
.../Backup-and-Restore/BACKUP.md | 7 +
docs/sidebars.json | 2 +-
.../community/developer-guide/benchmark-tool.md | 2 +-
.../ecosystem/bi-integration/apache-superset.md | 57 +++++
docs/zh-CN/docs/ecosystem/flink-doris-connector.md | 2 +-
.../install/source-install/compilation-general.md | 12 +-
docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md | 6 +-
docs/zh-CN/docs/lakehouse/sql-dialect.md | 46 ++--
.../Backup-and-Restore/BACKUP.md | 7 +
.../main/java/org/apache/doris/common/Config.java | 3 +
.../org/apache/doris/analysis/DateLiteral.java | 8 +-
.../doris/catalog/BuiltinTableValuedFunctions.java | 4 +-
.../apache/doris/catalog/CloudTabletStatMgr.java | 273 +++++++++++++++++++++
.../java/org/apache/doris/catalog/Database.java | 5 +
.../main/java/org/apache/doris/catalog/Env.java | 9 +-
.../apache/doris/catalog/ListPartitionItem.java | 21 ++
.../main/java/org/apache/doris/catalog/MTMV.java | 11 +-
.../org/apache/doris/catalog/PartitionItem.java | 14 ++
.../apache/doris/catalog/RangePartitionItem.java | 18 ++
.../java/org/apache/doris/catalog/Replica.java | 17 ++
.../apache/doris/common/util/PropertyAnalyzer.java | 3 +
.../apache/doris/datasource/InternalCatalog.java | 2 +-
.../datasource/iceberg/IcebergExternalCatalog.java | 9 +
.../iceberg/IcebergGlueExternalCatalog.java | 5 +-
.../iceberg/IcebergHMSExternalCatalog.java | 1 +
.../iceberg/IcebergHadoopExternalCatalog.java | 1 +
.../iceberg/IcebergRestExternalCatalog.java | 3 +-
.../datasource/jdbc/client/JdbcDB2Client.java | 36 +++
.../datasource/jdbc/client/JdbcOracleClient.java | 6 +-
.../datasource/property/PropertyConverter.java | 16 +-
.../property/constants/S3Properties.java | 8 +
.../apache/doris/journal/bdbje/BDBJEJournal.java | 7 +
.../src/main/java/org/apache/doris/load/Load.java | 10 +-
.../apache/doris/mtmv/MTMVPartitionSyncConfig.java | 57 +++++
.../doris/mtmv/MTMVPartitionSyncTimeUnit.java | 36 +++
.../org/apache/doris/mtmv/MTMVPartitionUtil.java | 11 +-
.../org/apache/doris/mtmv/MTMVPropertyUtil.java | 138 +++++++++++
.../org/apache/doris/mtmv/MTMVRelatedTableIf.java | 31 ++-
.../org/apache/doris/mtmv/MTMVRelationManager.java | 10 +-
.../main/java/org/apache/doris/mtmv/MTMVUtil.java | 127 +++++++++-
.../glue/translator/PhysicalPlanTranslator.java | 14 +-
.../java/org/apache/doris/nereids/memo/Group.java | 7 +-
.../apache/doris/nereids/memo/StructInfoMap.java | 123 ++++++++++
.../doris/nereids/parser/LogicalPlanBuilder.java | 6 +-
.../processor/post/RuntimeFilterGenerator.java | 34 ++-
.../rules/analysis/CollectJoinConstraint.java | 2 +-
.../rules/exploration/join/InnerJoinLAsscom.java | 7 +-
.../exploration/join/InnerJoinLAsscomProject.java | 1 -
.../exploration/join/InnerJoinLeftAssociate.java | 1 -
.../join/InnerJoinLeftAssociateProject.java | 1 -
.../exploration/join/InnerJoinRightAssociate.java | 1 -
.../join/InnerJoinRightAssociateProject.java | 1 -
.../rules/exploration/join/JoinCommute.java | 7 +-
.../rules/exploration/join/JoinExchange.java | 1 -
.../exploration/join/JoinExchangeBothProject.java | 1 -
.../exploration/join/JoinExchangeLeftProject.java | 1 -
.../exploration/join/JoinExchangeRightProject.java | 1 -
.../join/LogicalJoinSemiJoinTranspose.java | 8 +-
.../join/LogicalJoinSemiJoinTransposeProject.java | 7 +-
.../rules/exploration/join/OuterJoinAssoc.java | 1 -
.../exploration/join/OuterJoinAssocProject.java | 1 -
.../rules/exploration/join/OuterJoinLAsscom.java | 1 -
.../exploration/join/OuterJoinLAsscomProject.java | 1 -
.../join/PushDownProjectThroughInnerOuterJoin.java | 4 +-
.../join/SemiJoinSemiJoinTranspose.java | 1 -
.../join/SemiJoinSemiJoinTransposeProject.java | 11 +-
.../mv/InitMaterializationContextHook.java | 7 +-
.../rules/expression/ExpressionOptimization.java | 4 +-
.../expression/rules/NullSafeEqualToEqual.java | 62 +++++
.../rules/implementation/AggregateStrategies.java | 33 +--
.../rules/rewrite/FindHashConditionForJoin.java | 3 +-
.../rules/rewrite/PruneOlapScanPartition.java | 15 +-
.../rules/rewrite/PushFilterInsideJoin.java | 11 +-
.../nereids/rules/rewrite/SemiJoinCommute.java | 1 -
.../rewrite/TransposeSemiJoinLogicalJoin.java | 1 -
.../TransposeSemiJoinLogicalJoinProject.java | 10 +-
.../functions/agg/AggregateFunction.java | 3 +
.../expressions/functions/agg/GroupConcat.java | 9 +
.../expressions/functions/scalar/JsonObject.java | 3 +
.../{ActiveQueries.java => ActiveBeTasks.java} | 17 +-
.../expressions/functions/table/ActiveQueries.java | 2 +-
.../trees/expressions/literal/DateLiteral.java | 46 ++--
.../visitor/TableValuedFunctionVisitor.java | 5 +
.../nereids/trees/plans/algebra/Aggregate.java | 2 +-
.../plans/commands/UpdateMvByPartitionCommand.java | 19 +-
.../plans/commands/info/AlterMTMVPropertyInfo.java | 39 +--
.../trees/plans/commands/info/CreateMTMVInfo.java | 49 +---
.../nereids/trees/plans/logical/LogicalJoin.java | 14 +-
.../trees/plans/physical/AbstractPhysicalJoin.java | 8 +-
.../trees/plans/physical/AbstractPhysicalPlan.java | 10 +
.../plans/physical/PhysicalNestedLoopJoin.java | 7 +-
.../org/apache/doris/nereids/util/DateUtils.java | 8 +-
.../org/apache/doris/nereids/util/JoinUtils.java | 10 +
.../org/apache/doris/planner/HashJoinNode.java | 25 +-
.../org/apache/doris/planner/RuntimeFilter.java | 9 +
.../main/java/org/apache/doris/qe/Coordinator.java | 77 +-----
.../java/org/apache/doris/qe/PointQueryExec.java | 19 +-
.../java/org/apache/doris/qe/ResultReceiver.java | 22 +-
.../java/org/apache/doris/qe/SessionVariable.java | 14 ++
.../java/org/apache/doris/qe/StmtExecutor.java | 3 +-
.../org/apache/doris/rpc/BackendServiceProxy.java | 6 +-
.../apache/doris/rpc/TCustomProtocolFactory.java | 41 ++++
.../apache/doris/service/FrontendServiceImpl.java | 22 +-
....java => ActiveBeTasksTableValuedFunction.java} | 50 +---
.../ActiveQueriesTableValuedFunction.java | 4 +-
.../doris/tablefunction/TableValuedFunctionIf.java | 2 +
.../java/org/apache/doris/mtmv/MTMVUtilTest.java | 113 +++++++++
.../doris/nereids/memo/StructInfoMapTest.java | 81 ++++++
.../nereids/rules/expression/FoldConstantTest.java | 2 +-
.../expression/rules/NullSafeEqualToEqualTest.java | 69 ++++++
.../rules/rewrite/EliminateJoinByUniqueTest.java | 14 --
.../expressions/literal/DateTimeLiteralTest.java | 7 +
.../commands/UpdateMvByPartitionCommandTest.java | 21 ++
.../org/apache/doris/nereids/util/PlanChecker.java | 2 +
.../org/apache/doris/rewrite/FEFunctionsTest.java | 2 +-
.../apache/doris/utframe/TestWithFeService.java | 28 +++
gensrc/thrift/Types.thrift | 3 +-
.../data/correctness_p0/test_equal_for_null.out | 2 +
.../jdbc/test_db2_jdbc_catalog.out | 20 ++
.../mtmv_p0/test_hive_limit_partition_mtmv.out | 11 +
.../data/mtmv_p0/test_limit_partition_mtmv.out | 17 ++
.../routine_load_mapping/routine_load_mapping.out | 9 +
.../data/mv_p0/routine_load_mapping/test | 1 +
.../test_mv_partition/test_mv_partition.out} | 3 +-
.../data/nereids_p0/hint/test_distribute.out | 2 +-
.../data/nereids_p0/hint/test_leading.out | 146 +----------
.../datetime_functions/test_date_function.out | 6 +
.../string_functions/test_string_function.out | 24 ++
.../group_concat.out} | 7 +-
.../shape/query10.out | 62 ++---
.../shape/query35.out | 72 +++---
.../shape/query45.out | 48 ++--
.../noStatsRfPrune/query10.out | 40 +--
.../noStatsRfPrune/query35.out | 42 ++--
.../noStatsRfPrune/query45.out | 45 ++--
.../no_stats_shape/query10.out | 40 +--
.../no_stats_shape/query35.out | 42 ++--
.../no_stats_shape/query45.out | 45 ++--
.../rf_prune/query10.out | 54 ++--
.../rf_prune/query35.out | 70 +++---
.../rf_prune/query45.out | 48 ++--
.../nereids_tpcds_shape_sf100_p0/shape/query10.out | 54 ++--
.../nereids_tpcds_shape_sf100_p0/shape/query35.out | 72 +++---
.../nereids_tpcds_shape_sf100_p0/shape/query45.out | 48 ++--
.../datetime_functions/test_date_function.out | 6 +
.../test_unique_model_schema_value_change.out | 80 ++++++
.../correctness_p0/test_equal_for_null.groovy | 3 +-
.../jdbc/test_db2_jdbc_catalog.groovy | 24 ++
.../mtmv_p0/test_hive_limit_partition_mtmv.groovy | 162 ++++++++++++
.../mtmv_p0/test_limit_partition_mtmv.groovy | 240 ++++++++++++++++++
.../routine_load_mapping.groovy | 52 ++++
.../test_mv_partition/test_mv_partition.groovy | 64 +++++
.../suites/nereids_p0/hint/test_distribute.groovy | 2 +-
.../suites/nereids_p0/hint/test_leading.groovy | 14 +-
.../datetime_functions/test_date_function.groovy | 2 +
.../string_functions/test_string_function.groovy | 8 +
.../suites/nereids_syntax_p0/group_concat.groovy | 45 +++-
.../datetime_functions/test_date_function.groovy | 2 +
.../test_unique_model_schema_value_change.groovy | 155 ++++++++++++
.../max_msg_size_of_result_receiver.groovy | 61 +++++
180 files changed, 3516 insertions(+), 1103 deletions(-)
create mode 100644 docs/en/docs/ecosystem/bi-integration/apache-superset.md
create mode 100644 docs/zh-CN/docs/ecosystem/bi-integration/apache-superset.md
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/CloudTabletStatMgr.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionSyncConfig.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionSyncTimeUnit.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPropertyUtil.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/StructInfoMap.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/NullSafeEqualToEqual.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/table/{ActiveQueries.java
=> ActiveBeTasks.java} (77%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/rpc/TCustomProtocolFactory.java
copy
fe/fe-core/src/main/java/org/apache/doris/tablefunction/{ActiveQueriesTableValuedFunction.java
=> ActiveBeTasksTableValuedFunction.java} (53%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mtmv/MTMVUtilTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/memo/StructInfoMapTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/NullSafeEqualToEqualTest.java
create mode 100644
regression-test/data/mtmv_p0/test_hive_limit_partition_mtmv.out
create mode 100644 regression-test/data/mtmv_p0/test_limit_partition_mtmv.out
create mode 100644
regression-test/data/mv_p0/routine_load_mapping/routine_load_mapping.out
create mode 100644 regression-test/data/mv_p0/routine_load_mapping/test
copy regression-test/data/{correctness_p0/test_equal_for_null.out =>
mv_p0/test_mv_partition/test_mv_partition.out} (76%)
copy regression-test/data/{correctness_p0/test_equal_for_null.out =>
nereids_syntax_p0/group_concat.out} (67%)
create mode 100644
regression-test/suites/mtmv_p0/test_hive_limit_partition_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_limit_partition_mtmv.groovy
create mode 100644
regression-test/suites/mv_p0/routine_load_mapping/routine_load_mapping.groovy
create mode 100644
regression-test/suites/mv_p0/test_mv_partition/test_mv_partition.groovy
create mode 100644
regression-test/suites/variable_p0/max_msg_size_of_result_receiver.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
