This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a change to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 9458cd8c782 Support show auto analyze pending jobs. (#31926)
omit 14f606fbc97 Support follower sync query columns to master. (#31859)
omit a2c22c63e63 Support column level health value. (#31794)
omit 2a0b3add07d High priority queue and map. (#31509)
omit 0f97217b72b Collect high priority columns. (#31235)
add a1fa1daeeaa [fix](jdbc catalog) Change Druid Pool dependency to
version 1.2.5 (#31783)
add d979a922af4 [chore](ci) add back required to test (#31795)
add 092757aec12 [test](leading) add leading tpc-ds regression test cases
(#31681)
add d5b390ac7e3 [chore](ci) add back pipelinex in master (#31799)
add e9980e221de [chore](ci) remove required of pipelinex (#31806)
add 3ba048569e5 [pipelineX](debug) Refactor code and complete debug string
(#31733)
add 0cfef27b8d5 [fix](mtmv) Avoiding the occurrence of null pointers in
logs due to the deletion of MTMV (#31722)
add 2d0e616faee [fix](cloud-merge) Fix missing the whereColumn field
(#31786)
add 5c448fe0553 [fix](cloud) enable all p0 cases in cloud p0 (#31693)
add db487ca4ef5 [feature](merge-cloud) compaction for mow table (#31774)
add 2dd271c9703 [chore](fe) Speed up fe-core checkstyle (#31785)
add 49ca2650121 [fix](group commit) handle group commit create plan error
(#31757)
add 79d796e63ed [fix](cast) fix wrong result while cast string to float
(#31781)
add abb075da412 [enhancement](test) Test the unique model by modify a key
type from TINYINT to other type (#31713)
add 3f33425e4c2 [fix](move-memtable) free resources when fail to open
stream (#31762)
add 10950e464c3 [fix](Nereids) let OrToIn rewritten result have stable
order (#31731)
add 739963bcb5c [cloud](point query) support short circuit query in cloud
mode (#31650)
add f701c89a628 [fix](cloud) Fix regression case `test_dynamic_partition`
run in cloud mode (#31803)
add d916f9cbd39 [fix](update) Update set value should consider sequence
column (#31626)
add 0c2514377ae [test](Nereids) add grouping sets test (#31675)
add e7e1fd89fed [improvement](jdbc catalog) support sqlserver timestamp
type read (#31805)
add b8f13c1340e [refactor](jdbc catalog) split postgresql jdbc Executor
(#31730)
add 214eea0317c [fix](cloud) Fix `version_p0` run in cloud mode (#31810)
add d2688ae6021 [feature](merge-cloud) schema change for mow table (#31819)
add fcec33fed85 [improvement](alter table) cann't rewrite force property
(#31820)
add c3f8c7ad37b [fix](pipelineX) fix error distribution in
DistinctStreamingAggOperatorX (#31804)
add 69c32385e0f [fix](planner) fix ifnull and nvl function with one
parameters exception message anbugious (#31808)
add 3c3973433e7 [feature](function) Support for aggregate function
foreach combiner (#31526)
add faac84a85bd [fix](move-memtable) use tuple desc from schema param for
delta writer (#31747)
add 53850274eef [docs](jdbc catalog) add sqlserver timestamp type mapping
doc (#31832)
add 43d81263457 [fix](build)Fix index_tool build error (#31834)
add 26553fc899b [fix](partition) add log when tablet partition id eq 0
(#31796)
add 510c548d71f [fix](Hudi-Utils) Change the Pid obtaining method #31835
add cf49db726c3 [Improvement](profile)Add tvf active_be_tasks() #31815
add 40f159fb81c [fix](group commit) group commit may heap-use-after-free
if execute plan failed (#31839)
add 73d1fdc22d8 [docs](flink-connector) Fix Flink Doris Connector typo
(#31830)
add 799fec26dcb [opt](session variable) max_msg_size_of_result_receiver
#31809
add 60eb319eab6 [chore](ci) remove pipelinex_p0 (#31755)
add dbccbe0fbba [feat](Nereids): add struct info map in group (#31800)
add 2f3d5c5e135 [fix](cloud) Fix FE cannot work when restarting after
schema change (#31842)
add bd9a74ac9de [thirdparty](lib) Add base64 thirdparty lib (#31850)
add 1c2cefa3286 [Feature](TrinoConnector-Catalog) Fe supports
`trino-connector` catalog (#31768)
add 290897aeecb [Bug](top-n) init query_ctx runtime predicate before
operators prepare (#31871)
add fec72c44118 [fix](Nereids) json_object can't be odd parameters, need
even parameters (#31767)
add 3f997cedc94 [fix](cloud) Fix wrong error msg when not in cloud mode
(#31814)
add 1bede293e35 [Bug](load) fix wrong data in mv when routine load with
function mapping (#31787)
add 694a25313d6 [fix](multi-catalog) fix iceberg defalut credentials
proviers (#31704)
add 71ec81198a2 [Fix](Nereids) fix missing comment when creating table
(#31869)
add bc5bb1ed5a5 [Bug](partition) fix npe when prune partition with not
exist partition column in mv #31860
add 88c41e34513 [regression test] Test the unique model by modify a value
type from TINYINT to other type (#31682)
add 6d237335cd8 [deps](docker) update docker build env to add jdk17
(#31644)
add c5fcdabb26f [bugfix](coordinator) should use fragment id not profile
fragment id to cancel fragment (#31852)
add 7ed02637dd8 [fix](Nereids) fix group_concat(distinct) failed (#31873)
add dbb757382ad [enhancement](Nereids): support more condition
Date/DateTime Literal (#31858)
add d5514fa07e6 [enhancement](cloud) add CloudTabletStatMgr to capture
stats in cloud mode (#31818)
add 59c3d8b9ae6 [fix](regression test) fix alter force property in cloud
mode (#31874)
add 3864fa5281f [improvement](jdbc catalog) opt get db2 schema list & xml
type mapping (#31856)
add bdbad5bf7b0 [typo](doc) Add a full backup case and modify sidebars
(#31801)
add 09efa2611f0 [Bug](fix) fix or and "<=>" cause coredump in query
(#31884)
add 6415a11245f [type](doc) Standardize the name in the sql-dialect
document and modify the format (#31636)
add 09f2643ed9f [enhancement](nereids) allow reorder mark join (#30644)
add d73b50fb10d [enhancement](nereids) show nullaware semi join in plan
(#31738)
add c35a415fbdc [opt](invert index) add level to time series compaction
policy (#31488)
add 13178b2d3f8 [fix](http stream) http stream support
memtable_on_sink_node header (#31866)
add 21b70883c83 [fix](mtmv) support null value in partition for updating
(#31843)
add b4ce532033d [fix](money_format) fix money_format #31883
add e18514ab31b [enhancement](cloud) update rows and versions after load
(#31750)
add 68d771319e8 [fix](function) fix date_format function execution error
on fe (#31645)
add 6f12e9f559d [doc](apache-superset)add doc for apache superset
compatible (#31314)
add a5f23de7f92 [bug](meta) exit when get RollbackException in observer
(#31687)
add 2fa8768a3f2 [cloud](point query) enable short circuit query in cloud
(#31897)
add 33dd8221ae3 [enhance](mtmv) MTMV Use partial partition of base table
(#31632)
add 39aeb95181d (cloud-merge) Fix the serialization of CloudTablet (#31864)
add 9edc370a5e4 [chore](build) Using multithread to accelerate FE
compilation (#31855)
add fa2daee176c [Fix](Nereids) fix hint cases with random result (#31865)
add fc7cb3305ea [feat](nereids) support null safe eq runtime filter (FE
part) (#31655)
new 46fd6a6d8c5 Collect high priority columns. (#31235)
new d8bcda0c26e High priority queue and map. (#31509)
new 41ea3618925 Support column level health value. (#31794)
new 12d657241d2 Support follower sync query columns to master. (#31859)
new 87f3b0745fb Support show auto analyze pending jobs. (#31926)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9458cd8c782)
\
N -- N -- N refs/heads/high-priority-column (87f3b0745fb)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/comment-to-trigger-teamcity.yml | 21 +-
be/src/agent/task_worker_pool.cpp | 23 +-
be/src/cloud/cloud_base_compaction.cpp | 13 +-
be/src/cloud/cloud_cumulative_compaction.cpp | 13 +-
be/src/cloud/cloud_meta_mgr.cpp | 72 +-
be/src/cloud/cloud_schema_change_job.cpp | 80 +-
be/src/cloud/cloud_schema_change_job.h | 3 +
be/src/cloud/cloud_tablet.cpp | 55 ++
be/src/cloud/cloud_tablet.h | 7 +
be/src/cloud/pb_convert.cpp | 4 +
be/src/http/action/http_stream.cpp | 4 +
be/src/index-tools/index_tool.cpp | 2 +-
be/src/olap/base_tablet.cpp | 160 ++++
be/src/olap/base_tablet.h | 17 +
be/src/olap/cumulative_compaction.cpp | 3 +
be/src/olap/cumulative_compaction_policy.h | 8 +
.../cumulative_compaction_time_series_policy.cpp | 108 ++-
.../cumulative_compaction_time_series_policy.h | 4 +
be/src/olap/data_dir.cpp | 4 +-
be/src/olap/full_compaction.cpp | 3 +
be/src/olap/rowset/rowset_meta.h | 6 +
be/src/olap/tablet.cpp | 160 ----
be/src/olap/tablet.h | 15 -
be/src/olap/tablet_meta.cpp | 17 +-
be/src/olap/tablet_meta.h | 10 +-
be/src/olap/wal/wal_table.cpp | 38 +-
be/src/olap/wal/wal_table.h | 2 +-
.../exec/distinct_streaming_aggregation_operator.h | 2 +-
be/src/pipeline/exec/scan_operator.cpp | 2 +-
be/src/pipeline/pipeline_x/dependency.cpp | 5 +-
.../pipeline_x/pipeline_x_fragment_context.cpp | 24 +-
be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 7 +-
be/src/runtime/exec_env.h | 3 +
be/src/runtime/exec_env_init.cpp | 6 +
be/src/runtime/group_commit_mgr.cpp | 42 +-
be/src/runtime/group_commit_mgr.h | 5 +-
be/src/runtime/runtime_query_statistics_mgr.cpp | 50 ++
be/src/runtime/runtime_query_statistics_mgr.h | 5 +
be/src/service/internal_service.cpp | 13 +-
be/src/service/internal_service.h | 16 +-
be/src/service/point_query_executor.cpp | 17 +-
be/src/service/point_query_executor.h | 4 +-
.../vec/aggregate_functions/aggregate_function.h | 2 +-
.../aggregate_function_approx_count_distinct.h | 2 +-
.../aggregate_functions/aggregate_function_avg.h | 2 +-
.../aggregate_function_avg_weighted.h | 2 +-
.../aggregate_function_binary.h | 2 +-
.../aggregate_functions/aggregate_function_bit.h | 2 +-
.../aggregate_function_bitmap.h | 4 +-
.../aggregate_function_bitmap_agg.h | 2 +-
.../aggregate_function_collect.h | 2 +-
.../aggregate_functions/aggregate_function_count.h | 4 +-
.../aggregate_function_count_by_enum.h | 2 +-
.../aggregate_functions/aggregate_function_covar.h | 2 +-
.../aggregate_function_distinct.h | 2 +-
.../aggregate_function_foreach.cpp | 64 ++
.../aggregate_function_foreach.h | 264 +++++++
.../aggregate_function_group_concat.h | 2 +-
.../aggregate_function_histogram.h | 2 +-
.../aggregate_function_hll_union_agg.h | 2 +-
.../aggregate_function_java_udaf.h | 2 +-
.../aggregate_functions/aggregate_function_map.h | 2 +-
.../aggregate_function_min_max.h | 2 +-
.../aggregate_function_min_max_by.h | 2 +-
.../aggregate_functions/aggregate_function_null.h | 4 +-
.../aggregate_function_orthogonal_bitmap.h | 2 +-
.../aggregate_function_percentile_approx.h | 10 +-
.../aggregate_function_product.h | 2 +-
.../aggregate_function_quantile_state.h | 2 +-
.../aggregate_function_reader_first_last.h | 2 +-
.../aggregate_function_retention.h | 2 +-
.../aggregate_functions/aggregate_function_rpc.h | 2 +-
.../aggregate_function_sequence_match.h | 2 +-
.../aggregate_function_simple_factory.cpp | 3 +
.../aggregate_function_simple_factory.h | 26 +-
.../aggregate_functions/aggregate_function_sort.h | 2 +-
.../aggregate_function_state_union.h | 2 +-
.../aggregate_function_stddev.h | 2 +-
.../aggregate_functions/aggregate_function_sum.h | 2 +-
.../aggregate_functions/aggregate_function_topn.h | 2 +-
.../aggregate_functions/aggregate_function_uniq.h | 2 +-
.../aggregate_function_window.h | 14 +-
.../aggregate_function_window_funnel.h | 2 +-
.../data_types/serde/data_type_nullable_serde.cpp | 2 +-
be/src/vec/exec/scan/vmeta_scanner.cpp | 22 +-
be/src/vec/exec/scan/vmeta_scanner.h | 2 +
be/src/vec/exec/scan/vscan_node.cpp | 2 +-
be/src/vec/functions/function.cpp | 2 +-
be/src/vec/functions/function_string.h | 30 +-
be/src/vec/runtime/vdata_stream_recvr.h | 14 +-
be/src/vec/sink/load_stream_stub.cpp | 4 +-
be/src/vec/sink/writer/vtablet_writer_v2.cpp | 2 +-
be/test/olap/wal/wal_manager_test.cpp | 2 -
build.sh | 12 +-
cloud/src/meta-service/meta_service.h | 4 +-
cloud/src/meta-service/meta_service_txn.cpp | 61 +-
cloud/test/meta_service_test.cpp | 6 +
docker/compilation/Dockerfile | 21 +-
.../sqlserver/init/03-create-table.sql | 4 +
.../docker-compose/sqlserver/init/04-insert.sql | 2 +
.../ecosystem/bi-integration/apache-superset.md | 53 ++
.../install/source-install/compilation-general.md | 33 +-
docs/en/docs/lakehouse/multi-catalog/jdbc.md | 40 +-
docs/en/docs/lakehouse/sql-dialect.md | 44 +-
.../Backup-and-Restore/BACKUP.md | 7 +
.../Create/CREATE-TABLE.md | 9 +
docs/sidebars.json | 2 +-
.../ecosystem/bi-integration/apache-superset.md | 57 ++
docs/zh-CN/docs/ecosystem/flink-doris-connector.md | 2 +-
.../install/source-install/compilation-general.md | 12 +-
docs/zh-CN/docs/lakehouse/multi-catalog/jdbc.md | 6 +-
docs/zh-CN/docs/lakehouse/sql-dialect.md | 46 +-
.../Backup-and-Restore/BACKUP.md | 7 +
.../Create/CREATE-TABLE.md | 8 +
.../src/main/java/org/apache/doris/hudi/Utils.java | 12 +-
.../org/apache/doris/jdbc/JdbcExecutorFactory.java | 2 +
...bcExecutor.java => PostgreSQLJdbcExecutor.java} | 73 +-
.../apache/doris/jdbc/SQLServerJdbcExecutor.java | 14 +
fe/check/checkstyle/checkstyle.xml | 3 -
fe/fe-common/pom.xml | 4 +
.../main/java/org/apache/doris/common/Config.java | 7 +
.../doris/trinoconnector/TrinoColumnMetadata.java | 141 ++++
.../TrinoConnectorPluginManager.java | 147 ++++
.../TrinoConnectorServicesProvider.java | 138 ++++
fe/fe-core/pom.xml | 31 +-
.../main/java/org/apache/doris/alter/Alter.java | 28 +-
.../java/org/apache/doris/alter/RollupJobV2.java | 15 +-
.../apache/doris/alter/SchemaChangeHandler.java | 8 +
.../org/apache/doris/alter/SchemaChangeJobV2.java | 1 +
.../org/apache/doris/analysis/DateLiteral.java | 8 +-
.../main/java/org/apache/doris/analysis/Expr.java | 1 +
.../apache/doris/analysis/FunctionCallExpr.java | 2 +
.../analysis/ModifyTablePropertiesClause.java | 17 +
.../java/org/apache/doris/analysis/UpdateStmt.java | 14 +-
.../java/org/apache/doris/backup/RestoreJob.java | 1 +
.../doris/catalog/BuiltinTableValuedFunctions.java | 4 +-
.../apache/doris/catalog/CloudTabletStatMgr.java | 273 +++++++
.../java/org/apache/doris/catalog/Database.java | 5 +
.../main/java/org/apache/doris/catalog/Env.java | 71 +-
.../java/org/apache/doris/catalog/Function.java | 14 +
.../org/apache/doris/catalog/FunctionRegistry.java | 10 +-
.../doris/catalog/InternalSchemaInitializer.java | 5 +-
.../apache/doris/catalog/ListPartitionItem.java | 21 +
.../main/java/org/apache/doris/catalog/MTMV.java | 11 +-
.../java/org/apache/doris/catalog/OlapTable.java | 14 +
.../org/apache/doris/catalog/PartitionItem.java | 14 +
.../apache/doris/catalog/RangePartitionItem.java | 18 +
.../java/org/apache/doris/catalog/Replica.java | 17 +
.../java/org/apache/doris/catalog/TableIf.java | 2 +-
.../org/apache/doris/catalog/TableProperty.java | 18 +-
.../apache/doris/cloud/catalog/CloudPartition.java | 22 +-
.../cloud/common/util/CloudPropertyAnalyzer.java | 2 +
.../transaction/CloudGlobalTransactionMgr.java | 59 ++
.../apache/doris/common/util/PropertyAnalyzer.java | 39 +
.../apache/doris/datasource/CatalogFactory.java | 4 +
.../apache/doris/datasource/ExternalCatalog.java | 3 +
.../apache/doris/datasource/InitCatalogLog.java | 1 +
.../apache/doris/datasource/InitDatabaseLog.java | 1 +
.../apache/doris/datasource/InternalCatalog.java | 22 +-
.../apache/doris/datasource/TableFormatType.java | 3 +-
.../datasource/iceberg/IcebergExternalCatalog.java | 9 +
.../iceberg/IcebergGlueExternalCatalog.java | 5 +-
.../iceberg/IcebergHMSExternalCatalog.java | 1 +
.../iceberg/IcebergHadoopExternalCatalog.java | 1 +
.../iceberg/IcebergRestExternalCatalog.java | 3 +-
.../datasource/jdbc/client/JdbcDB2Client.java | 36 +
.../datasource/jdbc/client/JdbcOracleClient.java | 6 +-
.../jdbc/client/JdbcSQLServerClient.java | 1 +
.../datasource/property/PropertyConverter.java | 16 +-
.../property/constants/S3Properties.java | 8 +
.../constants/TrinoConnectorProperties.java | 44 ++
.../TrinoConnectorExternalCatalog.java | 336 +++++++++
.../TrinoConnectorExternalCatalogFactory.java} | 22 +-
.../TrinoConnectorExternalDatabase.java} | 23 +-
.../TrinoConnectorExternalTable.java | 239 ++++++
.../source/TrinoConnectorScanNode.java | 304 ++++++++
.../source/TrinoConnectorSource.java | 91 +++
.../trinoconnector/source/TrinoConnectorSplit.java | 52 ++
.../apache/doris/job/extensions/mtmv/MTMVTask.java | 10 +-
.../apache/doris/journal/bdbje/BDBJEJournal.java | 7 +
.../src/main/java/org/apache/doris/load/Load.java | 10 +-
.../org/apache/doris/master/ReportHandler.java | 1 +
.../apache/doris/mtmv/MTMVPartitionSyncConfig.java | 57 ++
.../MTMVPartitionSyncTimeUnit.java} | 29 +-
.../org/apache/doris/mtmv/MTMVPartitionUtil.java | 11 +-
.../org/apache/doris/mtmv/MTMVPropertyUtil.java | 138 ++++
.../org/apache/doris/mtmv/MTMVRelatedTableIf.java | 31 +-
.../org/apache/doris/mtmv/MTMVRelationManager.java | 10 +-
.../main/java/org/apache/doris/mtmv/MTMVUtil.java | 127 +++-
.../glue/translator/ExpressionTranslator.java | 11 +
.../glue/translator/PhysicalPlanTranslator.java | 14 +-
.../java/org/apache/doris/nereids/memo/Group.java | 7 +-
.../apache/doris/nereids/memo/StructInfoMap.java | 123 ++++
.../doris/nereids/parser/LogicalPlanBuilder.java | 6 +-
.../processor/post/RuntimeFilterGenerator.java | 34 +-
.../rules/analysis/CollectJoinConstraint.java | 4 +-
.../rules/exploration/join/InnerJoinLAsscom.java | 7 +-
.../exploration/join/InnerJoinLAsscomProject.java | 1 -
.../exploration/join/InnerJoinLeftAssociate.java | 1 -
.../join/InnerJoinLeftAssociateProject.java | 1 -
.../exploration/join/InnerJoinRightAssociate.java | 1 -
.../join/InnerJoinRightAssociateProject.java | 1 -
.../rules/exploration/join/JoinCommute.java | 7 +-
.../rules/exploration/join/JoinExchange.java | 1 -
.../exploration/join/JoinExchangeBothProject.java | 1 -
.../exploration/join/JoinExchangeLeftProject.java | 1 -
.../exploration/join/JoinExchangeRightProject.java | 1 -
.../join/LogicalJoinSemiJoinTranspose.java | 8 +-
.../join/LogicalJoinSemiJoinTransposeProject.java | 7 +-
.../rules/exploration/join/OuterJoinAssoc.java | 1 -
.../exploration/join/OuterJoinAssocProject.java | 1 -
.../rules/exploration/join/OuterJoinLAsscom.java | 1 -
.../exploration/join/OuterJoinLAsscomProject.java | 1 -
.../join/PushDownProjectThroughInnerOuterJoin.java | 4 +-
.../join/SemiJoinSemiJoinTranspose.java | 1 -
.../join/SemiJoinSemiJoinTransposeProject.java | 11 +-
.../mv/InitMaterializationContextHook.java | 7 +-
.../rules/expression/ExpressionOptimization.java | 4 +-
.../expression/rules/NullSafeEqualToEqual.java | 62 ++
.../nereids/rules/expression/rules/OrToIn.java | 13 +-
.../rules/expression/rules/SimplifyRange.java | 2 +-
.../rules/implementation/AggregateStrategies.java | 33 +-
.../rules/rewrite/FindHashConditionForJoin.java | 3 +-
.../rules/rewrite/PruneOlapScanPartition.java | 15 +-
.../rules/rewrite/PushFilterInsideJoin.java | 11 +-
.../nereids/rules/rewrite/SemiJoinCommute.java | 1 -
.../rewrite/TransposeSemiJoinLogicalJoin.java | 1 -
.../TransposeSemiJoinLogicalJoinProject.java | 10 +-
...uilder.java => AggCombinerFunctionBuilder.java} | 34 +-
.../functions/agg/AggregateFunction.java | 3 +
.../expressions/functions/agg/GroupConcat.java | 9 +
...UnionCombinator.java => ForEachCombinator.java} | 38 +-
.../functions/combinator/MergeCombinator.java | 4 +-
.../functions/combinator/StateCombinator.java | 4 +-
.../functions/combinator/UnionCombinator.java | 4 +-
.../expressions/functions/scalar/JsonObject.java | 3 +
.../{ActiveQueries.java => ActiveBeTasks.java} | 17 +-
.../expressions/functions/table/ActiveQueries.java | 2 +-
.../trees/expressions/literal/DateLiteral.java | 46 +-
.../trees/expressions/literal/FloatLiteral.java | 2 +-
.../visitor/AggregateFunctionVisitor.java | 5 +
.../visitor/TableValuedFunctionVisitor.java | 5 +
.../nereids/trees/plans/algebra/Aggregate.java | 2 +-
.../doris/nereids/trees/plans/algebra/Filter.java | 4 +-
.../trees/plans/commands/UpdateCommand.java | 13 +-
.../plans/commands/UpdateMvByPartitionCommand.java | 19 +-
.../plans/commands/info/AlterMTMVPropertyInfo.java | 39 +-
.../trees/plans/commands/info/CreateMTMVInfo.java | 49 +-
.../nereids/trees/plans/logical/LogicalJoin.java | 14 +-
.../trees/plans/physical/AbstractPhysicalJoin.java | 8 +-
.../trees/plans/physical/AbstractPhysicalPlan.java | 10 +
.../plans/physical/PhysicalNestedLoopJoin.java | 7 +-
.../org/apache/doris/nereids/util/DateUtils.java | 8 +-
.../org/apache/doris/nereids/util/JoinUtils.java | 10 +
.../org/apache/doris/persist/gson/GsonUtils.java | 21 +-
.../org/apache/doris/planner/HashJoinNode.java | 25 +-
.../org/apache/doris/planner/OlapScanNode.java | 2 +-
.../org/apache/doris/planner/OlapTableSink.java | 3 +-
.../org/apache/doris/planner/RuntimeFilter.java | 9 +
.../apache/doris/planner/SingleNodePlanner.java | 2 +-
.../main/java/org/apache/doris/qe/Coordinator.java | 77 +-
.../java/org/apache/doris/qe/PointQueryExec.java | 53 +-
.../java/org/apache/doris/qe/ResultReceiver.java | 22 +-
.../java/org/apache/doris/qe/SessionVariable.java | 19 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 3 +-
.../org/apache/doris/rpc/BackendServiceProxy.java | 6 +-
.../TCustomProtocolFactory.java} | 30 +-
.../apache/doris/service/FrontendServiceImpl.java | 61 +-
.../apache/doris/statistics/StatisticalType.java | 3 +-
....java => ActiveBeTasksTableValuedFunction.java} | 51 +-
.../ActiveQueriesTableValuedFunction.java | 4 +-
.../doris/tablefunction/TableValuedFunctionIf.java | 2 +
.../org/apache/doris/task/CreateReplicaTask.java | 5 +
.../doris/task/UpdateTabletMetaInfoTask.java | 5 +
.../doris/transaction/GlobalTransactionMgr.java | 5 +
.../transaction/GlobalTransactionMgrIface.java | 3 +
.../java/org/apache/doris/mtmv/MTMVUtilTest.java | 113 +++
.../doris/nereids/memo/StructInfoMapTest.java | 81 +++
.../nereids/rules/expression/FoldConstantTest.java | 2 +-
.../expression/rules/NullSafeEqualToEqualTest.java | 69 ++
.../rules/rewrite/EliminateJoinByUniqueTest.java | 14 -
.../doris/nereids/rules/rewrite/OrToInTest.java | 10 +
.../expressions/literal/DateTimeLiteralTest.java | 7 +
.../commands/UpdateMvByPartitionCommandTest.java | 21 +
.../org/apache/doris/nereids/util/PlanChecker.java | 2 +
.../org/apache/doris/rewrite/FEFunctionsTest.java | 2 +-
.../java/org/apache/doris/task/AgentTaskTest.java | 2 +-
.../apache/doris/utframe/TestWithFeService.java | 28 +
fe/pom.xml | 62 +-
gensrc/proto/cloud.proto | 12 +-
gensrc/proto/olap_file.proto | 3 +
gensrc/thrift/AgentService.thrift | 2 +
gensrc/thrift/Descriptors.thrift | 7 +
gensrc/thrift/FrontendService.thrift | 12 +-
gensrc/thrift/PlanNodes.thrift | 15 +
gensrc/thrift/Types.thrift | 4 +-
.../test_distinct_streaming_agg_local_shuffle.out | 25 +
.../data/correctness_p0/test_equal_for_null.out | 2 +
.../unique/test_unique_table_new_sequence.out | 20 +-
.../jdbc/test_db2_jdbc_catalog.out | 20 +
.../jdbc/test_sqlserver_jdbc_catalog.out | 7 +
.../data/function_p0/test_agg_foreach.out | 28 +
.../mtmv_p0/test_hive_limit_partition_mtmv.out | 11 +
.../data/mtmv_p0/test_limit_partition_mtmv.out | 17 +
.../routine_load_mapping/routine_load_mapping.out | 9 +
.../data/mv_p0/routine_load_mapping/test | 1 +
.../test_mv_partition/test_mv_partition.out} | 3 +-
.../data/nereids_hint_tpcds_p0/shape/query1.out | 47 ++
.../data/nereids_hint_tpcds_p0/shape/query24.out | 65 ++
.../data/nereids_hint_tpcds_p0/shape/query64.out | 139 ++++
.../data/nereids_hint_tpcds_p0/shape/query67.out | 40 +
.../data/nereids_hint_tpcds_p0/shape/query72.out | 73 ++
.../data/nereids_hint_tpcds_p0/shape/query78.out | 65 ++
.../data/nereids_p0/hint/test_distribute.out | 2 +-
.../data/nereids_p0/hint/test_leading.out | 146 +---
.../cast_function/test_cast_function.out | 3 +
.../datetime_functions/test_date_function.out | 6 +
.../string_functions/test_string_function.out | 24 +
.../test_grouping_sets_combination.out | 184 +++++
.../group_concat.out} | 7 +-
.../shape/query10.out | 62 +-
.../shape/query35.out | 72 +-
.../shape/query45.out | 48 +-
.../noStatsRfPrune/query10.out | 40 +-
.../noStatsRfPrune/query35.out | 42 +-
.../noStatsRfPrune/query45.out | 45 +-
.../no_stats_shape/query10.out | 40 +-
.../no_stats_shape/query35.out | 42 +-
.../no_stats_shape/query45.out | 45 +-
.../rf_prune/query10.out | 54 +-
.../rf_prune/query35.out | 70 +-
.../rf_prune/query45.out | 48 +-
.../nereids_tpcds_shape_sf100_p0/shape/query10.out | 54 +-
.../nereids_tpcds_shape_sf100_p0/shape/query35.out | 72 +-
.../nereids_tpcds_shape_sf100_p0/shape/query45.out | 48 +-
.../cast_function/test_cast_function.out | 3 +
.../datetime_functions/test_date_function.out | 6 +
.../test_unique_model_schema_key_change.out | 50 ++
.../test_unique_model_schema_value_change.out | 80 ++
.../data/statistics/test_partition_first_load.csv | 2 +
.../data/statistics/test_strict_mode_1.csv | 8 +
.../data/statistics/test_strict_mode_2.csv | 8 +
regression-test/data/update/test_update.out | 49 ++
.../org/apache/doris/regression/suite/Suite.groovy | 4 +-
.../cloud_p0/conf/regression-conf-custom.groovy | 1 -
regression-test/pipeline/cloud_p0/prepare.sh | 3 +-
regression-test/pipeline/common/teamcity-utils.sh | 5 +-
.../alter_p0/test_alter_force_property.groovy | 49 ++
.../alter_p2/test_alter_colocate_group.groovy | 4 +
.../alter_p2/test_alter_table_property.groovy | 16 +-
.../modify_replica_use_partition.groovy | 40 +-
.../modify_replica_use_partition_by_hdfs.groovy | 38 +-
...est_distinct_streaming_agg_local_shuffle.groovy | 102 +++
.../correctness_p0/test_equal_for_null.groovy | 3 +-
.../jdbc/test_db2_jdbc_catalog.groovy | 24 +
.../jdbc/test_sqlserver_jdbc_catalog.groovy | 3 +
.../suites/function_p0/test_agg_foreach.groovy | 95 +++
.../operate_internal_schema.groovy | 18 +-
.../test_migrate_disk_with_publish_version.groovy | 3 +
.../suites/mtmv_p0/test_build_mtmv.groovy | 16 +-
.../mtmv_p0/test_hive_limit_partition_mtmv.groovy | 162 +++++
.../mtmv_p0/test_limit_partition_mtmv.groovy | 240 ++++++
.../routine_load_mapping.groovy | 52 ++
.../test_mv_partition/test_mv_partition.groovy | 64 ++
.../nereids_function_p0/scalar_function/N.groovy | 16 +-
.../ddl/gen_shape.py | 0
.../ddl/shape.tmpl | 0
.../suites/nereids_hint_tpcds_p0/load.groovy | 809 +++++++++++++++++++++
.../nereids_hint_tpcds_p0/shape/query1.groovy | 84 +++
.../nereids_hint_tpcds_p0/shape/query24.groovy | 143 ++++
.../nereids_hint_tpcds_p0/shape/query64.groovy | 279 +++++++
.../nereids_hint_tpcds_p0/shape/query67.groovy | 121 +++
.../nereids_hint_tpcds_p0/shape/query72.groovy | 91 +++
.../nereids_hint_tpcds_p0/shape/query78.groovy | 154 ++++
.../suites/nereids_p0/hint/test_distribute.groovy | 2 +-
.../suites/nereids_p0/hint/test_leading.groovy | 14 +-
.../cast_function/test_cast_function.groovy | 1 +
.../datetime_functions/test_date_function.groovy | 2 +
.../string_functions/test_string_function.groovy | 8 +
.../test_grouping_sets_combination.groovy | 191 +++++
.../suites/nereids_syntax_p0/group_concat.groovy | 45 +-
.../test_dynamic_partition.groovy | 12 +
.../test_multi_column_partition.groovy | 4 +-
.../suites/point_query_p0/test_point_query.groovy | 32 +-
.../test_point_query_cluster_key.groovy | 32 +-
.../cast_function/test_cast_function.groovy | 1 +
.../conditional_functions/test_ifnull.groovy | 9 +
.../conditional_functions/test_nvl.groovy | 8 +
.../datetime_functions/test_date_function.groovy | 2 +
...e_column_with_delete_drop_column_dup_key.groovy | 8 +-
...olumn_with_delete_drop_column_unique_key.groovy | 4 +-
.../test_alter_table_property_bucket.groovy | 10 +-
.../test_agg_keys_schema_change.groovy | 4 +-
.../test_agg_mv_schema_change.groovy | 4 +-
.../test_agg_rollup_schema_change.groovy | 4 +-
.../test_agg_vals_schema_change.groovy | 4 +-
.../schema_change_p0/test_alter_parition.groovy | 10 +-
.../test_alter_table_column.groovy | 4 +-
.../test_delete_schema_change.groovy | 82 +++
.../schema_change_p0/test_delete_schema_change.sql | 47 --
.../test_dup_keys_schema_change.groovy | 4 +-
.../test_dup_mv_schema_change.groovy | 4 +-
.../test_dup_rollup_schema_change.groovy | 4 +-
.../test_dup_vals_schema_change.groovy | 4 +-
.../test_enable_light_schema_change.groovy | 12 +-
.../suites/schema_change_p0/test_partition.groovy | 38 +-
.../test_partition_schema_change.groovy | 80 ++
.../test_partition_schema_change.sql | 48 --
.../schema_change_p0/test_rename_column.groovy | 4 +-
.../test_uniq_delete_sign_schema_change.groovy | 4 +-
.../test_uniq_keys_schema_change.groovy | 4 +-
.../test_uniq_mv_schema_change.groovy | 4 +-
.../test_uniq_rollup_schema_change.groovy | 6 +-
.../test_uniq_seq_col_schema_change.groovy | 4 +-
.../test_uniq_vals_schema_change.groovy | 4 +-
.../test_unique_model_schema_key_change.groovy | 154 ++++
.../test_unique_model_schema_value_change.groovy | 155 ++++
.../test_update_schema_change.groovy | 6 +-
.../analyze_stats_partition_first_load.groovy | 89 +++
...lyze_stats_triggered_by_update_row_count.groovy | 176 +++++
...triggered_by_update_row_count_streamload.groovy | 112 +++
.../test_partial_update_schema_change.groovy | 8 +-
...t_partial_update_schema_change_row_store.groovy | 8 +-
regression-test/suites/update/test_update.groovy | 41 +-
.../max_msg_size_of_result_receiver.groovy | 61 ++
regression-test/suites/variant_p0/desc.groovy | 2 +-
.../suites/version_p0/version_p0.groovy | 3 +-
thirdparty/build-thirdparty.sh | 21 +
thirdparty/vars.sh | 7 +
429 files changed, 10869 insertions(+), 1946 deletions(-)
create mode 100644
be/src/vec/aggregate_functions/aggregate_function_foreach.cpp
create mode 100644 be/src/vec/aggregate_functions/aggregate_function_foreach.h
create mode 100644 docs/en/docs/ecosystem/bi-integration/apache-superset.md
create mode 100644 docs/zh-CN/docs/ecosystem/bi-integration/apache-superset.md
copy
fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/{SQLServerJdbcExecutor.java
=> PostgreSQLJdbcExecutor.java} (64%)
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/trinoconnector/TrinoColumnMetadata.java
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/trinoconnector/TrinoConnectorPluginManager.java
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/trinoconnector/TrinoConnectorServicesProvider.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/CloudTabletStatMgr.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/property/constants/TrinoConnectorProperties.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/trinoconnector/TrinoConnectorExternalCatalog.java
copy
fe/fe-core/src/main/java/org/apache/doris/datasource/{TableFormatType.java =>
trinoconnector/TrinoConnectorExternalCatalogFactory.java} (62%)
copy
fe/fe-core/src/main/java/org/apache/doris/datasource/{TableFormatType.java =>
trinoconnector/TrinoConnectorExternalDatabase.java} (53%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/trinoconnector/TrinoConnectorExternalTable.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/trinoconnector/source/TrinoConnectorScanNode.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/trinoconnector/source/TrinoConnectorSource.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/trinoconnector/source/TrinoConnectorSplit.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPartitionSyncConfig.java
copy
fe/fe-core/src/main/java/org/apache/doris/{datasource/TableFormatType.java =>
mtmv/MTMVPartitionSyncTimeUnit.java} (65%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mtmv/MTMVPropertyUtil.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/StructInfoMap.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/NullSafeEqualToEqual.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/{AggStateFunctionBuilder.java
=> AggCombinerFunctionBuilder.java} (74%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/combinator/{UnionCombinator.java
=> ForEachCombinator.java} (67%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/table/{ActiveQueries.java
=> ActiveBeTasks.java} (77%)
copy
fe/fe-core/src/main/java/org/apache/doris/{datasource/TableFormatType.java =>
rpc/TCustomProtocolFactory.java} (51%)
copy
fe/fe-core/src/main/java/org/apache/doris/tablefunction/{ActiveQueriesTableValuedFunction.java
=> ActiveBeTasksTableValuedFunction.java} (52%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mtmv/MTMVUtilTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/memo/StructInfoMapTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/NullSafeEqualToEqualTest.java
create mode 100644
regression-test/data/correctness_p0/test_distinct_streaming_agg_local_shuffle.out
create mode 100644 regression-test/data/function_p0/test_agg_foreach.out
create mode 100644
regression-test/data/mtmv_p0/test_hive_limit_partition_mtmv.out
create mode 100644 regression-test/data/mtmv_p0/test_limit_partition_mtmv.out
create mode 100644
regression-test/data/mv_p0/routine_load_mapping/routine_load_mapping.out
create mode 100644 regression-test/data/mv_p0/routine_load_mapping/test
copy regression-test/data/{correctness_p0/test_equal_for_null.out =>
mv_p0/test_mv_partition/test_mv_partition.out} (76%)
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query1.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query24.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query64.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query67.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query72.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query78.out
create mode 100644
regression-test/data/nereids_rules_p0/grouping_sets/test_grouping_sets_combination.out
copy regression-test/data/{correctness_p0/test_equal_for_null.out =>
nereids_syntax_p0/group_concat.out} (67%)
create mode 100644
regression-test/data/statistics/test_partition_first_load.csv
create mode 100644 regression-test/data/statistics/test_strict_mode_1.csv
create mode 100644 regression-test/data/statistics/test_strict_mode_2.csv
create mode 100644 regression-test/data/update/test_update.out
create mode 100644
regression-test/suites/alter_p0/test_alter_force_property.groovy
create mode 100644
regression-test/suites/correctness_p0/test_distinct_streaming_agg_local_shuffle.groovy
create mode 100644 regression-test/suites/function_p0/test_agg_foreach.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_hive_limit_partition_mtmv.groovy
create mode 100644
regression-test/suites/mtmv_p0/test_limit_partition_mtmv.groovy
create mode 100644
regression-test/suites/mv_p0/routine_load_mapping/routine_load_mapping.groovy
create mode 100644
regression-test/suites/mv_p0/test_mv_partition/test_mv_partition.groovy
copy regression-test/suites/{nereids_tpcds_shape_sf1000_p0 =>
nereids_hint_tpcds_p0}/ddl/gen_shape.py (100%)
copy regression-test/suites/{nereids_tpcds_shape_sf1000_p0 =>
nereids_hint_tpcds_p0}/ddl/shape.tmpl (100%)
create mode 100644 regression-test/suites/nereids_hint_tpcds_p0/load.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query1.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query24.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query64.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query67.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query72.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query78.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/grouping_sets/test_grouping_sets_combination.groovy
create mode 100644
regression-test/suites/schema_change_p0/test_delete_schema_change.groovy
delete mode 100644
regression-test/suites/schema_change_p0/test_delete_schema_change.sql
create mode 100644
regression-test/suites/schema_change_p0/test_partition_schema_change.groovy
delete mode 100644
regression-test/suites/schema_change_p0/test_partition_schema_change.sql
create mode 100644
regression-test/suites/statistics/analyze_stats_partition_first_load.groovy
create mode 100644
regression-test/suites/statistics/analyze_stats_triggered_by_update_row_count.groovy
create mode 100644
regression-test/suites/statistics/analyze_stats_triggered_by_update_row_count_streamload.groovy
create mode 100644
regression-test/suites/variable_p0/max_msg_size_of_result_receiver.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
