This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch auto-pick-50691-branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git
discard bbf65dcfad2 [fix](test)Fix the unstable test. (#50691) add c4e2f05563e branch-2.1: [fix](ut) fix unstable FE ut case for schema change job #50694 (#50887) add 33df5ba180f 2.1.10-rc01 (#50917) add c4d0e1e6935 branch-2.1: [fix](job scheduler) specifies both startTime and immediate, it will trigger one fewer task execution #50624 (#50897) add 01f70deb8b9 branch-2.1: [fix](nereids) fix parse date time exception #50810 (#50900) add 82d1375dc54 branch-2.1: [fix](Nereids) we should also push down expr in join's mark conjuncts #50886 (#50955) add 040cacd6353 branch-2.1: [fix](cooldown) allow cooldown_ttl = 0 when altering storage policy #50830 (#50856) add 13fbc9efa66 branch-2.1: [fix](hive) fix write hive partition by Doris #50864 (#50921) add e3fbf397226 branch-2.1: [fix](jdbc catalog) fix a jdbc catalog npe #50901 (#50928) add 505c9af95a1 [fix](inverted index) fix query error (#50860) (#50909) add 35af34f1cec branch-2.1: [fix](binlog) Record rollup index info for alterJob binlog #50850, #50337 (#50874) add fd378a303b9 branch-2.1: [Fix](external catalog) where tables in the information_schema could not be displayed #49607 (#50878) add 080bc8cbbe8 branch-2.1: [enhancement](compaction) generate multiple compaction tasks each round #49547 (#50991) add 127deb6d2a4 [Fix](Variant) fix array with predicate push down (#50969) add 5611a3988b4 [Fix](JsonPath) return null when meet unknown escape sequence, example '$.name\\k' (#50930) add 967adc3ce10 branch-2.1: [feature](restore) introduce AgentBoundedBatchTask to manage concurrent restore tasks #50740 (#50845) add 52fa3b5784b branch-2.1: [fix](mtmv)before mtmv refresh,check column type if change (#50730) add 51b39d09921 [fix](join)Consider mark join when computing right_col_idx(#50720) (#50727) add edb8a514148 branch-2.1: [fix](nereids) fix create view use null literal #49881 (#51006) add 3b60e91a0c1 branch-2.1:[statistics](regression)Add utf-8 encoding regression test. (#50466) (#51016) add b1a71fab2ce branch-2.1: [fix](pipelinex) fix null aware left anti join instance num #51053 (#51067) add 6fe7c9087e3 branch-2.1: [hotfix](jdbc catalog) Fix jdbcclient repeated initialization (#51038) add 71deeec2941 [conf](fe) Print jvm `ClassHistogram` in fe gc log after full gc (#44010) (#51007) add d63815fa574 branch-2.1: [fix](mv) refresh failed while open enable_single_replica_insert #50986 (#51021) add 5c344ea043d branch-2.1: [opt](docker) add a script flag to control load data or not #51065 (#51083) add 16b3e5ff18c [improve](information schema) introduce routine load job system table (#48963) (#49286) add 974263d83c5 [fix](join) Should not use the build block's size to resize mark_join_flags (#50993) (#51089) add 0287bc01c47 branch-2.1: [fix](ddl) Wrong result of scalar type `toString` for DATETIMEV2 #50237 (#50905) add ffbb10200fe [test](mtmv)Verifying master-slave consistency after schema change (#51106) add 81b5efa560a branch-2.1: [fix](planner)comparing partition column name should be case insensitive when pruning partition in DeleteJob #50674 (#51114) add fbad523a13c [cherry-pick](branch-21) pick (#50913) (#51072) add 5bc1618c2df branch-2.1: [fix](Nereids) could not work well when check precision for null literal #50815 (#50899) add 9e0e40c1e6c branch-2.1: [fix](nereids) Not use rule FOUR_PHASE_AGGREGATE_WITH_DISTINCT_WITH_FULL_DISTRIBUTE when mustUseMultiDistinctAgg #51099 (#51140) add 3bc24da9c2a branch-2.1: [fix](meta) fix timeout for syncing image when FE scales out #51120 (#51149) add 8cca998ce6c branch-2.1:[fix](case)fix mark_join and right_semi_mark_join has same table name… (#51158) add df464f84b15 branch-2.1: [fix](load) fix the error msg of task submission failure for memory back pressure #51078 (#51131) add fc9beac4681 branch-2.1: [fix](arrow-flight-sql) Separate arrow-flight-sql connection and mysql connection (#51110) add 16521017add [fix](iceberg)Table operations are not supported for catalogs of the dlf type for 2.1 #50696 (#51112) add 6513f9e01f7 [cherry-pick](branch-21)add some check for udf when result is null (#51084) (#51196) add 701fc5fa2cc [fix](export) remove task from map finally (#51171) (#51207) add ac65fed0edc branch-2.1: [fix](jdbc test) Add more connections to mysql docker #50970 (#51210) add 67e400483d0 [branch-2.1 fix](expr) fix expr set result column id even when state is not ok #50938 (#51225) add 73ee4c90909 branch-2.1: [chore](cataglog) Unlimit db data size quota (#51108) (#51235) add c22fc7d8e53 branch-2.1: [test](mtmv)fix master-slave consistency after schema change #51191 (#51252) add 9713c7a1e50 [cherry-pick](branch-2.1) Pick "[Fix](case) Fix cold data compaction fault injection case (#49699)" (#51249) add c838678b181 [bugfix](nerids) complete the implementation of the concat method. (#51141) #51259 (#51273) add dd89a78d300 branch-2.1:[fix](parquet/orc) Disable string dictionary filtering when predicate express is not binary pred and in pred (#50749) (#51266) add 7d2134daded branch-2.1: [fix](arrow) Rollback Arrow version 19.0.1 -> 17.0.0, Fix MacOS compile error and decimal type error when convert to Parquet (#51291) add bbb63ee20db [regression-test](improvement) show line number when global var defined (#51256) add 4e3700eccf8 [branch-2.1: [Fix](Catalog)Ensure preExecutionAuthenticator is properly initialized (#51279) add 91abfe38cdc [Optimize](orc-reader) Optimize stripe footer multiple reads in orc reader. (#51276) add 25f5d5fb1e9 [fix](nereids) fix colocate agg + join compute wrong result (#48934) (#51313) add 06a6d0c5e98 branch-2.1: [fix](agent) cancel agent task when it is rejected by agent-task-pool #51138 (#51212) add 42f4498c9d7 branch-2.1: [fix](iceberg) fix the iceberg eq-delete filter resize-fill bug. #51253 (#51316) add 92fa037e5aa branch-2.1: [enhancement](memoryfailed) throw exception if block.serialize meet error status #51335 (#51365) add ee74a4062c0 branch-2.1: [Fix] (nereids) Fix incorrect hive references in LogicalIcebergTableSink and update PlanType #49306 (#51287) add c22db94c3a2 [opt](mtmv) optimize mtmv rewrite performance (#49514) (#51309) add 8cb499ec7cc branch-2.1: [opt](audit) add new variable audit_plugin_max_insert_stmt_length to limit the length of insert stmt #51314 (#51391) add ffb6fd44b52 branch-2.1: [bugfix](fold) Skip constant folding in from_base64 function to fix binary data handling #50453 (#51413) add 2b9d8542418 [regression-test](fix) fix global vars (#51424) add 47a714e8c2e branch-2.1:[case](auth)fix case test_dml_select_udf_auth (#51449) add 2eee0ebdafe [regression-test](fix) global var (#51474) add 14cb4c90adb [Fix](Load) fix load hang when the instance cancel with memory tracker before exe… (#51428) add 831d2c148a8 branch-2.1:[enhance](mtmv)cache table snapshot in refresh context (#50855) (#51493) add c8daf2e4fb0 branch-2.1: [fix](cache prune) Fix _stale_sweep_time_s not working when its value less than 0 #42281 (#51189) add fc19a29dbe1 [fix](test)Fix the unstable test. (#50691) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (bbf65dcfad2) \ N -- N -- N refs/heads/auto-pick-50691-branch-2.1 (fc19a29dbe1) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: be/src/apache-orc | 2 +- be/src/common/config.cpp | 2 +- be/src/exec/schema_scanner.cpp | 3 + be/src/exec/schema_scanner/schema_helper.cpp | 9 + be/src/exec/schema_scanner/schema_helper.h | 6 + .../schema_routine_load_job_scanner.cpp | 199 +++++++ .../schema_routine_load_job_scanner.h | 41 +- be/src/olap/rowset/segment_v2/segment.h | 6 +- be/src/olap/tablet.cpp | 10 +- be/src/pipeline/exec/hashjoin_probe_operator.cpp | 31 +- be/src/pipeline/exec/hashjoin_probe_operator.h | 6 + be/src/runtime/memory/lru_cache_policy.h | 2 +- be/src/runtime/plan_fragment_executor.cpp | 4 +- .../routine_load/routine_load_task_executor.cpp | 29 +- .../routine_load/routine_load_task_executor.h | 2 +- .../aggregate_functions/aggregate_function_sort.h | 8 +- be/src/vec/common/hash_table/join_hash_table.h | 44 +- be/src/vec/common/pod_array.h | 4 + be/src/vec/exec/format/orc/vorc_reader.cpp | 34 +- .../exec/format/parquet/vparquet_group_reader.cpp | 38 +- be/src/vec/exec/format/table/equality_delete.cpp | 6 +- be/src/vec/exec/join/process_hash_table_probe.h | 10 +- .../vec/exec/join/process_hash_table_probe_impl.h | 139 +++-- be/src/vec/exec/join/vhash_join_node.cpp | 5 + be/src/vec/exec/join/vhash_join_node.h | 3 + .../exprs/lambda_function/varray_map_function.cpp | 17 +- be/src/vec/exprs/vcast_expr.cpp | 2 + be/src/vec/exprs/vexpr.cpp | 4 + be/src/vec/functions/function_json.cpp | 19 +- .../exprs/vexpr_evalute_inverted_index_test.cpp | 172 ++++++ conf/fe.conf | 4 +- .../docker-compose/hive/hadoop-hive-2x.env.tpl | 2 + .../docker-compose/hive/hadoop-hive-3x.env.tpl | 2 +- .../docker-compose/hive/hadoop-hive.env.tpl | 4 + .../scripts/create_preinstalled_scripts/run77.hql | 30 + .../docker-compose/hive/scripts/hive-metastore.sh | 8 + docker/thirdparties/docker-compose/mysql/my.cnf | 3 +- docker/thirdparties/run-thirdparties-docker.sh | 7 + .../apache/doris/common/jni/vec/VectorColumn.java | 52 +- .../main/java/org/apache/doris/common/Config.java | 41 +- .../authentication/HadoopAuthenticator.java | 7 + .../authentication/PreExecutionAuthenticator.java | 15 + .../main/java/org/apache/doris/alter/Alter.java | 22 +- .../java/org/apache/doris/alter/RollupJobV2.java | 16 + .../org/apache/doris/analysis/SchemaTableType.java | 4 +- .../java/org/apache/doris/backup/RestoreJob.java | 42 +- .../org/apache/doris/binlog/AlterJobRecord.java | 17 + .../java/org/apache/doris/catalog/Database.java | 22 +- .../main/java/org/apache/doris/catalog/Env.java | 11 +- .../java/org/apache/doris/catalog/SchemaTable.java | 24 + .../java/org/apache/doris/common/TreeNode.java | 14 + .../doris/common/profile/SummaryProfile.java | 17 +- .../apache/doris/common/util/PropertyAnalyzer.java | 8 +- .../apache/doris/datasource/ExternalCatalog.java | 19 + .../apache/doris/datasource/ExternalDatabase.java | 10 +- .../apache/doris/datasource/InternalCatalog.java | 3 - .../doris/datasource/hive/HMSExternalCatalog.java | 14 +- .../doris/datasource/hive/HMSTransaction.java | 12 +- .../doris/datasource/hive/HiveMetadataOps.java | 2 +- .../iceberg/IcebergDLFExternalCatalog.java | 33 ++ .../datasource/iceberg/IcebergExternalCatalog.java | 10 +- .../iceberg/IcebergHadoopExternalCatalog.java | 13 +- .../doris/datasource/jdbc/JdbcExternalCatalog.java | 51 +- .../datasource/paimon/source/PaimonScanNode.java | 2 +- .../doris/job/base/JobExecutionConfiguration.java | 10 +- .../apache/doris/job/extensions/mtmv/MTMVTask.java | 42 ++ .../main/java/org/apache/doris/load/DeleteJob.java | 3 +- .../load/routineload/KafkaRoutineLoadJob.java | 12 +- .../doris/load/routineload/RoutineLoadJob.java | 63 +- .../doris/load/routineload/RoutineLoadManager.java | 10 + .../load/routineload/RoutineLoadProgress.java | 2 +- .../load/routineload/RoutineLoadStatistic.java | 1 + .../java/org/apache/doris/master/MasterImpl.java | 4 + .../org/apache/doris/master/ReportHandler.java | 10 +- .../org/apache/doris/mtmv/MTMVHookService.java | 10 +- .../java/org/apache/doris/mtmv/MTMVJobManager.java | 5 +- .../org/apache/doris/mtmv/MTMVPartitionUtil.java | 27 +- .../java/org/apache/doris/mtmv/MTMVPlanUtil.java | 172 ++++++ .../org/apache/doris/mtmv/MTMVRefreshContext.java | 10 + .../org/apache/doris/mtmv/MTMVRelationManager.java | 41 +- .../org/apache/doris/mtmv/MTMVRewriteUtil.java | 45 +- .../java/org/apache/doris/mtmv/MTMVService.java | 47 +- .../org/apache/doris/mysql/AcceptListener.java | 7 +- .../org/apache/doris/nereids/CascadesContext.java | 5 + .../org/apache/doris/nereids/NereidsPlanner.java | 21 +- .../java/org/apache/doris/nereids/PlannerHook.java | 12 + .../org/apache/doris/nereids/StatementContext.java | 43 +- .../nereids/analyzer/UnboundIcebergTableSink.java | 4 +- .../glue/translator/PhysicalPlanTranslator.java | 15 + .../jobs/executor/TablePartitionCollector.java | 46 ++ .../apache/doris/nereids/memo/StructInfoMap.java | 4 + .../doris/nereids/parser/LogicalPlanBuilder.java | 32 +- .../org/apache/doris/nereids/rules/RuleType.java | 1 + .../mv/AbstractMaterializedViewRule.java | 231 +++----- .../rules/exploration/mv/HyperGraphComparator.java | 41 +- .../mv/InitMaterializationContextHook.java | 17 +- .../exploration/mv/MaterializationContext.java | 37 +- ...terializedViewAggregateOnNoneAggregateRule.java | 12 +- .../rules/exploration/mv/PartitionCompensator.java | 199 +++++++ .../nereids/rules/exploration/mv/StructInfo.java | 113 +--- .../expression/rules/FoldConstantRuleOnBE.java | 18 +- .../rules/implementation/AggregateStrategies.java | 2 + ...ogicalOlapTableSinkToPhysicalOlapTableSink.java | 3 +- .../rules/rewrite/PruneFileScanPartition.java | 1 - .../PushDownExpressionsInHashCondition.java | 59 +- .../rules/rewrite/QueryPartitionCollector.java | 95 +++ .../expressions/functions/SearchSignature.java | 4 +- .../functions/executable/StringArithmetic.java | 9 +- .../apache/doris/nereids/trees/plans/PlanType.java | 1 + .../plans/commands/info/AlterMTMVRenameInfo.java | 6 +- .../plans/commands/info/AlterMTMVReplaceInfo.java | 11 +- .../trees/plans/commands/info/BaseViewInfo.java | 14 +- .../trees/plans/commands/info/CreateMTMVInfo.java | 104 +--- .../commands/info/DistributionDescriptor.java | 4 + .../plans/logical/LogicalIcebergTableSink.java | 10 +- .../nereids/trees/plans/logical/LogicalJoin.java | 17 +- .../plans/visitor/ExpressionLineageReplacer.java | 27 +- .../org/apache/doris/planner/HashJoinNode.java | 8 - .../org/apache/doris/planner/PlanFragment.java | 4 +- .../java/org/apache/doris/planner/PlanNode.java | 4 - .../org/apache/doris/policy/StoragePolicy.java | 4 +- .../java/org/apache/doris/qe/AuditLogHelper.java | 42 +- .../java/org/apache/doris/qe/ConnectContext.java | 2 +- .../{ConnectScheduler.java => ConnectPoolMgr.java} | 86 +-- .../java/org/apache/doris/qe/ConnectScheduler.java | 176 ++---- .../java/org/apache/doris/qe/GlobalVariable.java | 11 + .../java/org/apache/doris/qe/SessionVariable.java | 9 + .../scheduler/manager/TransientTaskManager.java | 2 +- .../java/org/apache/doris/service/ExecuteEnv.java | 2 +- .../apache/doris/service/FrontendServiceImpl.java | 58 +- .../arrowflight/DorisFlightSqlProducer.java | 49 +- .../service/arrowflight/DorisFlightSqlService.java | 12 +- .../sessions/FlightSessionsManager.java | 7 + .../sessions/FlightSessionsWithTokenManager.java | 22 +- .../sessions/FlightSqlConnectContext.java | 15 +- .../sessions/FlightSqlConnectPoolMgr.java | 74 +++ .../arrowflight/tokens/FlightTokenManagerImpl.java | 18 +- .../java/org/apache/doris/task/AgentBatchTask.java | 42 +- .../apache/doris/task/AgentBoundedBatchTask.java | 278 +++++++++ .../main/java/org/apache/doris/task/AgentTask.java | 11 + .../org/apache/doris/task/AgentTaskExecutor.java | 19 +- .../java/org/apache/doris/task/AgentTaskQueue.java | 13 + .../apache/doris/catalog/CreateFunctionTest.java | 10 +- .../apache/doris/common/proc/DbsProcDirTest.java | 4 +- .../dlf/client/IcebergDLFExternalCatalogTest.java | 14 + ...bleNameComparedLowercaseMetaCacheFalseTest.java | 9 + .../job/base/JobExecutionConfigurationTest.java | 11 +- .../apache/doris/mtmv/MTMVPartitionUtilTest.java | 21 + .../org/apache/doris/mtmv/MTMVPlanUtilTest.java | 282 +++++++++ .../org/apache/doris/mtmv/MTMVRewriteUtilTest.java | 41 +- .../doris/nereids/memo/StructInfoMapTest.java | 10 +- .../doris/nereids/mv/IdStatisticsMapTest.java | 3 +- .../nereids/mv/MtmvCacheNewConnectContextTest.java | 6 +- .../doris/nereids/mv/MvTableIdIsLongTest.java | 4 +- .../nereids/mv/OptimizeGetAvailableMvsTest.java | 269 +++++++++ .../exploration/mv/PartitionCompensatorTest.java | 186 ++++++ .../PushDownExpressionsInHashConditionTest.java | 83 +++ .../nereids/trees/expressions/LiteralTest.java | 8 +- .../org/apache/doris/nereids/util/PlanChecker.java | 12 +- .../doris/plugin/audit/AuditLogBuilderTest.java | 641 ++++++++++++++++++++- gensrc/script/gen_build_version.sh | 4 +- gensrc/thrift/Descriptors.thrift | 3 +- gensrc/thrift/FrontendService.thrift | 32 + .../create_view_nereids_fix_null.out | Bin 0 -> 153 bytes .../hive/test_string_dict_filter.out | Bin 11747 -> 20131 bytes .../hive/write/test_hive_write_partitions.out | Bin 73239 -> 74043 bytes .../data/inverted_index_p0/test_expr_in_null.out | Bin 0 -> 151 bytes .../mtmv_p0/test_base_add_col_multi_level_mtmv.out | Bin 0 -> 403 bytes ...est_base_alter_col_comment_multi_level_mtmv.out | Bin 0 -> 443 bytes .../test_base_alter_col_type_multi_level_mtmv.out | Bin 0 -> 452 bytes .../mtmv_p0/test_base_comment_multi_level_mtmv.out | Bin 0 -> 403 bytes .../test_base_drop_col_multi_level_mtmv.out | Bin 0 -> 428 bytes .../mtmv_p0/test_base_drop_multi_level_mtmv.out | Bin 0 -> 400 bytes .../mtmv_p0/test_base_recreate_on_commit_mtmv.out | Bin 0 -> 288 bytes .../test_base_rename_col_multi_level_mtmv.out | Bin 0 -> 436 bytes .../mtmv_p0/test_base_rename_multi_level_mtmv.out | Bin 0 -> 408 bytes .../test_base_rename_mv_multi_level_mtmv.out | Bin 0 -> 421 bytes .../mtmv_p0/test_base_rename_on_commit_mtmv.out | Bin 0 -> 445 bytes .../test_base_rename_p_mv_multi_level_mtmv.out | Bin 0 -> 429 bytes .../mtmv_p0/test_base_replace_multi_level_mtmv.out | Bin 0 -> 1075 bytes .../test_base_replace_mv_multi_level_mtmv.out | Bin 0 -> 910 bytes .../mtmv_p0/test_base_replace_on_commit_mtmv.out | Bin 0 -> 349 bytes .../data/mtmv_p0/test_column_type_mtmv.out | Bin 0 -> 291 bytes .../data/mtmv_p0/test_immediate_starttime_mtmv.out | Bin 0 -> 134 bytes .../partition_mv_rewrite.out | Bin 2856 -> 4048 bytes .../unioin_rewrite_grace_big.out} | Bin 2856 -> 3997 bytes .../data/query_p0/join/mark_join/mark_join.out | Bin 239 -> 1007 bytes .../join/mark_join/right_semi_mark_join.out | Bin 0 -> 568 bytes .../array_functions/test_array_map_function.out | Bin 2417 -> 2522 bytes .../json_functions/test_json_function.out | Bin 1295 -> 1341 bytes .../data/variant_p0/array_with_predicate.out | Bin 0 -> 261 bytes .../org/apache/doris/regression/suite/Suite.groovy | 4 +- .../doris/regression/util/LoggerUtils.groovy | 14 +- .../main/java/org/apache/doris/udf/StringTest.java | 3 + regression-test/pipeline/external/conf/fe.conf | 4 +- regression-test/pipeline/p0/conf/fe.conf | 4 +- regression-test/pipeline/p1/conf/fe.conf | 4 +- .../suites/account_p0/test_property_session.groovy | 4 +- .../suites/audit/test_audit_log_behavior.groovy | 24 +- .../test_account_management_grant_auth.groovy | 6 +- .../test_account_management_role_auth.groovy | 4 +- .../test_account_management_user_auth.groovy | 6 +- .../auth_call/test_assistant_command_auth.groovy | 6 +- .../auth_call/test_cluster_management_auth.groovy | 6 +- .../auth_call/test_database_management_auth.groovy | 2 +- .../suites/auth_call/test_ddl_backup_auth.groovy | 4 +- .../suites/auth_call/test_ddl_catalog_auth.groovy | 14 +- .../auth_call/test_ddl_colocate_group_auth.groovy | 4 +- .../suites/auth_call/test_ddl_database_auth.groovy | 14 +- .../auth_call/test_ddl_encryptkey_auth.groovy | 4 +- .../suites/auth_call/test_ddl_file_auth.groovy | 6 +- .../suites/auth_call/test_ddl_function_auth.groovy | 6 +- .../suites/auth_call/test_ddl_index_auth.groovy | 4 +- .../suites/auth_call/test_ddl_job_auth.groovy | 4 +- .../auth_call/test_ddl_mask_view_auth.groovy | 4 +- .../suites/auth_call/test_ddl_mtmv_auth.groovy | 22 +- .../suites/auth_call/test_ddl_mv_auth.groovy | 10 +- .../auth_call/test_ddl_part_table_auth.groovy | 6 +- .../auth_call/test_ddl_policy_storage_auth.groovy | 4 +- .../auth_call/test_ddl_repository_auth.groovy | 4 +- .../suites/auth_call/test_ddl_resource_auth.groovy | 4 +- .../suites/auth_call/test_ddl_restore_auth.groovy | 6 +- .../auth_call/test_ddl_row_policy_auth.groovy | 6 +- .../auth_call/test_ddl_sql_block_rule_auth.groovy | 4 +- .../suites/auth_call/test_ddl_table_auth.groovy | 38 +- .../suites/auth_call/test_ddl_view_auth.groovy | 18 +- .../auth_call/test_ddl_workload_group_auth.groovy | 4 +- .../suites/auth_call/test_dml_analyze_auth.groovy | 6 +- .../auth_call/test_dml_broker_load_auth.groovy | 6 +- .../auth_call/test_dml_cancel_profile_auth.groovy | 4 +- .../auth_call/test_dml_delete_table_auth.groovy | 8 +- .../auth_call/test_dml_export_table_auth.groovy | 6 +- .../suites/auth_call/test_dml_insert_auth.groovy | 8 +- .../test_dml_multi_routine_load_auth.groovy | 8 +- .../auth_call/test_dml_mysql_load_auth.groovy | 4 +- .../suites/auth_call/test_dml_outfile_auth.groovy | 4 +- .../auth_call/test_dml_routine_load_auth.groovy | 8 +- .../auth_call/test_dml_select_udf_auth.groovy | 5 +- .../auth_call/test_dml_stream_load_auth.groovy | 4 +- .../auth_call/test_dml_update_table_auth.groovy | 6 +- .../auth_call/test_hive_base_case_auth.groovy | 24 +- .../suites/auth_call/test_show_backend_auth.groovy | 4 +- .../suites/auth_call/test_show_broker_auth.groovy | 4 +- .../suites/auth_call/test_show_charset_auth.groovy | 6 +- .../test_show_convert_light_sc_auth.groovy | 4 +- .../auth_call/test_show_create_table_auth.groovy | 10 +- .../suites/auth_call/test_show_data_auth.groovy | 6 +- .../auth_call/test_show_database_id_auth.groovy | 4 +- .../auth_call/test_show_dynamic_table_auth.groovy | 4 +- .../auth_call/test_show_frontend_auth.groovy | 4 +- .../suites/auth_call/test_show_grant_auth.groovy | 4 +- .../suites/auth_call/test_show_proc_auth.groovy | 4 +- .../auth_call/test_show_query_stats_auth.groovy | 2 +- .../auth_call/test_show_repository_auth.groovy | 4 +- .../suites/auth_call/test_show_tablet_auth.groovy | 4 +- .../auth_call/test_show_typecast_auth.groovy | 2 +- .../suites/auth_p0/test_catalogs_auth.groovy | 4 +- .../suites/auth_p0/test_mtmv_auth.groovy | 4 +- .../auth_p0/test_partition_values_tvf_auth.groovy | 4 +- .../suites/auth_p0/test_partitions_auth.groovy | 4 +- .../suites/auth_p0/test_query_tvf_auth.groovy | 4 +- .../suites/auth_p0/test_select_count_auth.groovy | 6 +- .../test_overall_compaction_status.groovy | 2 +- ...e_compaction_with_variant_inverted_index.groovy | 8 +- .../test_single_replica_compaction.groovy | 6 +- .../test_sequence_col_datetimev2.groovy | 98 ++++ .../create_view_nereids_fix_null.groovy | 57 ++ .../hive/test_string_dict_filter.groovy | 36 ++ .../hive/write/test_hive_write_partitions.groovy | 22 +- .../jdbc/test_mysql_jdbc_statistics.groovy | 2 +- .../external_table_p2/tvf/test_iceberg_meta.groovy | 4 +- .../insert_group_commit_with_exception.groovy | 20 +- .../test_group_commit_interval_ms_property.groovy | 2 +- .../inverted_index_p0/test_expr_in_null.groovy | 68 +++ .../test_index_like_select.groovy | 4 +- .../test_variant_index_format_v1.groovy | 2 +- .../suites/javaudf_p0/test_javaudf_array.groovy | 11 + .../suites/javaudf_p0/test_javaudf_int.groovy | 26 + .../suites/javaudf_p0/test_javaudf_string.groovy | 13 + .../test_routine_load_error_info.groovy | 174 +++--- .../test_routine_load_job_info_system_table.groovy | 142 +++++ .../test_base_add_col_multi_level_mtmv.groovy | 140 +++++ ..._base_alter_col_comment_multi_level_mtmv.groovy | 140 +++++ ...est_base_alter_col_type_multi_level_mtmv.groovy | 140 +++++ .../test_base_comment_multi_level_mtmv.groovy | 139 +++++ .../test_base_drop_col_multi_level_mtmv.groovy | 141 +++++ .../mtmv_p0/test_base_drop_multi_level_mtmv.groovy | 152 +++++ .../test_base_recreate_on_commit_mtmv.groovy | 86 +++ .../test_base_rename_col_multi_level_mtmv.groovy | 140 +++++ .../test_base_rename_multi_level_mtmv.groovy | 141 +++++ .../test_base_rename_mv_multi_level_mtmv.groovy | 148 +++++ .../mtmv_p0/test_base_rename_on_commit_mtmv.groovy | 105 ++++ .../test_base_rename_p_mv_multi_level_mtmv.groovy | 154 +++++ .../test_base_replace_multi_level_mtmv.groovy | 180 ++++++ .../test_base_replace_mv_multi_level_mtmv.groovy | 194 +++++++ .../test_base_replace_on_commit_mtmv.groovy | 100 ++++ .../suites/mtmv_p0/test_column_type_mtmv.groovy | 61 ++ ...groovy => test_immediate_starttime_mtmv.groovy} | 77 ++- .../suites/mtmv_p0/test_mtmv_outfile.groovy | 2 +- .../suites/mtmv_p0/test_recreate_table_mtmv.groovy | 3 + .../mv/availability/grace_period.groovy | 4 +- .../partition_mv_rewrite.groovy | 293 +++++----- .../rewrite_duration_exceeded.groovy | 156 +++++ .../unioin_rewrite_grace_big.groovy} | 312 +++++----- .../suites/nereids_syntax_p0/agg_4_phase.groovy | 10 + .../decimal_percision_compute.groovy | 48 ++ .../distribute/colocate_agg_join.groovy | 84 +++ .../nereids_syntax_p0/test_cast_datetime.groovy | 14 + .../query_p0/join/mark_join/mark_join.groovy | 251 +++++++- .../join/mark_join/right_semi_mark_join.groovy | 109 ++++ .../array_functions/test_array_map_function.groovy | 26 + .../json_functions/test_json_function.groovy | 5 + .../rollup/test_materialized_view_bitmap.groovy | 2 +- .../rollup/test_materialized_view_hll.groovy | 2 +- ...test_materialized_view_hll_with_light_sc.groovy | 2 +- .../test_uniq_rollup_schema_change.groovy | 17 +- .../suites/statistics/test_utf_encoding.groovy | 53 ++ .../suites/variant_p0/array_with_predicate.groovy | 37 ++ .../suites/variant_p0/complexjson.groovy | 2 +- samples/arrow-flight-sql/java/pom.xml | 4 +- thirdparty/vars.sh | 2 + 321 files changed, 8712 insertions(+), 1893 deletions(-) create mode 100644 be/src/exec/schema_scanner/schema_routine_load_job_scanner.cpp copy fe/fe-core/src/main/java/org/apache/doris/task/AgentTaskExecutor.java => be/src/exec/schema_scanner/schema_routine_load_job_scanner.h (51%) create mode 100644 be/test/vec/exprs/vexpr_evalute_inverted_index_test.cpp create mode 100755 docker/thirdparties/docker-compose/hive/scripts/create_preinstalled_scripts/run77.hql create mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/TablePartitionCollector.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/PartitionCompensator.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/QueryPartitionCollector.java copy fe/fe-core/src/main/java/org/apache/doris/qe/{ConnectScheduler.java => ConnectPoolMgr.java} (61%) create mode 100644 fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSqlConnectPoolMgr.java create mode 100644 fe/fe-core/src/main/java/org/apache/doris/task/AgentBoundedBatchTask.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/mtmv/MTMVPlanUtilTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/nereids/mv/OptimizeGetAvailableMvsTest.java create mode 100644 fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/mv/PartitionCompensatorTest.java create mode 100644 regression-test/data/ddl_p0/create_view_nereids/create_view_nereids_fix_null.out create mode 100644 regression-test/data/inverted_index_p0/test_expr_in_null.out create mode 100644 regression-test/data/mtmv_p0/test_base_add_col_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_alter_col_comment_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_alter_col_type_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_comment_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_drop_col_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_drop_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_recreate_on_commit_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_rename_col_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_rename_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_rename_mv_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_rename_on_commit_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_rename_p_mv_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_replace_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_replace_mv_multi_level_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_base_replace_on_commit_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_column_type_mtmv.out create mode 100644 regression-test/data/mtmv_p0/test_immediate_starttime_mtmv.out copy regression-test/data/nereids_rules_p0/mv/{ => partition_union_rewrite}/partition_mv_rewrite.out (68%) rename regression-test/data/nereids_rules_p0/mv/{partition_mv_rewrite.out => union_rewrite_grace_big/unioin_rewrite_grace_big.out} (69%) create mode 100644 regression-test/data/query_p0/join/mark_join/right_semi_mark_join.out create mode 100644 regression-test/data/variant_p0/array_with_predicate.out create mode 100644 regression-test/suites/correctness_p0/test_sequence_col_datetimev2.groovy create mode 100644 regression-test/suites/ddl_p0/create_view_nereids/create_view_nereids_fix_null.groovy create mode 100644 regression-test/suites/inverted_index_p0/test_expr_in_null.groovy create mode 100644 regression-test/suites/load_p0/routine_load/test_routine_load_job_info_system_table.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_add_col_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_alter_col_comment_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_alter_col_type_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_comment_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_drop_col_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_drop_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_recreate_on_commit_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_rename_col_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_rename_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_rename_mv_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_rename_on_commit_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_rename_p_mv_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_replace_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_replace_mv_multi_level_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_base_replace_on_commit_mtmv.groovy create mode 100644 regression-test/suites/mtmv_p0/test_column_type_mtmv.groovy copy regression-test/suites/mtmv_p0/{test_mtmv_outfile.groovy => test_immediate_starttime_mtmv.groovy} (50%) copy regression-test/suites/nereids_rules_p0/mv/{ => partition_union_rewrite}/partition_mv_rewrite.groovy (68%) create mode 100644 regression-test/suites/nereids_rules_p0/mv/rewrite_duration_exceeded/rewrite_duration_exceeded.groovy rename regression-test/suites/nereids_rules_p0/mv/{partition_mv_rewrite.groovy => union_rewrite_grace_big/unioin_rewrite_grace_big.groovy} (67%) create mode 100644 regression-test/suites/nereids_syntax_p0/decimal_percision_compute.groovy create mode 100644 regression-test/suites/nereids_syntax_p0/distribute/colocate_agg_join.groovy create mode 100644 regression-test/suites/query_p0/join/mark_join/right_semi_mark_join.groovy create mode 100644 regression-test/suites/statistics/test_utf_encoding.groovy create mode 100644 regression-test/suites/variant_p0/array_with_predicate.groovy --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org