This is an automated email from the ASF dual-hosted git repository.
kakachen pushed a change to branch data_lake_reader_refactoring
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 562e67709c8 fix
discard ca633f860d0 [refactor](be) Migrate parquet test files to new NVI
init_reader API
discard e037094cfda update.
discard 5a9ef7f4eb9 [fix](be) Fix standalone reader init for Parquet and ORC
discard 2110b9c9ef9 [refactor](be) Move fill-column computation to
on_before_init_reader
discard b4ff04520d2 [refactor](be) Move column-filling logic from
GenericReader to TableFormatReader
discard 59ace1b6f1e [refactor](be) Introduce TableFormatReader with auto
column filling
discard 8e33b1c6ba6 [refactor](be) Apply NVI template to get_next_block
discard cdde6f17502 [refactor](be) Unify all standalone readers to use NVI
init_reader
discard 82311b28826 unify reader init.
discard 347f48f5b7c fix.
discard ef7fb03e700 1. Refactor count agg push down. 2. Load query decoupling.
3. unify FE default value.
discard 0d5a1e53463 [refactoring](multi-catalog)data_lake_reader_refactoring.
add 8e706d1a076 [fix](job) fix StreamingInsertJob incorrectly counting
canceled tasks (#61894)
add 4434ba4e7b5 [refine](predicate) reduce HybridSet/InListPredicate
template bloat and fix create_string_value_set bug (#61999)
add d4c19dac5c6 [fix](regression) Add external regression stage timing
summary (#61869)
add 4eab83d0332 [improvement](build) Limit hive bootstrap data by version
(#61852)
add c1168522ed5 [feature](fe) Integrate OIDC authentication, MySQL login
bridge, and role mapping (#61819)
add e61837c24eb [fix](backup) Fix UnsupportedOperationException in
RestoreCommand by copying properties before modification (#61890)
add e4ed81fdb1b [fix](spill) Fix revocable_mem_size to return 0 when
partition reaches max repartition depth (#61973)
add 20b05a9a6d8 [chore](ci) Remove author association check from
opencode-review workflow (#62074)
add 0adc79074b1 [fix](variant) preserve subcolumns after row-store partial
update (#62067)
add 1ce2e40dc4f [fix](scan) Fix missing predicate filter when Native and
JNI readers are mixed in FileScanner (#61929)
add d5e95188554 [Bug](scan) Preserve IN_LIST runtime filter predicates
when key range is a scope range (#62027)
add 632f79197c2 [Exec](be) Support offset prue column and null column in
BE (#61888)
add 6fb376a094c [opt](memory) Remove unused indexId field from
CloudReplica (#62080)
add 84c1c36575e [fix](compaction) Remove upper limit for
vertical_compaction_num_columns_per_group (#61992)
add 977c1f1134c [Bug](pipeline) do not treat BLOCKED->FINISHED as error
when wake_up_early is true (#61989)
add ac27e4b7690 [env](compiler) Reduce compile time of
aggregate_function_reader_replace.cpp (#62047)
add 9689b8288df [fix](fe) Enable fuzzy for enable_query_cache (#62087)
add 3160231a3e1 [Bug](pipeline) do not set shared hash table signaled when
task terminated (#62056)
add 186a53ebe7d [opt](memory) Remove unused dbId field from CloudReplica
(#62079)
add d1e09c408eb [feat](fe) Add ClusterGuard SPI interface for
cluster-level policy enforcement (#62031)
add a8c890545ec [fix](jdbc) Preserve query tvf column aliases across JDBC
catalogs (#61939)
add 157a35e2580 [opt](explode) Optimize explode_outer and posexplode
(#62069)
add 0a3bea51271 [refactor](fe) Decouple SlotRef dependencies:
GlobalVariable, Column serialization, Expr visitor (#61918)
add 405541b2de5 [fix](tvf) fix wrong backend assignment for insert into
local tvf (#62143)
add a770f16d9f3 [Feature](compaction) add CompactionTaskTracker with
system table and HTTP API (#61696)
add 4ad3989c712 [fix](fe) Fix null pointer exception in sessionVariables
after upgrade (#61959)
add 16682030e14 [bugfix] fix show partitions command for external catalog
validations incompatible with partitions tvf (#62134)
add d224c08fbb6 [opt](memory) Remove unused fields from CloudReplica
(#62125)
add f754d728b99 [refactor](build) Extract third-party filesystem JAR
packaging into post-build.sh (#62144)
add 03290d2399a [feature](tso) Add global monotonically increasing
Timestamp Oracle(TSO) (#61199)
add a0b9e3784f9 [Bug](function) fix wrong result of window funnel v2 +
deduplication/fixed mode (#62043)
add 9c7ec9e132c [fix](tpcds) Change sr_ticket_number and ss_ticket_number
to bigint in SF10000 schema (#62158)
add 2cf8a386e28 [fix](lineage) Propagate connectAttributes through FE
forward path for scheduleInfo (#62082)
add 958731935a3 [chore](beut) using uuid as dir name to make the test more
stable (#62127)
add 32d40262c3f [fix](metrics) Fix prepared statement QPS metrics not
counted when audit log disabled (#61621)
add 0e4ffd0b40a [refine](be) remove -Wno-deprecated-declarations and fix
all deprecated usages (#62098)
add 63654958c67 [fix](audit) Fix UnboundAlias digest when name comes from
child (#62160)
add 361bbed335a [fix](profile) populate inverted index metrics for
SEARCH() query path (#62121)
add 094438f551c [refactor](fe) Introduce pluggable filesystem SPI to unify
cloud/storage and fe-core/fs (#62023)
add a2bc37f021e [fix](auth) Fix HTTP API authentication framework for
admin operations (#60761)
add 0ba5b455250 [refactor](fe) Decouple DecimalLiteral from
SessionVariable (#62166)
add b31412ae784 [chore](cloud) Change range iterator check pattern from
do...while() to while() to get better robustness (#59019)
add 7bd2ddae9fc [bugfix] fix doris build failure because of no cuda
support scene, and add FAISS_ENABLE_GPU switch to enable it or not by user,
default value as OFF (#62133)
add c39633520a0 [fix](fe) Fix NoSuchElementException when count with
MATCH_ALL expression (#62172)
add b95bf645c4b [refactor](fe) Decouple analysis.DateLiteral from
nereids.util.DateUtils (#62154)
add 7ad64890477 [fix](memtable) using memtable memory instead of load
memory to calculate adaptive write buffer size (#62104)
add 9c226f5bf0f [fix](regression) isolate ann_index_p0 table names (#62178)
add ebf08f296db [fix](doris catalog) FragmentMgr should not cancel virtual
doris cluster query (#62135)
add e41a7fbe85e [chore](fe) clean some code (#62171)
add 1cf74d131ef [fix](paimon) add jindo oss support and token propagation
(#62106)
add 6ea8fa5d44e [fix](pipeline) Prevent wake_up() from submitting
finalized task to scheduler (#62155)
add 629ab2fcae5 [improvement](be) Optimize array distance functions to
avoid material… (#61969)
add 2f172a943dd [refine](code) Add comments and benchmarks for ColumnView.
(#61678)
add c9c2a5c1353 [feature](RoutineLoad) Support RoutineLoad IAM auth
(#61324)
add 06d114b4ce1 [Fix](pyudf) Fix import conflicts for modules with the
same top-level name (#62167)
add ff36db5b6fb [fix](memory) track IO layer read buffers via
MemTrackerLimiter using PODArray (#62032)
add 78da3c9d4e2 [fix](delete) avoid quorum stalls on delete push failures
(#61647)
add bb217873838 [improve](job) Support internal pass-through properties
with "__" prefix in streaming job (#62193)
add 0ccaaf423da [fix](test) Make S3FileWriter boundary test multipart mock
thread-safe (#62149)
add 2ea624eb2e3 [chore](fe-dep)Update aircompressor version to 2.0.3
(#62239)
add d2a9be1cb0d [fix](test) fix unstable test agg_strategy_variable
(#62169)
add 91d2bc2566c [opt](memory) limit max block bytes per batch in the write
path (#61821)
add dc9b25f1ca2 [chore](physcial plan) remove useless call of SlotDesc's
parent (#62233)
add 443ae22a7c1 [Chore](pipeline) make wake up do not return error (#62190)
add 1063e2094c4 [chore](RF) use a mock tuple descriptor to init
CTEScanNode (#62235)
add dded5cd2898 [fix](auditlog) fix AuditLoaderPlugin occasional
occurrence garbled Chinese characters in audit logs (#61902)
add 589b6a551fd [fix](cloud) Avoid recycle the instance which has
successor (#62234)
add b6e5f22ff56 [fix](balance) Fix PartitionRebalancer generating invalid
moves to BEs without required storage medium (#62206)
add eb2567d97ea [opt](nereids) optimize length(str_col) by only read
offset sub column (#62205)
add 61b241ce8a6 [Enhancement](optimizer) Support multi-mode CTE inline
strategy (#60601)
add 2014238a5f6 [fix](fe) Fix SHOW BACKENDS field order mismatch (#62207)
add fa7584cfa36 [opt](mv) simplify get base column names' way (#62228)
add 5d7d362ef88 [refactoring](multi-catalog)data_lake_reader_refactoring.
add 57d1d99b50c 1. Refactor count agg push down. 2. Load query decoupling.
3. unify FE default value.
add e47dc56a8a0 fix.
add bb6cb3730b5 unify reader init.
add 2db5544a82f [refactor](be) Unify all standalone readers to use NVI
init_reader
add 15417ab85b8 [refactor](be) Apply NVI template to get_next_block
add bd9a12f7f96 [refactor](be) Introduce TableFormatReader with auto
column filling
add 9986fb26eba [refactor](be) Move column-filling logic from
GenericReader to TableFormatReader
add dd072c05366 [refactor](be) Move fill-column computation to
on_before_init_reader
add f6e99d6b32a [fix](be) Fix standalone reader init for Parquet and ORC
add 85790ffef86 update.
add fde3a876c39 [refactor](be) Migrate parquet test files to new NVI
init_reader API
add 787fde86daf fix
add 023626897b6 remove _fill_columns_from_path.
add 0fc9862e387 update
add a486bb83080 update.
add 489263cd454 update push down agg.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (562e67709c8)
\
N -- N -- N refs/heads/data_lake_reader_refactoring (489263cd454)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/opencode-review.yml | 7 +-
.gitleaks.toml | 28 +
.licenserc.yaml | 1 +
.rat-excludes | 1 +
be/CMakeLists.txt | 1 -
be/benchmark/benchmark_column_view.hpp | 484 +++++++++++
be/benchmark/benchmark_fastunion.hpp | 2 +-
be/benchmark/benchmark_hll_merge.hpp | 2 +-
be/benchmark/benchmark_main.cpp | 7 +-
be/benchmark/benchmark_string.hpp | 4 +-
be/benchmark/binary_cast_benchmark.hpp | 2 +-
be/src/agent/task_worker_pool.cpp | 8 +-
be/src/cloud/cloud_base_compaction.h | 7 +
be/src/cloud/cloud_compaction_action.cpp | 12 +-
be/src/cloud/cloud_cumulative_compaction.h | 7 +
be/src/cloud/cloud_full_compaction.h | 7 +
be/src/cloud/cloud_storage_engine.cpp | 129 ++-
be/src/cloud/cloud_storage_engine.h | 10 +-
be/src/cloud/pb_convert.cpp | 12 +
be/src/common/config.cpp | 7 +
be/src/common/config.h | 7 +
be/src/common/multi_version.h | 9 +-
be/src/core/column/column.h | 9 +
be/src/core/column/column_array.cpp | 4 -
be/src/core/column/column_execute_util.h | 24 +
be/src/core/column/column_nullable.h | 8 +
be/src/core/column/column_string.h | 23 +
be/src/core/column/column_vector.cpp | 1 -
be/src/core/column/column_vector.h | 7 +-
be/src/core/data_type/data_type.cpp | 2 -
be/src/core/data_type/data_type_number_base.cpp | 3 +-
be/src/core/data_type/data_type_number_base.h | 2 +-
be/src/core/data_type/primitive_type.h | 10 +-
.../data_type_serde/data_type_number_serde.cpp | 13 +-
.../core/data_type_serde/data_type_number_serde.h | 6 +-
be/src/core/field.cpp | 6 -
be/src/exec/operator/hashjoin_build_sink.cpp | 33 +-
be/src/exec/operator/olap_scan_operator.cpp | 84 +-
be/src/exec/operator/operator.h | 1 -
.../partitioned_aggregation_source_operator.cpp | 5 +
.../partitioned_hash_join_probe_operator.cpp | 5 +
be/src/exec/operator/scan_operator.cpp | 41 +-
be/src/exec/operator/table_function_operator.cpp | 210 +++--
be/src/exec/pipeline/dependency.cpp | 2 +-
be/src/exec/pipeline/pipeline_task.cpp | 47 +-
be/src/exec/pipeline/pipeline_task.h | 19 +-
be/src/exec/scan/file_scanner.cpp | 115 +--
be/src/exec/scan/file_scanner.h | 2 -
be/src/exec/scan/scanner.cpp | 11 +
.../aggregate_function_approx_count_distinct.h | 2 +-
.../aggregate/aggregate_function_min_max_impl.h | 1 -
.../aggregate_function_reader_first_last.h | 304 +------
.../aggregate_function_reader_replace.cpp | 262 +++++-
.../aggregate_function_window_funnel_v2.h | 266 +++---
be/src/exprs/create_predicate_function.h | 17 +-
.../exprs/function/array/function_array_distance.h | 173 ++--
be/src/exprs/function/function_search.cpp | 113 ++-
be/src/exprs/function/functions_comparison.h | 1 -
be/src/exprs/function/in.h | 2 +-
be/src/exprs/function/round.h | 8 +-
be/src/exprs/hybrid_set.h | 65 +-
.../exprs/lambda_function/varray_sort_function.cpp | 3 +-
be/src/exprs/table_function/table_function.h | 1 +
be/src/exprs/table_function/vexplode.cpp | 2 +-
be/src/exprs/table_function/vexplode_v2.cpp | 3 +-
be/src/format/csv/csv_reader.cpp | 14 +-
be/src/format/generic_reader.h | 12 +-
be/src/format/json/new_json_reader.cpp | 7 +-
be/src/format/orc/vorc_reader.cpp | 206 +----
be/src/format/orc/vorc_reader.h | 19 +-
be/src/format/parquet/vparquet_reader.cpp | 106 ++-
be/src/format/parquet/vparquet_reader.h | 5 +
be/src/format/table/iceberg_reader_mixin.h | 31 +-
be/src/format/table/paimon_reader.cpp | 2 -
be/src/format/table/transactional_hive_reader.cpp | 1 -
.../schema_compaction_tasks_scanner.cpp | 504 +++++++++++
.../schema_compaction_tasks_scanner.h} | 32 +-
.../information_schema/schema_rowsets_scanner.cpp | 11 +
be/src/information_schema/schema_scanner.cpp | 3 +
be/src/io/fs/buffered_reader.cpp | 22 +-
be/src/io/fs/buffered_reader.h | 4 +-
be/src/io/fs/hdfs_file_system.cpp | 8 +-
be/src/io/fs/http_file_reader.cpp | 14 +-
be/src/io/fs/http_file_reader.h | 3 +-
be/src/load/memtable/memtable.cpp | 10 +-
be/src/load/routine_load/data_consumer.cpp | 43 +-
be/src/load/routine_load/data_consumer.h | 4 +
be/src/runtime/aws_msk_iam_auth.cpp | 517 +++++++++++
be/src/runtime/aws_msk_iam_auth.h | 169 ++++
be/src/runtime/fragment_mgr.cpp | 18 +-
be/src/runtime/query_context.cpp | 2 +
be/src/runtime/query_context.h | 3 +-
be/src/service/http/action/compaction_action.cpp | 51 +-
.../http/action/compaction_profile_action.cpp | 270 ++++++
...health_action.h => compaction_profile_action.h} | 15 +-
be/src/service/http/action/health_action.h | 4 +-
be/src/service/http/action/http_stream.cpp | 11 +-
be/src/service/http/action/http_stream.h | 5 +-
be/src/service/http/action/pad_rowset_action.cpp | 2 +-
be/src/service/http/action/stream_load.cpp | 11 +-
be/src/service/http/action/stream_load.h | 6 +-
be/src/service/http/action/stream_load_2pc.cpp | 6 +-
be/src/service/http/action/stream_load_2pc.h | 5 +-
be/src/service/http/http_handler_with_auth.cpp | 4 +
be/src/service/http_service.cpp | 12 +
be/src/storage/compaction/base_compaction.h | 6 +
be/src/storage/compaction/compaction.cpp | 90 +-
be/src/storage/compaction/compaction.h | 22 +
be/src/storage/compaction/cumulative_compaction.h | 6 +
be/src/storage/compaction/full_compaction.h | 6 +
be/src/storage/compaction_task_tracker.cpp | 271 ++++++
be/src/storage/compaction_task_tracker.h | 203 +++++
be/src/storage/data_dir.cpp | 29 +-
.../storage/index/ann/cmake-protect/CMakeLists.txt | 6 +-
.../inverted/inverted_index_compound_reader.cpp | 2 +-
be/src/storage/merger.cpp | 10 +-
be/src/storage/merger.h | 6 +-
be/src/storage/olap_server.cpp | 71 +-
be/src/storage/predicate/column_predicate.h | 3 +-
be/src/storage/predicate/comparison_predicate.h | 6 -
be/src/storage/predicate/in_list_predicate.h | 67 +-
be/src/storage/predicate/null_predicate.h | 1 -
.../predicate/predicate_creator_in_list_in.cpp | 60 +-
.../predicate/predicate_creator_in_list_not_in.cpp | 60 +-
be/src/storage/rowset/rowset.cpp | 3 +-
be/src/storage/rowset/rowset.h | 5 +-
be/src/storage/rowset/rowset_meta.h | 4 +
be/src/storage/rowset_builder.cpp | 4 +-
be/src/storage/segment/binary_dict_page.cpp | 52 +-
be/src/storage/segment/binary_plain_page.h | 39 +-
be/src/storage/segment/column_reader.cpp | 429 +++++++++-
be/src/storage/segment/column_reader.h | 65 +-
be/src/storage/segment/options.h | 1 +
be/src/storage/segment/segment_writer.cpp | 5 +
be/src/storage/segment/vertical_segment_writer.cpp | 5 +
be/src/storage/storage_engine.h | 14 +-
.../storage/task/engine_publish_version_task.cpp | 65 +-
be/src/storage/task/engine_publish_version_task.h | 37 +-
be/src/storage/txn/txn_manager.cpp | 14 +-
be/src/storage/txn/txn_manager.h | 6 +-
be/src/udf/python/python_server.py | 8 +-
.../exec/operator/analytic_sink_operator_test.cpp | 2 +-
...istinct_streaming_aggregation_operator_test.cpp | 2 +-
.../operator/exchange_source_operator_test.cpp | 2 +-
be/test/exec/operator/hashjoin_build_sink_test.cpp | 262 ++++++
.../operator/partition_sort_sink_operator_test.cpp | 2 +-
...artitioned_aggregation_source_operator_test.cpp | 38 +
.../partitioned_hash_join_probe_operator_test.cpp | 45 +
.../exec/operator/query_cache_operator_test.cpp | 2 +-
be/test/exec/operator/repeat_operator_test.cpp | 2 +-
be/test/exec/operator/set_operator_test.cpp | 4 +-
be/test/exec/operator/sort_operator_test.cpp | 5 +-
.../exec/operator/table_function_operator_test.cpp | 639 +++++++++++++-
be/test/exec/operator/union_operator_test.cpp | 6 +-
be/test/exec/pipeline/dummy_task_queue.h | 9 +-
be/test/exec/pipeline/pipeline_task_test.cpp | 75 +-
be/test/exec/scan/vfile_scanner_exception_test.cpp | 41 +
.../exec/sink/arrow_result_block_buffer_test.cpp | 8 +-
be/test/exec/sink/result_block_buffer_test.cpp | 6 +-
.../exprs/aggregate/vec_window_funnel_v2_test.cpp | 743 ++++++++++++++++
be/test/exprs/function/cast/cast_test.h | 7 +-
be/test/format/parquet/parquet_expr_test.cpp | 2 +-
be/test/io/fs/s3_file_writer_test.cpp | 30 +-
be/test/runtime/aws_msk_iam_auth_test.cpp | 244 ++++++
.../fragment_mgr_cross_cluster_cancel_test.cpp | 156 ++++
.../workload_group/workload_group_manager_test.cpp | 26 +-
be/test/service/http/http_auth_test.cpp | 99 ++-
be/test/storage/compaction_task_tracker_test.cpp | 769 +++++++++++++++++
be/test/storage/segment/column_reader_test.cpp | 40 +-
.../inverted_index_compound_reader_test.cpp | 57 ++
be/test/storage/storage_engine_test.cpp | 28 +-
be/test/storage/txn/txn_manager_test.cpp | 28 +-
be/test/testutil/mock/mock_runtime_state.h | 4 +-
build.sh | 117 +--
cloud/src/meta-service/http_encode_key.cpp | 5 +-
cloud/src/meta-service/meta_service.cpp | 17 +-
cloud/src/meta-service/meta_service_job.cpp | 8 +-
cloud/src/meta-service/meta_service_resource.cpp | 8 +-
.../src/meta-service/meta_service_tablet_stats.cpp | 4 +-
cloud/src/meta-service/meta_service_txn.cpp | 16 +-
cloud/src/meta-store/blob_message.cpp | 9 +-
cloud/src/meta-store/txn_kv.cpp | 2 +-
cloud/src/recycler/checker.cpp | 84 +-
cloud/src/recycler/meta_checker.cpp | 4 +-
cloud/src/recycler/recycler.cpp | 49 +-
cloud/src/recycler/recycler.h | 11 +-
cloud/src/recycler/util.cpp | 4 +-
cloud/src/resource-manager/resource_manager.cpp | 4 +-
cloud/test/bvars_test.cpp | 4 +-
cloud/test/mem_txn_kv_test.cpp | 4 +-
cloud/test/meta_service_test.cpp | 14 +-
cloud/test/recycler_test.cpp | 28 +-
cloud/test/txn_kv_test.cpp | 12 +-
conf/fe.conf | 5 +
docker/runtime/doris-compose/command.py | 14 +-
.../docker-compose/hive/hadoop-hive.env.tpl | 1 +
.../hive/scripts/bootstrap/bootstrap-groups.sh | 167 ++++
.../bootstrap/hive2_only.preinstalled_hql.list | 4 +
.../hive/scripts/bootstrap/hive2_only.run_sh.list | 1 +
.../scripts/bootstrap/hive3_only.download_dir.list | 3 +
.../hive/scripts/bootstrap/hive3_only.run_sh.list | 2 +
.../docker-compose/hive/scripts/hive-metastore.sh | 118 ++-
.../hive/scripts/prepare-hive-data.sh | 140 ++-
docker/thirdparties/docker-health.sh | 108 +++
docker/thirdparties/jindofs-helpers.sh | 116 ++-
docker/thirdparties/run-thirdparties-docker.sh | 30 +
...irdparties-docker-hive-bootstrap-groups-test.sh | 64 ++
.../run-thirdparties-docker-hive-restart-test.sh | 86 ++
fe/.mvn/maven-build-cache-config.xml | 16 +
fe/be-java-extensions/paimon-scanner/pom.xml | 8 +-
.../paimon-scanner/src/main/resources/package.xml | 6 +
fe/be-java-extensions/preload-extensions/pom.xml | 1 -
.../doris/authentication/AuthenticationResult.java | 46 +-
.../doris/authentication/BasicPrincipal.java | 32 +
.../org/apache/doris/authentication/Principal.java | 10 +
.../authentication/AuthenticationResultTest.java | 19 +
.../doris/authentication/BasicPrincipalTest.java | 11 +-
.../fe-authentication-handler/pom.xml | 15 +-
.../handler/AuthenticationOutcome.java | 10 +
.../handler/AuthenticationService.java | 84 +-
.../handler/AuthenticationOutcomeTest.java | 41 +
.../handler/AuthenticationPluginManagerTest.java | 5 +
.../handler/AuthenticationServiceTest.java | 274 +++++-
.../pom.xml | 57 +-
.../DefinitionBackedRoleMappingEvaluator.java | 113 +++
.../IntegrationPropertyRoleMappingEvaluator.java | 200 +++++
.../rolemapping/NoOpRoleMappingEvaluator.java} | 36 +-
.../rolemapping/RoleMappingDefinition.java | 118 +++
.../RoleMappingDefinitionProvider.java} | 11 +-
.../rolemapping/RoleMappingEvaluator.java} | 27 +-
.../rolemapping/UnifiedRoleMappingCelEngine.java | 495 +++++++++++
.../oidc/playground/RoleMappingCelPlayground.java | 152 ++++
.../playground/RoleMappingCelPlaygroundTest.java} | 35 +-
...ntegrationPropertyRoleMappingEvaluatorTest.java | 85 ++
.../UnifiedRoleMappingCelEngineTest.java | 180 ++++
fe/fe-authentication/pom.xml | 1 +
.../apache/doris/catalog/NameSpaceContext.java} | 36 +-
fe/fe-common/pom.xml | 48 ++
.../main/java/org/apache/doris/common/Config.java | 53 ++
.../java/org/apache/doris/common/ErrorCode.java | 2 +
.../java/org/apache/doris/qe/GlobalVariable.java | 107 ++-
.../main/java/org/apache/doris/qe/VarAttrDef.java | 87 ++
fe/fe-core/pom.xml | 137 +--
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 3 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 12 +
.../src/main/java/org/apache/doris/DorisFE.java | 7 +
.../main/java/org/apache/doris/alter/Alter.java | 5 +-
.../doris/alter/MaterializedViewHandler.java | 6 +-
.../apache/doris/alter/SchemaChangeHandler.java | 17 +-
.../java/org/apache/doris/analysis/ColumnDef.java | 8 +-
.../org/apache/doris/analysis/ColumnRefExpr.java | 12 -
.../org/apache/doris/analysis/DateLiteral.java | 218 +----
.../apache/doris/analysis/DateLiteralUtils.java | 282 ++++++
.../org/apache/doris/analysis/DecimalLiteral.java | 53 +-
.../apache/doris/analysis/DecimalLiteralUtils.java | 84 ++
.../main/java/org/apache/doris/analysis/Expr.java | 29 -
.../doris/analysis/ExprToColumnLabelVisitor.java | 51 --
.../doris/analysis/ExprToExprNameVisitor.java | 65 ++
.../doris/analysis/ExprToStringValueVisitor.java | 2 +-
.../apache/doris/analysis/FunctionCallExpr.java | 11 -
.../org/apache/doris/analysis/LiteralExpr.java | 12 +-
.../org/apache/doris/analysis/MVColumnItem.java | 71 +-
.../java/org/apache/doris/analysis/ParseNode.java | 13 -
.../apache/doris/analysis/PartitionExprUtil.java | 2 +-
.../org/apache/doris/analysis/SchemaTableType.java | 4 +-
.../java/org/apache/doris/analysis/Separator.java | 51 +-
.../java/org/apache/doris/analysis/SlotRef.java | 37 -
.../org/apache/doris/analysis/StatementBase.java | 25 +-
.../org/apache/doris/analysis/StorageBackend.java | 42 +-
.../org/apache/doris/analysis/TableSample.java | 16 +-
.../org/apache/doris/analysis/VariableExpr.java | 5 -
.../org/apache/doris/analysis/VirtualSlotRef.java | 11 -
.../AuthenticationIntegrationMgr.java | 31 +-
.../AuthenticationIntegrationRuntime.java | 55 +-
.../doris/authentication/RoleMappingMeta.java | 142 ++++
.../doris/authentication/RoleMappingMgr.java | 279 ++++++
.../org/apache/doris/backup/BackupHandler.java | 14 +-
.../java/org/apache/doris/backup/BackupJob.java | 6 +-
.../java/org/apache/doris/backup/Repository.java | 639 +++++++++-----
.../org/apache/doris/backup/RepositoryMgr.java | 8 +-
.../java/org/apache/doris/backup/RestoreJob.java | 13 +-
.../org/apache/doris/catalog/AliasFunction.java | 12 +-
.../org/apache/doris/catalog/AzureResource.java | 127 ++-
.../apache/doris/catalog/CatalogRecycleBin.java | 3 +-
.../apache/doris/catalog/CloudTabletStatMgr.java | 20 +-
.../main/java/org/apache/doris/catalog/Column.java | 314 +------
.../org/apache/doris/catalog/ColumnToProtobuf.java | 201 +++++
.../org/apache/doris/catalog/ColumnToThrift.java | 173 ++++
.../java/org/apache/doris/catalog/Database.java | 60 +-
.../java/org/apache/doris/catalog/DatabaseIf.java | 27 -
.../main/java/org/apache/doris/catalog/Env.java | 108 ++-
.../org/apache/doris/catalog/HdfsStorageVault.java | 35 +-
.../doris/catalog/LocalTabletInvertedIndex.java | 19 +-
.../main/java/org/apache/doris/catalog/MTMV.java | 3 +
.../doris/catalog/MaterializedIndexMeta.java | 3 +
.../java/org/apache/doris/catalog/OlapTable.java | 29 +-
.../java/org/apache/doris/catalog/Replica.java | 19 -
.../java/org/apache/doris/catalog/S3Resource.java | 130 ++-
.../java/org/apache/doris/catalog/SchemaTable.java | 42 +
.../org/apache/doris/catalog/TableIndexes.java | 9 +-
.../org/apache/doris/catalog/TableProperty.java | 13 +
.../apache/doris/catalog/TabletInvertedIndex.java | 4 +-
.../main/java/org/apache/doris/catalog/View.java | 4 +
.../catalog/constraint/ForeignKeyConstraint.java | 5 +-
.../apache/doris/clone/LoadStatisticForTag.java | 13 +-
.../apache/doris/clone/PartitionRebalancer.java | 2 +-
.../cloud/alter/CloudSchemaChangeHandler.java | 3 +
.../apache/doris/cloud/backup/CloudRestoreJob.java | 4 +-
.../apache/doris/cloud/catalog/CloudReplica.java | 79 --
.../cloud/datasource/CloudInternalCatalog.java | 9 +-
.../apache/doris/cloud/load/CleanCopyJobTask.java | 27 +-
.../java/org/apache/doris/cloud/load/CopyJob.java | 2 +-
.../doris/cloud/load/CopyLoadPendingTask.java | 68 +-
.../org/apache/doris/cloud/stage/StageUtil.java | 74 +-
.../apache/doris/cloud/storage/AzureRemote.java | 336 --------
.../org/apache/doris/cloud/storage/BosRemote.java | 60 --
.../org/apache/doris/cloud/storage/CosRemote.java | 179 ----
.../apache/doris/cloud/storage/DefaultRemote.java | 364 --------
.../org/apache/doris/cloud/storage/MockRemote.java | 110 ---
.../org/apache/doris/cloud/storage/ObjectInfo.java | 144 ++++
.../doris/cloud/storage/ObjectInfoAdapter.java | 201 +++++
.../org/apache/doris/cloud/storage/ObsRemote.java | 104 ---
.../org/apache/doris/cloud/storage/OssRemote.java | 191 -----
.../org/apache/doris/cloud/storage/RemoteBase.java | 272 ------
.../org/apache/doris/cloud/storage/S3Remote.java | 105 ---
.../org/apache/doris/cluster/ClusterGuard.java | 65 ++
.../ClusterGuardException.java} | 17 +-
.../apache/doris/cluster/ClusterGuardFactory.java | 88 ++
.../NoOpClusterGuard.java} | 34 +-
.../doris/common/plugin/CloudPluginDownloader.java | 44 +-
.../apache/doris/common/proc/BackendsProcDir.java | 6 +-
.../doris/common/proc/BuildIndexProcDir.java | 3 +-
.../doris/common/proc/PartitionsProcDir.java | 3 +-
.../apache/doris/common/proc/RollupProcDir.java | 3 +-
.../doris/common/proc/SchemaChangeProcDir.java | 3 +-
.../org/apache/doris/common/util/BrokerReader.java | 19 +-
.../org/apache/doris/common/util/BrokerUtil.java | 274 ------
.../org/apache/doris/common/util/LocationPath.java | 2 +-
.../apache/doris/common/util/PropertyAnalyzer.java | 31 +-
.../org/apache/doris/datasource/CatalogIf.java | 3 -
.../apache/doris/datasource/ExternalCatalog.java | 10 +-
.../apache/doris/datasource/ExternalDatabase.java | 5 -
.../org/apache/doris/datasource/FileGroupInfo.java | 5 +-
.../doris/datasource/FilePartitionUtils.java | 126 +++
.../apache/doris/datasource/FileQueryScanNode.java | 3 +-
.../apache/doris/datasource/InternalCatalog.java | 21 +-
.../org/apache/doris/datasource/hive/AcidUtil.java | 98 +--
.../doris/datasource/hive/HMSExternalCatalog.java | 22 +-
.../doris/datasource/hive/HMSTransaction.java | 277 +++---
.../datasource/hive/HiveExternalMetaCache.java | 62 +-
.../org/apache/doris/datasource/hive/HiveUtil.java | 9 +-
.../doris/datasource/iceberg/IcebergUtils.java | 19 +-
.../datasource/iceberg/fileio/DelegateFileIO.java | 62 +-
.../iceberg/fileio/DelegateInputFile.java | 4 +-
.../iceberg/fileio/DelegateOutputFile.java | 32 +-
.../fileio/DelegateSeekableInputStream.java | 4 +-
.../doris/datasource/jdbc/JdbcExternalCatalog.java | 7 +-
.../datasource/jdbc/util/JdbcFieldSchema.java | 5 +-
.../maxcompute/source/MaxComputeScanNode.java | 10 +-
.../paimon/source/PaimonPredicateConverter.java | 3 +-
.../storage/AbstractS3CompatibleProperties.java | 9 +
.../property/storage/AzureProperties.java | 8 +
.../datasource/property/storage/COSProperties.java | 9 +
.../datasource/property/storage/OBSProperties.java | 9 +
.../datasource/property/storage/OSSProperties.java | 3 +
.../source/TrinoConnectorPredicateConverter.java | 2 +-
.../java/org/apache/doris/fs/DirectoryLister.java | 7 +-
.../main/java/org/apache/doris/fs/FileSystem.java | 147 ----
.../java/org/apache/doris/fs/FileSystemCache.java | 33 +-
.../org/apache/doris/fs/FileSystemDescriptor.java | 105 +++
.../apache/doris/fs/FileSystemDirectoryLister.java | 23 +-
.../org/apache/doris/fs/FileSystemFactory.java | 193 ++++-
.../org/apache/doris/fs/FileSystemIOException.java | 65 --
.../apache/doris/fs/FileSystemPluginManager.java | 146 ++++
.../apache/doris/fs/FileSystemProviderImpl.java | 42 -
.../org/apache/doris/fs/FsStorageTypeAdapter.java | 59 ++
.../org/apache/doris/fs/LocalDfsFileSystem.java | 199 -----
.../org/apache/doris/fs/PersistentFileSystem.java | 57 --
.../apache/doris/fs/RemoteFileRemoteIterator.java | 47 -
.../java/org/apache/doris/fs/SchemaTypeMapper.java | 6 +
.../apache/doris/fs/SpiSwitchingFileSystem.java | 206 +++++
.../doris/fs/StoragePropertiesConverter.java | 92 ++
.../org/apache/doris/fs/StorageTypeMapper.java | 68 --
.../fs/TransactionScopeCachingDirectoryLister.java | 35 +-
.../java/org/apache/doris/fs/io/DorisInput.java | 54 --
.../org/apache/doris/fs/io/DorisInputFile.java | 75 --
.../org/apache/doris/fs/io/DorisInputStream.java | 51 --
.../org/apache/doris/fs/io/DorisOutputFile.java | 59 --
.../org/apache/doris/fs/io/hdfs/HdfsInput.java | 107 ---
.../org/apache/doris/fs/io/hdfs/HdfsInputFile.java | 155 ----
.../apache/doris/fs/io/hdfs/HdfsInputStream.java | 180 ----
.../apache/doris/fs/io/hdfs/HdfsOutputFile.java | 95 ---
.../apache/doris/fs/io/hdfs/HdfsOutputStream.java | 135 ---
.../org/apache/doris/fs/obj/AzureObjStorage.java | 626 --------------
.../java/org/apache/doris/fs/obj/ObjStorage.java | 70 --
.../java/org/apache/doris/fs/obj/S3ObjStorage.java | 862 -------------------
.../doris/fs/operations/BrokerFileOperations.java | 163 ----
.../apache/doris/fs/operations/BrokerOpParams.java | 57 --
.../doris/fs/operations/HDFSFileOperations.java | 135 ---
.../apache/doris/fs/operations/HDFSOpParams.java | 72 --
.../org/apache/doris/fs/operations/OpParams.java | 54 --
.../apache/doris/fs/remote/AzureFileSystem.java | 89 --
.../apache/doris/fs/remote/BrokerFileSystem.java | 712 ----------------
.../org/apache/doris/fs/remote/ObjFileSystem.java | 187 ----
.../org/apache/doris/fs/remote/RemoteFile.java | 117 ---
.../apache/doris/fs/remote/RemoteFileSystem.java | 72 --
.../org/apache/doris/fs/remote/S3FileSystem.java | 143 ----
.../doris/fs/remote/SwitchingFileSystem.java | 131 ---
.../apache/doris/fs/remote/dfs/DFSFileSystem.java | 597 -------------
.../remote/dfs/DFSFileSystemPhantomReference.java | 44 -
.../apache/doris/fs/remote/dfs/OFSFileSystem.java | 27 -
.../fs/remote/dfs/RemoteFSPhantomManager.java | 127 ---
.../doris/httpv2/controller/BaseController.java | 1 +
.../apache/doris/httpv2/rest/BackendsAction.java | 4 -
.../doris/httpv2/rest/BootstrapFinishAction.java | 4 +-
.../doris/httpv2/rest/CheckDecommissionAction.java | 6 +-
.../doris/httpv2/rest/CheckWalSizeAction.java | 6 +-
.../apache/doris/httpv2/rest/ConnectionAction.java | 6 +-
.../apache/doris/httpv2/rest/CopyIntoAction.java | 15 +-
.../apache/doris/httpv2/rest/DebugPointAction.java | 15 +-
.../doris/httpv2/rest/ExtraBasepathAction.java | 4 +-
.../doris/httpv2/rest/FeVersionInfoAction.java | 7 +-
.../apache/doris/httpv2/rest/GetLogFileAction.java | 7 +-
.../doris/httpv2/rest/GetSmallFileAction.java | 7 +-
.../org/apache/doris/httpv2/rest/HealthAction.java | 6 +-
.../apache/doris/httpv2/rest/MetaInfoAction.java | 16 +-
.../doris/httpv2/rest/MetaReplayerCheckAction.java | 7 +-
.../apache/doris/httpv2/rest/MetricsAction.java | 5 +-
.../apache/doris/httpv2/rest/MinidumpAction.java | 7 +-
.../apache/doris/httpv2/rest/ProfileAction.java | 11 +-
.../apache/doris/httpv2/rest/QueryStatsAction.java | 26 +-
.../doris/httpv2/rest/RestBaseController.java | 18 +
.../apache/doris/httpv2/rest/RowCountAction.java | 7 +-
.../apache/doris/httpv2/rest/SetConfigAction.java | 6 +-
.../doris/httpv2/rest/StorageTypeCheckAction.java | 7 +-
.../org/apache/doris/httpv2/rest/TSOAction.java | 97 +++
.../doris/httpv2/rest/manager/ClusterAction.java | 11 +-
.../doris/httpv2/rest/manager/HttpUtils.java | 8 +-
.../doris/httpv2/rest/manager/NodeAction.java | 13 +-
.../httpv2/rest/manager/QueryProfileAction.java | 9 +-
.../doris/httpv2/restv2/ClusterGuardAction.java | 161 ++++
.../apache/doris/httpv2/restv2/ImportAction.java | 30 +-
.../java/org/apache/doris/info/TableNameInfo.java | 75 +-
.../org/apache/doris/info/TableNameInfoUtils.java | 104 +++
.../java/org/apache/doris/info/TableRefInfo.java | 4 +-
.../insert/streaming/StreamingInsertJob.java | 9 +-
.../insert/streaming/StreamingJobProperties.java | 7 +
.../job/offset/s3/S3SourceOffsetProvider.java | 51 +-
.../org/apache/doris/journal/JournalEntity.java | 18 +
.../apache/doris/journal/local/LocalJournal.java | 8 +
.../main/java/org/apache/doris/load/DeleteJob.java | 21 +-
.../main/java/org/apache/doris/load/ExportJob.java | 3 +-
.../main/java/org/apache/doris/load/ExportMgr.java | 12 +-
.../doris/load/loadv2/BrokerLoadPendingTask.java | 28 +-
.../load/routineload/kafka/KafkaConfiguration.java | 15 +
.../kafka/KafkaDataSourceProperties.java | 80 +-
.../java/org/apache/doris/master/Checkpoint.java | 2 -
.../java/org/apache/doris/metric/MetricRepo.java | 28 +
.../org/apache/doris/mtmv/MTMVPartitionUtil.java | 9 +-
.../java/org/apache/doris/mtmv/MTMVService.java | 12 +-
.../org/apache/doris/mysql/AcceptListener.java | 1 +
.../org/apache/doris/mysql/MysqlAuthPacket.java | 2 +-
.../org/apache/doris/mysql/MysqlCapability.java | 12 +-
.../java/org/apache/doris/mysql/MysqlProto.java | 29 +-
.../mysql/authenticate/AuthenticateResponse.java | 66 +-
.../authenticate/AuthenticationFailureSummary.java | 101 +++
.../mysql/authenticate/AuthenticatorManager.java | 167 +++-
.../MysqlAuthPacketCredentialExtractor.java | 90 ++
.../AuthenticationIntegrationAuthenticator.java | 66 +-
.../password/AuthPacketAwarePasswordResolver.java | 62 ++
.../password/NativePasswordResolver.java | 5 +-
.../plugin/AuthenticationPluginAuthenticator.java | 62 +-
.../org/apache/doris/mysql/privilege/Auth.java | 14 +-
.../apache/doris/mysql/privilege/RoleManager.java | 2 +-
.../doris/mysql/privilege/TablePrivEntry.java | 4 +-
.../doris/mysql/privilege/UserPropertyMgr.java | 14 +
.../org/apache/doris/nereids/StatementContext.java | 34 +-
.../doris/nereids/analyzer/UnboundAlias.java | 2 +-
.../org/apache/doris/nereids/cost/CostModel.java | 69 ++
.../glue/translator/PhysicalPlanTranslator.java | 117 +--
.../glue/translator/PlanTranslatorContext.java | 15 +-
.../doris/nereids/jobs/executor/Optimizer.java | 114 ++-
.../doris/nereids/load/NereidsFileGroupInfo.java | 6 +-
.../nereids/load/NereidsLoadPlanInfoCollector.java | 4 +-
.../doris/nereids/minidump/MinidumpUtils.java | 4 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 36 +
.../rules/expression/ExpressionOptimization.java | 4 +-
.../rules/expression/ExpressionRuleType.java | 1 +
.../expression/rules/StringEmptyToLengthRule.java | 113 +++
.../rules/implementation/AggregateStrategies.java | 14 +-
.../rewrite/AccessPathExpressionCollector.java | 69 ++
.../nereids/rules/rewrite/AccessPathInfo.java | 3 +
.../rules/rewrite/AccessPathPlanCollector.java | 14 +-
.../doris/nereids/rules/rewrite/CTEInline.java | 13 +-
.../doris/nereids/rules/rewrite/CTEInliner.java | 166 ++++
.../nereids/rules/rewrite/ForeignKeyContext.java | 5 +-
.../nereids/rules/rewrite/NestedColumnPruning.java | 215 ++++-
.../rules/rewrite/PruneOlapScanPartition.java | 2 +-
.../nereids/rules/rewrite/RewriteCteChildren.java | 2 +-
.../nereids/rules/rewrite/SlotTypeReplacer.java | 3 +-
.../doris/nereids/stats/FilterEstimation.java | 3 +-
.../apache/doris/nereids/trees/plans/PlanType.java | 2 +
.../trees/plans/commands/AddConstraintCommand.java | 9 +-
.../commands/AdminCancelRepairTableCommand.java | 2 +-
.../plans/commands/AdminCompactTableCommand.java | 2 +-
.../plans/commands/AdminRepairTableCommand.java | 2 +-
.../commands/AdminSetPartitionVersionCommand.java | 2 +-
.../plans/commands/AdminSetTableStatusCommand.java | 2 +-
.../plans/commands/AlterColumnStatsCommand.java | 2 +-
.../trees/plans/commands/AlterTableCommand.java | 2 +-
.../plans/commands/AlterTableStatsCommand.java | 2 +-
.../trees/plans/commands/AnalyzeTableCommand.java | 2 +-
.../plans/commands/CancelAlterTableCommand.java | 2 +-
.../plans/commands/CancelBuildIndexCommand.java | 2 +-
.../plans/commands/CleanQueryStatsCommand.java | 2 +-
.../commands/CreateMaterializedViewCommand.java | 6 +-
.../trees/plans/commands/CreatePolicyCommand.java | 2 +-
.../plans/commands/CreateRoleMappingCommand.java | 122 +++
.../trees/plans/commands/CreateStageCommand.java | 23 +-
.../trees/plans/commands/DeleteFromCommand.java | 3 +-
.../trees/plans/commands/DescribeCommand.java | 2 +-
.../plans/commands/DropCachedStatsCommand.java | 2 +-
.../plans/commands/DropConstraintCommand.java | 4 +-
.../commands/DropMaterializedViewCommand.java | 2 +-
.../plans/commands/DropRoleMappingCommand.java | 64 ++
.../trees/plans/commands/DropRowPolicyCommand.java | 2 +-
.../trees/plans/commands/DropStatsCommand.java | 2 +-
.../trees/plans/commands/DropStreamCommand.java | 2 +-
.../trees/plans/commands/DropTableCommand.java | 2 +-
.../trees/plans/commands/DropViewCommand.java | 2 +-
.../trees/plans/commands/ExecuteActionCommand.java | 2 +-
.../trees/plans/commands/ExplainCommand.java | 2 +-
.../trees/plans/commands/LockTablesCommand.java | 2 +-
.../plans/commands/RecoverPartitionCommand.java | 2 +-
.../trees/plans/commands/RecoverTableCommand.java | 2 +-
.../trees/plans/commands/RestoreCommand.java | 12 +-
.../trees/plans/commands/ShowAnalyzeCommand.java | 2 +-
.../commands/ShowColumnHistogramStatsCommand.java | 2 +-
.../plans/commands/ShowColumnStatsCommand.java | 2 +-
.../trees/plans/commands/ShowColumnsCommand.java | 2 +-
.../plans/commands/ShowConstraintsCommand.java | 4 +-
.../ShowCreateMaterializedViewCommand.java | 2 +-
.../plans/commands/ShowCreateStreamCommand.java | 2 +-
.../plans/commands/ShowCreateTableCommand.java | 2 +-
.../plans/commands/ShowCreateViewCommand.java | 2 +-
.../trees/plans/commands/ShowDataCommand.java | 2 +-
.../trees/plans/commands/ShowDataSkewCommand.java | 4 +-
.../plans/commands/ShowIndexStatsCommand.java | 2 +-
.../plans/commands/ShowPartitionsCommand.java | 4 +-
.../plans/commands/ShowQueryStatsCommand.java | 2 +-
.../commands/ShowQueuedAnalyzeJobsCommand.java | 2 +-
.../commands/ShowReplicaDistributionCommand.java | 2 +-
.../plans/commands/ShowReplicaStatusCommand.java | 2 +-
.../plans/commands/ShowTableStatsCommand.java | 2 +-
.../commands/ShowTabletsFromTableCommand.java | 2 +-
.../trees/plans/commands/ShowViewCommand.java | 4 +-
.../trees/plans/commands/TruncateTableCommand.java | 2 +-
.../trees/plans/commands/UnsetVariableCommand.java | 3 +-
.../trees/plans/commands/WarmUpClusterCommand.java | 2 +-
.../trees/plans/commands/info/AlterMTMVInfo.java | 2 +-
.../trees/plans/commands/info/AlterViewInfo.java | 2 +-
.../trees/plans/commands/info/BuildIndexOp.java | 2 +-
.../plans/commands/info/CancelMTMVTaskInfo.java | 2 +-
.../trees/plans/commands/info/CopyIntoInfo.java | 17 +-
.../trees/plans/commands/info/CreateIndexOp.java | 2 +-
.../trees/plans/commands/info/CreateMTMVInfo.java | 2 +-
.../plans/commands/info/CreateStreamInfo.java | 4 +-
.../plans/commands/info/CreateTableLikeInfo.java | 4 +-
.../trees/plans/commands/info/CreateViewInfo.java | 2 +-
.../trees/plans/commands/info/DropIndexOp.java | 2 +-
.../trees/plans/commands/info/DropMTMVInfo.java | 2 +-
.../trees/plans/commands/info/LabelNameInfo.java | 4 +-
.../commands/info/ModifyTablePropertiesOp.java | 20 +-
.../trees/plans/commands/info/PauseMTMVInfo.java | 2 +-
.../trees/plans/commands/info/RefreshMTMVInfo.java | 2 +-
.../trees/plans/commands/info/ResumeMTMVInfo.java | 2 +-
.../plans/commands/info/ShowCreateMTMVInfo.java | 2 +-
.../commands/insert/InsertIntoTVFCommand.java | 15 +-
.../commands/refresh/RefreshTableCommand.java | 2 +-
.../worker/job/UnassignedJobBuilder.java | 16 +-
.../worker/job/UnassignedLocalTVFSinkJob.java | 68 ++
.../plans/logical/LogicalCatalogRelation.java | 3 +-
.../plans/physical/PhysicalCatalogRelation.java | 3 +-
.../trees/plans/visitor/CommandVisitor.java | 10 +
.../apache/doris/nereids/types/VariantField.java | 20 +-
.../apache/doris/nereids/util/ExpressionUtils.java | 11 +
.../doris/persist/DropRoleMappingOperationLog.java | 58 ++
.../java/org/apache/doris/persist/EditLog.java | 28 +
.../apache/doris/persist/GlobalVarPersistInfo.java | 8 +-
.../org/apache/doris/persist/OperationType.java | 4 +
.../org/apache/doris/persist/gson/GsonUtils.java | 23 +-
.../doris/persist/meta/MetaPersistMethod.java | 29 +-
.../doris/persist/meta/PersistMetaModules.java | 4 +-
.../planner/BackendPartitionedSchemaScanNode.java | 1 +
.../java/org/apache/doris/planner/CTEScanNode.java | 5 +-
.../apache/doris/planner/MaterializationNode.java | 3 +-
.../org/apache/doris/planner/OlapScanNode.java | 5 +-
.../org/apache/doris/planner/OlapTableSink.java | 5 +-
.../org/apache/doris/planner/TVFTableSink.java | 3 +-
.../java/org/apache/doris/qe/AuditLogHelper.java | 154 ++--
.../java/org/apache/doris/qe/ConnectContext.java | 63 +-
.../java/org/apache/doris/qe/ConnectProcessor.java | 12 +-
.../java/org/apache/doris/qe/FEOpExecutor.java | 5 +
.../org/apache/doris/qe/MysqlConnectProcessor.java | 3 +
.../java/org/apache/doris/qe/SessionVariable.java | 945 +++++++++++----------
.../java/org/apache/doris/qe/StmtExecutor.java | 11 +-
.../main/java/org/apache/doris/qe/VariableMgr.java | 96 +--
.../apache/doris/statistics/AnalysisManager.java | 5 +-
.../doris/statistics/StatisticsJobAppender.java | 9 +-
.../doris/statistics/util/StatisticsUtil.java | 185 +---
.../org/apache/doris/system/SystemInfoService.java | 20 +
.../ExternalFileTableValuedFunction.java | 33 +-
.../ParquetMetadataTableValuedFunction.java | 14 +-
.../apache/doris/task/AlterInvertedIndexTask.java | 3 +-
.../org/apache/doris/task/AlterReplicaTask.java | 3 +-
.../org/apache/doris/task/CreateReplicaTask.java | 3 +-
.../main/java/org/apache/doris/task/PushTask.java | 6 +
.../doris/transaction/DatabaseTransactionMgr.java | 70 +-
.../doris/transaction/HiveTransactionManager.java | 10 +-
.../apache/doris/transaction/TableCommitInfo.java | 17 +-
.../transaction/TransactionManagerFactory.java | 6 +-
.../apache/doris/transaction/TransactionState.java | 11 +
.../main/java/org/apache/doris/tso/TSOService.java | 526 ++++++++++++
.../java/org/apache/doris/tso/TSOTimestamp.java | 247 ++++++
.../doris/alter/MaterializedViewHandlerTest.java | 2 -
.../org/apache/doris/analysis/DateLiteralTest.java | 99 ---
.../doris/analysis/DateLiteralUtilsTest.java | 427 ++++++++++
.../doris/analysis/DecimalLiteralUtilsTest.java | 277 ++++++
.../org/apache/doris/analysis/ExprToSqlTest.java | 37 +-
.../analysis/ExprToStringValueVisitorTest.java | 10 +-
.../AuthenticationIntegrationMgrTest.java | 52 +-
.../AuthenticationIntegrationRuntimeTest.java | 133 ++-
.../doris/authentication/RoleMappingMgrTest.java | 238 ++++++
.../org/apache/doris/backup/BackupJobTest.java | 11 +-
.../org/apache/doris/backup/BrokerStorageTest.java | 192 -----
.../org/apache/doris/backup/RepositoryTest.java | 498 +++++++++--
.../org/apache/doris/backup/RestoreJobTest.java | 3 +-
.../doris/catalog/SessionVariablesNullFixTest.java | 127 +++
.../org/apache/doris/catalog/UserPropertyTest.java | 30 +
.../java/org/apache/doris/clone/RebalanceTest.java | 81 ++
.../org/apache/doris/clone/RebalancerTestUtil.java | 16 +
.../doris/cloud/backup/CloudRestoreJobTest.java | 3 +-
.../org/apache/doris/cloud/load/CopyJobTest.java | 2 +-
.../doris/cloud/load/CopyLoadPendingTaskTest.java | 226 +++--
.../apache/doris/cloud/stage/StageUtilTest.java | 100 ++-
.../doris/cluster/ClusterGuardExceptionTest.java | 48 ++
.../doris/cluster/ClusterGuardFactoryTest.java | 392 +++++++++
.../apache/doris/cluster/NoOpClusterGuardTest.java | 55 ++
.../apache/doris/common/PropertyAnalyzerTest.java | 3 +-
.../doris/common/proc/BackendsProcDirTest.java | 38 +
.../apache/doris/common/util/BrokerUtilTest.java | 116 +--
.../apache/doris/common/util/LocationPathTest.java | 2 +-
.../apache/doris/common/util/TimeUtilsTest.java | 4 +-
.../apache/doris/datasource/PathVisibleTest.java | 33 +-
.../datasource/hive/HMSTransactionPathTest.java | 208 +++--
.../apache/doris/datasource/hive/HiveAcidTest.java | 255 +++---
.../doris/datasource/hive/HmsCommitTest.java | 110 +--
.../datasource/iceberg/IcebergPredicateTest.java | 7 +-
.../datasource/jdbc/source/JdbcScanNodeTest.java | 24 +-
.../datasource/jdbc/util/JdbcFieldSchemaTest.java | 74 ++
.../datasource/lakesoul/LakeSoulPredicateTest.java | 7 +-
.../metastore/PaimonDlfRestCatalogTest.java | 42 -
.../property/storage/OSSPropertiesTest.java | 2 +
.../TrinoConnectorPredicateTest.java | 15 +-
.../external/iceberg/IcebergHadoopCatalogTest.java | 90 --
.../doris/fs/FileSystemTransferUtilTest.java | 218 +++++
.../java/org/apache/doris/fs/MemoryFileSystem.java | 355 ++++++++
.../org/apache/doris/fs/MemoryFileSystemTest.java | 373 ++++++++
.../org/apache/doris/fs/SchemaTypeMapperTest.java | 1 +
.../doris/fs/SpiSwitchingFileSystemTest.java | 285 +++++++
...TransactionScopeCachingDirectoryListerTest.java | 52 +-
.../apache/doris/fs/obj/AzureObjStorageTest.java | 272 ------
.../org/apache/doris/fs/obj/MockedS3Client.java | 131 ---
.../org/apache/doris/fs/obj/S3FileSystemTest.java | 246 ------
.../doris/fs/obj/S3ObjStorageGlobListTest.java | 187 ----
.../org/apache/doris/fs/obj/S3ObjStorageTest.java | 557 ------------
.../doris/fs/remote/RemoteFileSystemTest.java | 195 -----
.../org/apache/doris/http/ForwardToMasterTest.java | 30 +-
.../apache/doris/httpv2/rest/HttpApiAuthTest.java | 158 ++++
.../apache/doris/info/TableNameInfoUtilsTest.java | 181 ++++
.../java/org/apache/doris/load/DeleteJobTest.java | 157 ++++
.../load/loadv2/BrokerLoadPendingTaskTest.java | 84 +-
.../load/routineload/KafkaAwsMskIamAuthTest.java | 491 +++++++++++
.../load/routineload/RoutineLoadManagerTest.java | 6 -
.../doris/master/MasterImplDeleteTaskTest.java | 152 ++++
.../java/org/apache/doris/mtmv/MTMVUtilTest.java | 2 +-
.../apache/doris/mysql/MysqlAuthPacketTest.java | 26 +-
.../apache/doris/mysql/MysqlCapabilityTest.java | 5 +-
.../doris/mysql/MysqlHandshakePacketTest.java | 3 +
.../org/apache/doris/mysql/MysqlProtoTest.java | 118 +++
.../authenticate/AuthenticatorManagerTest.java | 463 +++++++++-
.../MysqlAuthPacketCredentialExtractorTest.java | 90 ++
...AuthenticationIntegrationAuthenticatorTest.java | 26 +-
.../password/ClearPasswordResolverTest.java | 63 ++
.../AuthenticationPluginAuthenticatorTest.java | 156 ++++
.../org/apache/doris/mysql/privilege/AuthTest.java | 23 +
.../nereids/parser/NereidsParserDigestTest.java | 8 +-
.../nereids/parser/RoleMappingParserTest.java | 77 ++
.../rules/StringEmptyToLengthRuleTest.java | 141 +++
.../rules/rewrite/PruneNestedColumnTest.java | 96 +++
.../doris/nereids/trees/plans/ConstraintTest.java | 37 +-
.../trees/plans/CreateTableCommandTest.java | 21 +-
.../commands/CreateRepositoryCommandTest.java | 6 +-
.../trees/plans/commands/RestoreCommandTest.java | 16 +
.../plans/commands/ShowBackendsCommandTest.java | 34 +-
.../org/apache/doris/qe/AuditLogHelperTest.java | 104 +++
.../doris/qe/ConnectAttributesForwardTest.java | 128 +++
.../org/apache/doris/qe/ConnectContextTest.java | 43 +
.../org/apache/doris/qe/SessionVariablesTest.java | 2 +-
.../doris/statistics/util/StatisticsUtilTest.java | 30 -
.../transaction/DatabaseTransactionMgrTest.java | 120 +++
.../java/org/apache/doris/tso/TSOServiceTest.java | 563 ++++++++++++
.../org/apache/doris/tso/TSOTimestampTest.java | 154 ++++
.../doris/utframe/DemoMultiBackendsTest.java | 2 +-
.../extension/loader/ChildFirstClassLoader.java | 28 +
.../loader/DirectoryPluginRuntimeManager.java | 104 ++-
fe/fe-filesystem/ENV_TEST_README.md | 189 +++++
fe/fe-filesystem/fe-filesystem-api/pom.xml | 73 ++
.../org/apache/doris/filesystem/BlockInfo.java} | 34 +-
.../apache/doris/filesystem/DorisInputFile.java | 75 ++
.../apache/doris/filesystem/DorisInputStream.java | 46 +
.../apache/doris/filesystem/DorisOutputFile.java} | 22 +-
.../org/apache/doris/filesystem/FileEntry.java | 103 +++
.../org/apache/doris/filesystem/FileIterator.java} | 19 +-
.../org/apache/doris/filesystem/FileSystem.java | 180 ++++
.../doris/filesystem/FileSystemIOException.java} | 18 +-
.../doris/filesystem/FileSystemTransferUtil.java | 176 ++++
.../apache/doris/filesystem}/FileSystemType.java | 2 +-
.../apache/doris/filesystem}/FileSystemUtil.java | 47 +-
.../org/apache/doris/filesystem/GlobListing.java} | 51 +-
.../java/org/apache/doris/filesystem/Location.java | 140 +++
.../apache/doris/filesystem}/RemoteIterator.java | 2 +-
.../doris/filesystem}/SimpleRemoteIterator.java | 19 +-
.../org/apache/doris/filesystem/BlockInfoTest.java | 63 ++
.../org/apache/doris/filesystem/EnvTestBase.java | 100 +++
.../org/apache/doris/filesystem/FileEntryTest.java | 101 +++
.../filesystem/FileSystemDefaultMethodsTest.java | 223 +++++
.../filesystem/FileSystemTransferUtilTest.java | 85 ++
.../doris/filesystem/FileSystemUtilTest.java | 133 +++
.../apache/doris/filesystem/GlobListingTest.java | 65 ++
.../org/apache/doris/filesystem/LocationTest.java | 143 ++++
.../doris/filesystem/SimpleRemoteIteratorTest.java | 60 ++
.../fe-filesystem-azure}/pom.xml | 72 +-
.../src/main/assembly/plugin-zip.xml | 65 ++
.../doris/filesystem/azure/AzureFileSystem.java | 396 +++++++++
.../filesystem/azure/AzureFileSystemProvider.java | 60 ++
.../doris/filesystem/azure/AzureObjStorage.java | 531 ++++++++++++
.../apache/doris/filesystem/azure/AzureUri.java | 146 ++++
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../filesystem/azure/AzureFileSystemEnvTest.java | 211 +++++
.../azure/AzureObjStorageExtensionTest.java | 361 ++++++++
.../doris/filesystem/azure/AzureUriTest.java | 120 +++
fe/fe-filesystem/fe-filesystem-broker/pom.xml | 110 +++
.../src/main/assembly/plugin-zip.xml | 65 ++
.../filesystem/broker/BrokerClientFactory.java | 76 ++
.../doris/filesystem/broker/BrokerClientPool.java | 84 ++
.../broker/BrokerFileSystemProvider.java | 85 ++
.../doris/filesystem/broker/BrokerInputFile.java | 104 +++
.../doris/filesystem/broker/BrokerInputStream.java | 170 ++++
.../doris/filesystem/broker/BrokerOutputFile.java | 95 +++
.../filesystem/broker/BrokerOutputStream.java | 113 +++
.../filesystem/broker/BrokerSpiFileSystem.java | 258 ++++++
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../broker/BrokerFileSystemProviderTest.java | 149 ++++
.../broker/BrokerSpiFileSystemEnvTest.java | 176 ++++
.../filesystem/broker/BrokerSpiFileSystemTest.java | 268 ++++++
.../fe-filesystem-cos}/pom.xml | 67 +-
.../src/main/assembly/plugin-zip.xml | 65 ++
.../filesystem/cos/CosFileSystemProvider.java | 72 ++
.../apache/doris/filesystem/cos/CosObjStorage.java | 222 +++++
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../doris/filesystem/cos/CosFileSystemEnvTest.java | 207 +++++
.../doris/filesystem/cos/CosObjStorageTest.java | 207 +++++
.../fe-filesystem-hdfs}/pom.xml | 75 +-
.../src/main/assembly/plugin-zip.xml | 65 ++
.../doris/filesystem/hdfs/DFSFileSystem.java | 276 ++++++
.../doris/filesystem/hdfs/HdfsConfigBuilder.java | 58 ++
.../doris/filesystem/hdfs/HdfsFileIterator.java | 62 ++
.../filesystem/hdfs/HdfsFileSystemProvider.java | 66 ++
.../doris/filesystem/hdfs/HdfsInputFile.java | 73 ++
.../doris/filesystem/hdfs/HdfsOutputFile.java | 60 ++
.../filesystem/hdfs/HdfsSeekableInputStream.java | 94 ++
.../hdfs/KerberosHadoopAuthenticator.java | 65 ++
.../filesystem/hdfs/SimpleHadoopAuthenticator.java | 63 ++
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../filesystem/hdfs/DFSFileSystemEnvTest.java | 215 +++++
.../doris/filesystem/hdfs/DFSFileSystemTest.java | 125 +++
.../filesystem/hdfs/HdfsConfigBuilderTest.java | 85 ++
.../hdfs/HdfsSeekableInputStreamTest.java | 191 +++++
.../hdfs/KerberosHadoopAuthenticatorEnvTest.java | 115 +++
.../hdfs/SimpleHadoopAuthenticatorTest.java | 70 ++
.../fe-filesystem-local}/pom.xml | 62 +-
.../src/main/assembly/plugin-zip.xml | 65 ++
.../doris/filesystem/local/LocalFileSystem.java | 248 ++++++
.../filesystem/local/LocalFileSystemProvider.java} | 29 +-
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../filesystem/local/LocalFileSystemTest.java | 221 +++++
.../fe-filesystem-obs}/pom.xml | 67 +-
.../src/main/assembly/plugin-zip.xml | 65 ++
.../filesystem/obs/ObsFileSystemProvider.java | 72 ++
.../apache/doris/filesystem/obs/ObsObjStorage.java | 220 +++++
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../doris/filesystem/obs/ObsFileSystemEnvTest.java | 202 +++++
.../doris/filesystem/obs/ObsObjStorageTest.java | 197 +++++
.../fe-filesystem-oss}/pom.xml | 65 +-
.../src/main/assembly/plugin-zip.xml | 65 ++
.../filesystem/oss/OssFileSystemProvider.java | 72 ++
.../apache/doris/filesystem/oss/OssObjStorage.java | 227 +++++
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../doris/filesystem/oss/OssFileSystemEnvTest.java | 202 +++++
.../doris/filesystem/oss/OssObjStorageTest.java | 195 +++++
fe/fe-filesystem/fe-filesystem-s3/pom.xml | 126 +++
.../src/main/assembly/plugin-zip.xml | 65 ++
.../apache/doris/filesystem/s3/S3FileSystem.java | 538 ++++++++++++
.../doris/filesystem/s3/S3FileSystemProvider.java | 59 ++
.../apache/doris/filesystem/s3/S3ObjStorage.java | 644 ++++++++++++++
.../apache/doris/filesystem/s3/S3OutputStream.java | 93 ++
.../java/org/apache/doris/filesystem/s3/S3Uri.java | 67 ++
....apache.doris.filesystem.spi.FileSystemProvider | 18 +
.../doris/filesystem/s3/S3FileSystemEnvTest.java | 238 ++++++
.../doris/filesystem/s3/S3FileSystemTest.java | 207 +++++
.../doris/filesystem/s3/S3ObjStorageEnvTest.java | 225 +++++
.../doris/filesystem/s3/S3ObjStorageMockTest.java | 377 ++++++++
.../doris/filesystem/s3/S3ObjStorageTest.java | 214 +++++
.../doris/filesystem/s3/S3OutputStreamTest.java | 228 +++++
.../org/apache/doris/filesystem/s3/S3UriTest.java | 76 ++
fe/fe-filesystem/fe-filesystem-spi/pom.xml | 82 ++
.../doris/filesystem/spi/FileSystemProvider.java | 78 ++
.../doris/filesystem/spi/HadoopAuthenticator.java | 45 +
.../apache/doris/filesystem/spi/IOCallable.java} | 22 +-
.../apache/doris/filesystem/spi/ObjFileSystem.java | 124 +++
.../apache/doris/filesystem/spi/ObjStorage.java | 135 +++
.../apache/doris/filesystem/spi}/RemoteObject.java | 19 +-
.../doris/filesystem/spi}/RemoteObjects.java | 26 +-
.../apache/doris/filesystem/spi/RequestBody.java} | 33 +-
.../doris/filesystem/spi/StsCredentials.java} | 35 +-
.../doris/filesystem/spi/UploadPartResult.java} | 27 +-
.../doris/filesystem/spi/ObjFileSystemTest.java | 354 ++++++++
fe/fe-filesystem/pom.xml | 74 ++
.../apache/doris/foundation/fs/FsStorageType.java} | 23 +-
.../org/apache/doris/catalog/PatternType.java} | 13 +-
.../org/apache/doris/catalog/VariantField.java | 11 +-
fe/pom.xml | 39 +-
.../plugin/audit/custom/AuditLoaderPlugin.java | 2 +-
gensrc/proto/olap_file.proto | 5 +
gensrc/script/Makefile | 2 +-
gensrc/script/gen_build_version.sh | 4 +-
gensrc/thrift/AgentService.thrift | 1 +
gensrc/thrift/Descriptors.thrift | 1 +
gensrc/thrift/FrontendService.thrift | 1 +
post-build.sh | 232 +++++
regression-test/conf/regression-conf.groovy | 1 +
.../data/ann_index_p0/ivf_index_test.out | 12 +
.../data/ann_index_p0/ivf_on_disk_index_test.out | 30 +
.../test_rf_in_list_not_erased_by_scope_range.out} | 11 +-
.../external_table_p0/jdbc/test_jdbc_query_tvf.out | 6 +
.../tvf/test_hdfs_tvf_error_uri.out | 5 -
.../test_count_match_all_pushdown.out | 16 +
.../distinct_agg_strategy_selector.out | 41 -
.../agg_strategy/test_variables.out | 24 -
.../lazy_materialize_topn.out | 6 +
.../filter_push_down/push_filter_through.out | 8 +-
.../data/query_p0/aggregate/window_funnel_v2.out | 56 ++
.../eliminate_outer_join/eliminate_outer_join.out | 10 +-
.../query_p0/system/test_query_sys_rowsets.out | 1 +
.../system/test_query_sys_scan_rowsets.out | 1 +
.../data/query_p0/system/test_table_properties.out | 8 +-
.../search/test_search_inverted_index_profile.out | 22 +
.../data/shape_check/clickbench/query11.out | 2 +-
.../data/shape_check/clickbench/query12.out | 2 +-
.../data/shape_check/clickbench/query13.out | 2 +-
.../data/shape_check/clickbench/query14.out | 2 +-
.../data/shape_check/clickbench/query15.out | 2 +-
.../data/shape_check/clickbench/query22.out | 2 +-
.../data/shape_check/clickbench/query23.out | 2 +-
.../data/shape_check/clickbench/query25.out | 2 +-
.../data/shape_check/clickbench/query26.out | 2 +-
.../data/shape_check/clickbench/query27.out | 2 +-
.../data/shape_check/clickbench/query28.out | 2 +-
.../data/shape_check/clickbench/query29.out | 2 +-
.../data/shape_check/clickbench/query31.out | 2 +-
.../data/shape_check/clickbench/query32.out | 2 +-
.../data/shape_check/clickbench/query37.out | 2 +-
.../data/shape_check/clickbench/query38.out | 2 +-
.../tpcds_sf100/noStatsRfPrune/query11.out | 105 ++-
.../tpcds_sf100/noStatsRfPrune/query4.out | 171 ++--
.../tpcds_sf100/noStatsRfPrune/query74.out | 105 ++-
.../tpcds_sf100/no_stats_shape/query11.out | 105 ++-
.../tpcds_sf100/no_stats_shape/query4.out | 171 ++--
.../tpcds_sf100/no_stats_shape/query74.out | 105 ++-
.../shape_check/tpcds_sf100/rf_prune/query11.out | 115 ++-
.../shape_check/tpcds_sf100/rf_prune/query4.out | 169 ++--
.../shape_check/tpcds_sf100/rf_prune/query74.out | 115 ++-
.../data/shape_check/tpcds_sf100/shape/query11.out | 115 ++-
.../data/shape_check/tpcds_sf100/shape/query4.out | 169 ++--
.../data/shape_check/tpcds_sf100/shape/query74.out | 115 ++-
.../data/shape_check/tpcds_sf1000/hint/query11.out | 105 ++-
.../data/shape_check/tpcds_sf1000/hint/query4.out | 169 ++--
.../data/shape_check/tpcds_sf1000/hint/query74.out | 103 ++-
.../shape_check/tpcds_sf1000/shape/query11.out | 115 ++-
.../data/shape_check/tpcds_sf1000/shape/query4.out | 169 ++--
.../shape_check/tpcds_sf1000/shape/query74.out | 103 ++-
.../tpcds_sf1000_nopkfk/shape/query11.out | 129 +--
.../tpcds_sf1000_nopkfk/shape/query4.out | 202 +++--
.../tpcds_sf1000_nopkfk/shape/query74.out | 135 +--
.../shape_check/tpcds_sf10t_orc/shape/query11.out | 119 +--
.../shape_check/tpcds_sf10t_orc/shape/query4.out | 181 ++--
.../shape_check/tpcds_sf10t_orc/shape/query74.out | 119 +--
.../doris/regression/suite/SuiteCluster.groovy | 31 +-
.../pipeline/common/get-hive-bootstrap-groups.sh | 46 +-
regression-test/pipeline/common/github-utils.sh | 31 +
regression-test/pipeline/common/stage-timer.sh | 129 +++
.../pipeline/common/test_stage_timer.py | 149 ++++
.../pipeline/external/external-stage-timer.sh | 119 +++
.../suites/ann_index_p0/ann_index_basic.groovy | 22 +-
.../suites/ann_index_p0/ivf_index_test.groovy | 31 +-
.../ann_index_p0/ivf_on_disk_index_test.groovy | 18 +-
.../suites/auth/test_http_api_auth.groovy | 216 +++++
.../compaction/test_be_compaction_tasks.groovy | 167 ++++
.../test_compaction_profile_action.groovy | 193 +++++
.../test_inlineview_with_window_function.groovy | 5 +-
...est_rf_in_list_not_erased_by_scope_range.groovy | 91 ++
.../jdbc/test_jdbc_query_tvf.groovy | 5 +-
.../backup_restore_object_storage.groovy | 4 +-
.../tvf/test_hdfs_tvf_error_uri.groovy | 14 +-
.../test_count_match_all_pushdown.groovy | 83 ++
.../test_streaming_mysql_job_create_alter.groovy | 14 +-
.../load_p0/msk_load/test_routine_load_msk.groovy | 177 ++++
.../test_routine_load_msk_public_access.groovy | 105 +++
.../distinct_agg_strategy_selector.groovy | 10 +-
.../agg_strategy/test_variables.groovy | 7 +-
.../string_length_column_pruning.groovy | 338 ++++++++
.../query_p0/aggregate/window_funnel_v2.groovy | 412 +++++++++
.../test_cte_multiuse_no_union_not_inline.groovy | 145 ++++
.../test_cte_nondeterministic_not_inline.groovy | 123 +++
.../test_search_inverted_index_profile.groovy | 302 +++++++
...est_partition_rebalancer_medium_mismatch.groovy | 178 ++++
regression-test/suites/tso_p0/test_tso_api.groovy | 117 +++
.../tso_p0/test_tso_rowset_commit_tso.groovy | 78 ++
.../suites/variant_p0/delete_update.groovy | 40 +
.../predefine/predefined_typed_to_sparse.groovy | 2 +-
run-fe-ut.sh | 4 +-
run-fs-env-test.sh | 105 +++
.../ddl/create-tpcds-tables-sf10000.sql | 4 +-
944 files changed, 48991 insertions(+), 18243 deletions(-)
create mode 100644 .gitleaks.toml
create mode 100644 be/benchmark/benchmark_column_view.hpp
create mode 100644
be/src/information_schema/schema_compaction_tasks_scanner.cpp
copy be/src/{service/http/action/stream_load_2pc.h =>
information_schema/schema_compaction_tasks_scanner.h} (54%)
create mode 100644 be/src/runtime/aws_msk_iam_auth.cpp
create mode 100644 be/src/runtime/aws_msk_iam_auth.h
create mode 100644 be/src/service/http/action/compaction_profile_action.cpp
copy be/src/service/http/action/{health_action.h =>
compaction_profile_action.h} (78%)
create mode 100644 be/src/storage/compaction_task_tracker.cpp
create mode 100644 be/src/storage/compaction_task_tracker.h
create mode 100644 be/test/runtime/aws_msk_iam_auth_test.cpp
create mode 100644 be/test/runtime/fragment_mgr_cross_cluster_cancel_test.cpp
create mode 100644 be/test/storage/compaction_task_tracker_test.cpp
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/bootstrap/bootstrap-groups.sh
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/bootstrap/hive2_only.preinstalled_hql.list
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/bootstrap/hive2_only.run_sh.list
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/bootstrap/hive3_only.download_dir.list
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/bootstrap/hive3_only.run_sh.list
create mode 100644 docker/thirdparties/docker-health.sh
create mode 100644
docker/thirdparties/test/run-thirdparties-docker-hive-bootstrap-groups-test.sh
create mode 100644
docker/thirdparties/test/run-thirdparties-docker-hive-restart-test.sh
copy fe/fe-authentication/{fe-authentication-handler =>
fe-authentication-role-mapping}/pom.xml (52%)
create mode 100644
fe/fe-authentication/fe-authentication-role-mapping/src/main/java/org/apache/doris/authentication/rolemapping/DefinitionBackedRoleMappingEvaluator.java
create mode 100644
fe/fe-authentication/fe-authentication-role-mapping/src/main/java/org/apache/doris/authentication/rolemapping/IntegrationPropertyRoleMappingEvaluator.java
rename
fe/{fe-core/src/main/java/org/apache/doris/cloud/storage/ObjectFile.java =>
fe-authentication/fe-authentication-role-mapping/src/main/java/org/apache/doris/authentication/rolemapping/NoOpRoleMappingEvaluator.java}
(52%)
create mode 100644
fe/fe-authentication/fe-authentication-role-mapping/src/main/java/org/apache/doris/authentication/rolemapping/RoleMappingDefinition.java
copy fe/{fe-core/src/main/java/org/apache/doris/fs/FileSystemProvider.java =>
fe-authentication/fe-authentication-role-mapping/src/main/java/org/apache/doris/authentication/rolemapping/RoleMappingDefinitionProvider.java}
(76%)
copy fe/{fe-core/src/main/java/org/apache/doris/fs/io/ParsedPath.java =>
fe-authentication/fe-authentication-role-mapping/src/main/java/org/apache/doris/authentication/rolemapping/RoleMappingEvaluator.java}
(62%)
create mode 100644
fe/fe-authentication/fe-authentication-role-mapping/src/main/java/org/apache/doris/authentication/rolemapping/UnifiedRoleMappingCelEngine.java
create mode 100644
fe/fe-authentication/fe-authentication-role-mapping/src/test/java/org/apache/doris/authentication/plugin/oidc/playground/RoleMappingCelPlayground.java
rename
fe/{fe-core/src/main/java/org/apache/doris/cloud/storage/ListObjectsResult.java
=>
fe-authentication/fe-authentication-role-mapping/src/test/java/org/apache/doris/authentication/plugin/oidc/playground/RoleMappingCelPlaygroundTest.java}
(51%)
create mode 100644
fe/fe-authentication/fe-authentication-role-mapping/src/test/java/org/apache/doris/authentication/rolemapping/IntegrationPropertyRoleMappingEvaluatorTest.java
create mode 100644
fe/fe-authentication/fe-authentication-role-mapping/src/test/java/org/apache/doris/authentication/rolemapping/UnifiedRoleMappingCelEngineTest.java
copy fe/{fe-core/src/main/java/org/apache/doris/analysis/VirtualSlotRef.java
=> fe-catalog/src/main/java/org/apache/doris/catalog/NameSpaceContext.java}
(51%)
rename fe/{fe-core =>
fe-common}/src/main/java/org/apache/doris/qe/GlobalVariable.java (77%)
create mode 100644
fe/fe-common/src/main/java/org/apache/doris/qe/VarAttrDef.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteralUtils.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteralUtils.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/ExprToColumnLabelVisitor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/ExprToExprNameVisitor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/authentication/RoleMappingMeta.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/authentication/RoleMappingMgr.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/ColumnToProtobuf.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/catalog/ColumnToThrift.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/AzureRemote.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/BosRemote.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/CosRemote.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/DefaultRemote.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/MockRemote.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/ObjectInfo.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/ObjectInfoAdapter.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/ObsRemote.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/OssRemote.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/RemoteBase.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/cloud/storage/S3Remote.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/cluster/ClusterGuard.java
rename
fe/fe-core/src/main/java/org/apache/doris/{fs/remote/dfs/OSSHdfsFileSystem.java
=> cluster/ClusterGuardException.java} (70%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/cluster/ClusterGuardFactory.java
copy fe/fe-core/src/main/java/org/apache/doris/{analysis/VirtualSlotRef.java
=> cluster/NoOpClusterGuard.java} (52%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/FilePartitionUtils.java
delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/fs/FileSystem.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/FileSystemDescriptor.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/FileSystemIOException.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/FileSystemPluginManager.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/FileSystemProviderImpl.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/FsStorageTypeAdapter.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/LocalDfsFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/PersistentFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/RemoteFileRemoteIterator.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/SpiSwitchingFileSystem.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/StoragePropertiesConverter.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/StorageTypeMapper.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/DorisInput.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/DorisInputFile.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/DorisInputStream.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/DorisOutputFile.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/hdfs/HdfsInput.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/hdfs/HdfsInputFile.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/hdfs/HdfsInputStream.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/hdfs/HdfsOutputFile.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/io/hdfs/HdfsOutputStream.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/obj/AzureObjStorage.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/obj/ObjStorage.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/obj/S3ObjStorage.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/operations/BrokerFileOperations.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/operations/BrokerOpParams.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/operations/HDFSFileOperations.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/operations/HDFSOpParams.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/operations/OpParams.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/AzureFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/BrokerFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/ObjFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/RemoteFile.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/RemoteFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/S3FileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/SwitchingFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/dfs/DFSFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/dfs/DFSFileSystemPhantomReference.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/dfs/OFSFileSystem.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/fs/remote/dfs/RemoteFSPhantomManager.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/TSOAction.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/httpv2/restv2/ClusterGuardAction.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/info/TableNameInfoUtils.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mysql/authenticate/AuthenticationFailureSummary.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mysql/authenticate/MysqlAuthPacketCredentialExtractor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/mysql/authenticate/password/AuthPacketAwarePasswordResolver.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/StringEmptyToLengthRule.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/CTEInliner.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateRoleMappingCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/DropRoleMappingCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/job/UnassignedLocalTVFSinkJob.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/persist/DropRoleMappingOperationLog.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/tso/TSOService.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/tso/TSOTimestamp.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/analysis/DateLiteralTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/analysis/DateLiteralUtilsTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/analysis/DecimalLiteralUtilsTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/authentication/RoleMappingMgrTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/backup/BrokerStorageTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/SessionVariablesNullFixTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cluster/ClusterGuardExceptionTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cluster/ClusterGuardFactoryTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cluster/NoOpClusterGuardTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/jdbc/util/JdbcFieldSchemaTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/external/iceberg/IcebergHadoopCatalogTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/FileSystemTransferUtilTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/MemoryFileSystem.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/MemoryFileSystemTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/SpiSwitchingFileSystemTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/obj/AzureObjStorageTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/obj/MockedS3Client.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/obj/S3FileSystemTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/obj/S3ObjStorageGlobListTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/obj/S3ObjStorageTest.java
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/fs/remote/RemoteFileSystemTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/httpv2/rest/HttpApiAuthTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/info/TableNameInfoUtilsTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/load/DeleteJobTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/load/routineload/KafkaAwsMskIamAuthTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/master/MasterImplDeleteTaskTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mysql/authenticate/MysqlAuthPacketCredentialExtractorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mysql/authenticate/password/ClearPasswordResolverTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mysql/authenticate/plugin/AuthenticationPluginAuthenticatorTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/parser/RoleMappingParserTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rules/StringEmptyToLengthRuleTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/qe/AuditLogHelperTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/qe/ConnectAttributesForwardTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/tso/TSOServiceTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/tso/TSOTimestampTest.java
create mode 100644 fe/fe-filesystem/ENV_TEST_README.md
create mode 100644 fe/fe-filesystem/fe-filesystem-api/pom.xml
copy fe/{fe-core/src/main/java/org/apache/doris/analysis/VirtualSlotRef.java
=>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/BlockInfo.java}
(59%)
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/DorisInputFile.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/DorisInputStream.java
copy fe/{fe-core/src/main/java/org/apache/doris/fs/RemoteFiles.java =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/DorisOutputFile.java}
(71%)
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/FileEntry.java
copy
fe/{fe-core/src/main/java/org/apache/doris/fs/operations/FileOperations.java =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/FileIterator.java}
(69%)
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/FileSystem.java
rename fe/{fe-core/src/main/java/org/apache/doris/fs/RemoteFiles.java =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/FileSystemIOException.java}
(72%)
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/FileSystemTransferUtil.java
rename fe/{fe-core/src/main/java/org/apache/doris/fs =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem}/FileSystemType.java
(98%)
rename fe/{fe-core/src/main/java/org/apache/doris/fs =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem}/FileSystemUtil.java
(62%)
rename fe/{fe-core/src/main/java/org/apache/doris/fs/GlobListResult.java =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/GlobListing.java}
(50%)
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem/Location.java
rename fe/{fe-core/src/main/java/org/apache/doris/fs =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem}/RemoteIterator.java
(96%)
rename fe/{fe-core/src/main/java/org/apache/doris/fs =>
fe-filesystem/fe-filesystem-api/src/main/java/org/apache/doris/filesystem}/SimpleRemoteIterator.java
(80%)
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/BlockInfoTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/EnvTestBase.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/FileEntryTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/FileSystemDefaultMethodsTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/FileSystemTransferUtilTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/FileSystemUtilTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/GlobListingTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/LocationTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-api/src/test/java/org/apache/doris/filesystem/SimpleRemoteIteratorTest.java
copy fe/{be-java-extensions/paimon-scanner =>
fe-filesystem/fe-filesystem-azure}/pom.xml (54%)
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/main/java/org/apache/doris/filesystem/azure/AzureFileSystem.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/main/java/org/apache/doris/filesystem/azure/AzureFileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/main/java/org/apache/doris/filesystem/azure/AzureObjStorage.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/main/java/org/apache/doris/filesystem/azure/AzureUri.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/test/java/org/apache/doris/filesystem/azure/AzureFileSystemEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/test/java/org/apache/doris/filesystem/azure/AzureObjStorageExtensionTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-azure/src/test/java/org/apache/doris/filesystem/azure/AzureUriTest.java
create mode 100644 fe/fe-filesystem/fe-filesystem-broker/pom.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerClientFactory.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerClientPool.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerFileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerInputFile.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerInputStream.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerOutputFile.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerOutputStream.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/java/org/apache/doris/filesystem/broker/BrokerSpiFileSystem.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/test/java/org/apache/doris/filesystem/broker/BrokerFileSystemProviderTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/test/java/org/apache/doris/filesystem/broker/BrokerSpiFileSystemEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-broker/src/test/java/org/apache/doris/filesystem/broker/BrokerSpiFileSystemTest.java
copy fe/{be-java-extensions/paimon-scanner =>
fe-filesystem/fe-filesystem-cos}/pom.xml (55%)
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/main/java/org/apache/doris/filesystem/cos/CosFileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/main/java/org/apache/doris/filesystem/cos/CosObjStorage.java
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/test/java/org/apache/doris/filesystem/cos/CosFileSystemEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-cos/src/test/java/org/apache/doris/filesystem/cos/CosObjStorageTest.java
copy fe/{be-java-extensions/paimon-scanner =>
fe-filesystem/fe-filesystem-hdfs}/pom.xml (52%)
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/DFSFileSystem.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/HdfsConfigBuilder.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/HdfsFileIterator.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/HdfsFileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/HdfsInputFile.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/HdfsOutputFile.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/HdfsSeekableInputStream.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/KerberosHadoopAuthenticator.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/java/org/apache/doris/filesystem/hdfs/SimpleHadoopAuthenticator.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/DFSFileSystemEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/DFSFileSystemTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/HdfsConfigBuilderTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/HdfsSeekableInputStreamTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/KerberosHadoopAuthenticatorEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-hdfs/src/test/java/org/apache/doris/filesystem/hdfs/SimpleHadoopAuthenticatorTest.java
copy fe/{be-java-extensions/paimon-scanner =>
fe-filesystem/fe-filesystem-local}/pom.xml (51%)
create mode 100644
fe/fe-filesystem/fe-filesystem-local/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-local/src/main/java/org/apache/doris/filesystem/local/LocalFileSystem.java
copy fe/{fe-core/src/main/java/org/apache/doris/analysis/VirtualSlotRef.java
=>
fe-filesystem/fe-filesystem-local/src/main/java/org/apache/doris/filesystem/local/LocalFileSystemProvider.java}
(54%)
create mode 100644
fe/fe-filesystem/fe-filesystem-local/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-local/src/test/java/org/apache/doris/filesystem/local/LocalFileSystemTest.java
copy fe/{be-java-extensions/paimon-scanner =>
fe-filesystem/fe-filesystem-obs}/pom.xml (55%)
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/main/java/org/apache/doris/filesystem/obs/ObsFileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/main/java/org/apache/doris/filesystem/obs/ObsObjStorage.java
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/test/java/org/apache/doris/filesystem/obs/ObsFileSystemEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-obs/src/test/java/org/apache/doris/filesystem/obs/ObsObjStorageTest.java
copy fe/{be-java-extensions/paimon-scanner =>
fe-filesystem/fe-filesystem-oss}/pom.xml (54%)
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/main/java/org/apache/doris/filesystem/oss/OssFileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/main/java/org/apache/doris/filesystem/oss/OssObjStorage.java
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/test/java/org/apache/doris/filesystem/oss/OssFileSystemEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-oss/src/test/java/org/apache/doris/filesystem/oss/OssObjStorageTest.java
create mode 100644 fe/fe-filesystem/fe-filesystem-s3/pom.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/main/assembly/plugin-zip.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/main/java/org/apache/doris/filesystem/s3/S3FileSystem.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/main/java/org/apache/doris/filesystem/s3/S3FileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/main/java/org/apache/doris/filesystem/s3/S3ObjStorage.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/main/java/org/apache/doris/filesystem/s3/S3OutputStream.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/main/java/org/apache/doris/filesystem/s3/S3Uri.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/main/resources/META-INF/services/org.apache.doris.filesystem.spi.FileSystemProvider
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3FileSystemEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3FileSystemTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3ObjStorageEnvTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3ObjStorageMockTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3ObjStorageTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3OutputStreamTest.java
create mode 100644
fe/fe-filesystem/fe-filesystem-s3/src/test/java/org/apache/doris/filesystem/s3/S3UriTest.java
create mode 100644 fe/fe-filesystem/fe-filesystem-spi/pom.xml
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/FileSystemProvider.java
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/HadoopAuthenticator.java
rename
fe/{fe-core/src/main/java/org/apache/doris/fs/operations/FileOperations.java =>
fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/IOCallable.java}
(67%)
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/ObjFileSystem.java
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/ObjStorage.java
rename fe/{fe-core/src/main/java/org/apache/doris/fs/obj =>
fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi}/RemoteObject.java
(76%)
rename fe/{fe-core/src/main/java/org/apache/doris/fs/obj =>
fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi}/RemoteObjects.java
(72%)
rename fe/{fe-core/src/main/java/org/apache/doris/cloud/storage/TosRemote.java
=>
fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/RequestBody.java}
(53%)
copy fe/{fe-core/src/main/java/org/apache/doris/analysis/VirtualSlotRef.java
=>
fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/StsCredentials.java}
(53%)
rename fe/{fe-core/src/main/java/org/apache/doris/fs/io/ParsedPath.java =>
fe-filesystem/fe-filesystem-spi/src/main/java/org/apache/doris/filesystem/spi/UploadPartResult.java}
(65%)
create mode 100644
fe/fe-filesystem/fe-filesystem-spi/src/test/java/org/apache/doris/filesystem/spi/ObjFileSystemTest.java
create mode 100644 fe/fe-filesystem/pom.xml
rename
fe/{fe-core/src/main/java/org/apache/doris/fs/remote/dfs/JFSFileSystem.java =>
fe-foundation/src/main/java/org/apache/doris/foundation/fs/FsStorageType.java}
(66%)
rename fe/{fe-core/src/main/java/org/apache/doris/fs/FileSystemProvider.java
=> fe-type/src/main/java/org/apache/doris/catalog/PatternType.java} (78%)
create mode 100755 post-build.sh
copy regression-test/data/{external_table_p0/tvf/test_hdfs_tvf_error_uri.out
=> correctness_p0/test_rf_in_list_not_erased_by_scope_range.out} (60%)
create mode 100644
regression-test/data/inverted_index_p0/test_count_match_all_pushdown.out
create mode 100644
regression-test/data/nereids_rules_p0/defer_materialize_topn/lazy_materialize_topn.out
create mode 100644
regression-test/data/search/test_search_inverted_index_profile.out
copy gensrc/script/Makefile =>
regression-test/pipeline/common/get-hive-bootstrap-groups.sh (52%)
mode change 100644 => 100755
create mode 100644 regression-test/pipeline/common/stage-timer.sh
create mode 100644 regression-test/pipeline/common/test_stage_timer.py
create mode 100644 regression-test/pipeline/external/external-stage-timer.sh
create mode 100644 regression-test/suites/auth/test_http_api_auth.groovy
create mode 100644
regression-test/suites/compaction/test_be_compaction_tasks.groovy
create mode 100644
regression-test/suites/compaction/test_compaction_profile_action.groovy
create mode 100644
regression-test/suites/correctness_p0/test_rf_in_list_not_erased_by_scope_range.groovy
create mode 100644
regression-test/suites/inverted_index_p0/test_count_match_all_pushdown.groovy
create mode 100644
regression-test/suites/load_p0/msk_load/test_routine_load_msk.groovy
create mode 100644
regression-test/suites/load_p0/msk_load/test_routine_load_msk_public_access.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/column_pruning/string_length_column_pruning.groovy
create mode 100644
regression-test/suites/query_p0/cte/test_cte_multiuse_no_union_not_inline.groovy
create mode 100644
regression-test/suites/query_p0/cte/test_cte_nondeterministic_not_inline.groovy
create mode 100644
regression-test/suites/search/test_search_inverted_index_profile.groovy
create mode 100644
regression-test/suites/storage_medium_p0/test_partition_rebalancer_medium_mismatch.groovy
create mode 100644 regression-test/suites/tso_p0/test_tso_api.groovy
create mode 100644
regression-test/suites/tso_p0/test_tso_rowset_commit_tso.groovy
create mode 100755 run-fs-env-test.sh
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
