This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch auto-pick-69007-branch-4.2
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 611ea355163 [fix](cloud) Prevent stale delete rowset replay from
recreating tmp metadata (#69007)
add d10599dc0c6 branch-4.2: [fix](fe) Update vulnerable dependencies
(#69364)
add 0128b5fba2f branch-4.2: [fix](cloud) Get packed slice location through
the FileWriter interface #68106 (#69413)
add 4a7ef30003c branch-4.2: [fix](cloud) Reclaim orphaned rowset warmup
states #69325 (#69415)
add 7ad136799f2 branch-4.2: [fix](inverted index) Share BM25 statistics
across parallel scanners of a tablet #69352 (#69440)
add 2a0f0c27ed7 branch-4.2: [fix](regression) Fix flaky materialized view
load-open test #67138 (#69437)
add 747fff370b1 branch-4.2: [fix](fe) Propagate JSON properties for
multi-table routine load (#69453)
add c90d0c8de5f branch-4.2: [fix](be) Stop preallocating compression-sized
t-digest b… (#69470)
add 72df240a6d4 [fix](exec) Backport batched rowid reads for lazy
materialization to 4.2 (#69429)
add 56043de25cf branch-4.2: [fix](fe) Preserve inferred JSON column name
casing (#69451)
add 1b182a2816a branch-4.2: [fix](rbo) Give the constants materialized by
constant propagation a fresh exprId #69111 (#69406)
add 8e5b079051b [fix](fe) Allow table access through Iceberg JDBC V0
catalogs (branch-4.2) (#69485)
add 756a67000b7 [test](regression) Make map_agg NULL-key assertions
order-independent (#69446)
add 8e590aeb0fe branch-4.2: [fix](nereids) Allow sync MV rewrite across
strict-table SQL modes (#69476)
add 1ff7c75a33f branch-4.2: [test](regression) Run isolated external
suites concurrently #69310 (#69468)
add 202578e8c2b [fix](hudi) Keep storage credentials out of scanner logs
(branch-4.2) (#69479)
add 2e0ce88165d [fix](cloud) Clean stale routes when replaying replica
updates and before saving checkpoint images (#67636) (#69433)
add eaf1c85512d branch-4.2: [fix](fe) Check automatic partition limit
before creation (#69502)
add 2204b4c8993 branch-4.2: [fix](job) Prevent stale cloud broker load
retries (#69500)
add 301220850fb branch-4.2: [fix](meta) Rebuild view column lookup after
metadata recovery #69150 (#69483)
add 98ba8ae0eb4 branch-4.2: [fix](mow) Skip delete bitmap cache prefill
for non-running tablets #68436 (#69497)
add 18b118249ca branch-4.2: [fix](fe) Refresh S3 streaming source file
metadata #69363 (#69521)
add a4bc07f53a5 branch-4.2: [fix](regression) Stabilize the double write
schema change case #69459 (#69548)
add 7c3b2434386 branch-4.2: [fix](security) Stop logging sensitive FE RPC
requests #69472 (#69546)
add 00a36cdc9ab [fix](be) Keep credentials out of fragment execution logs
(branch-4.2) (#69543)
add 1046e88b3fd [fix](lakesoul) Avoid logging credentials on branch-4.2
(#69535)
add 72ccc4b3e53 branch-4.2: [fix](function) Skip hidden elements in NULL
subarrays when flattening #69321 (#69534)
add 114dc0f74cb branch-4.2: [fix](nereids) Fold UTC_DATE and UTC_TIME in
FE #69393 (#69520)
add 1afd1d04410 branch-4.2: [fix](cloud) Add the aggregation output rowset
to the delete bitmap replacement test #69349 (#69357)
add 17be18138ac branch-4.2: [fix](fe) Refresh backend heartbeat
connections after FQDN IP changes #69450 (#69518)
add bc57778c55e [fix](cloud) Prevent stale delete rowset replay from
recreating tmp metadata (#69007)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (611ea355163)
\
N -- N -- N refs/heads/auto-pick-69007-branch-4.2 (bc57778c55e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/cloud/cloud_internal_service.cpp | 19 +-
be/src/cloud/cloud_rowset_writer.cpp | 11 +-
be/src/cloud/cloud_tablet.cpp | 122 ++++--
be/src/cloud/cloud_tablet.h | 24 +-
be/src/cloud/cloud_tablet_mgr.cpp | 23 +-
be/src/cloud/cloud_warm_up_manager.cpp | 16 +-
be/src/cloud/config.cpp | 2 +
be/src/cloud/config.h | 1 +
be/src/cloud/delete_bitmap_file_writer.cpp | 3 +-
be/src/exec/operator/olap_scan_operator.cpp | 5 +
be/src/exec/rowid_fetcher.cpp | 115 ++++--
be/src/exec/scan/file_scanner.cpp | 6 +
be/src/exec/scan/file_scanner_v2.cpp | 7 +
be/src/exec/scan/olap_scanner.cpp | 14 +-
be/src/exec/scan/olap_scanner.h | 5 +
be/src/exec/scan/parallel_scanner_builder.cpp | 37 +-
be/src/exec/scan/parallel_scanner_builder.h | 16 +-
.../function/array/function_array_flatten.cpp | 40 +-
be/src/io/fs/file_writer.h | 12 +-
be/src/io/fs/packed_file_manager.h | 13 +-
be/src/io/fs/packed_file_writer.h | 5 +-
be/src/io/fs/packed_slice_location.h | 39 ++
be/src/runtime/fragment_mgr.cpp | 15 +-
be/src/service/point_query_executor.cpp | 5 +-
.../inverted/reader/collection_statistics.cpp | 27 ++
.../index/inverted/reader/collection_statistics.h | 32 ++
be/src/storage/segment/segment.cpp | 17 +-
be/src/storage/segment/segment.h | 5 +-
be/src/storage/tablet/base_tablet.cpp | 5 +
be/src/util/debug_util.cpp | 6 +
be/src/util/debug_util.h | 5 +
be/src/util/tdigest.h | 18 +-
be/test/cloud/cloud_tablet_mgr_test.cpp | 36 ++
be/test/cloud/cloud_tablet_test.cpp | 425 +++++++++++++++++----
be/test/exprs/aggregate/agg_percentile_test.cpp | 5 +-
.../exprs/function/function_array_flatten_test.cpp | 153 ++++++++
be/test/io/fs/packed_file_system_test.cpp | 7 +-
be/test/io/fs/packed_file_writer_test.cpp | 23 ++
be/test/runtime/fragment_mgr_log_test.cpp | 143 +++++++
.../inverted/reader/collection_statistics_test.cpp | 32 ++
.../variant/variant_column_writer_reader_test.cpp | 7 +-
be/test/util/fragment_params_log_test.cpp | 69 ++++
be/test/util/tdigest_test.cpp | 44 +++
cloud/test/meta_service_test.cpp | 1 +
.../org/apache/doris/avro/HDFSFileReaderTest.java | 72 ++++
.../apache/doris/hudi/HadoopHudiJniScanner.java | 3 +-
.../doris/hudi/HadoopHudiJniScannerLogTest.java | 137 +++++++
fe/be-java-extensions/lakesoul-scanner/pom.xml | 5 +
.../apache/doris/lakesoul/LakeSoulJniScanner.java | 2 +-
.../doris/lakesoul/LakeSoulJniScannerTest.java | 122 ++++++
.../src/test/resources/log4j2-test.xml | 25 ++
fe/be-java-extensions/pom.xml | 10 +
.../main/java/org/apache/doris/common/Config.java | 21 +-
fe/fe-core/pom.xml | 3 +-
.../main/java/org/apache/doris/catalog/Env.java | 12 +-
.../main/java/org/apache/doris/catalog/View.java | 1 +
.../org/apache/doris/cloud/catalog/CloudEnv.java | 27 ++
.../apache/doris/cloud/catalog/CloudReplica.java | 58 ++-
.../doris/cloud/catalog/CloudTabletRebalancer.java | 72 +++-
.../cloud/datasource/CloudInternalCatalog.java | 92 ++++-
.../doris/cloud/load/CloudBrokerLoadJob.java | 8 +
.../doris/cloud/system/CloudSystemInfoService.java | 8 +
.../apache/doris/datasource/ExternalCatalog.java | 20 +
.../apache/doris/datasource/InternalCatalog.java | 22 ++
.../datasource/iceberg/IcebergMetadataOps.java | 25 +-
.../lakesoul/source/LakeSoulScanNode.java | 3 -
.../job/offset/s3/S3EventSourceOffsetProvider.java | 6 +-
.../job/offset/s3/S3SourceOffsetProvider.java | 3 +-
.../java/org/apache/doris/load/loadv2/LoadJob.java | 5 +
.../java/org/apache/doris/master/Checkpoint.java | 19 +
.../org/apache/doris/nereids/StatementContext.java | 38 +-
.../doris/nereids/load/NereidsStreamLoadTask.java | 2 +
.../rules/analysis/SessionVarGuardRewriter.java | 72 ++++
.../mv/InitMaterializationContextHook.java | 2 +-
.../nereids/rules/rewrite/ConstantPropagation.java | 82 +++-
.../nereids/rules/rewrite/ExprIdRewriter.java | 139 ++++---
.../nereids/rules/rewrite/RewriteCteChildren.java | 89 ++++-
.../functions/executable/DateTimeAcquire.java | 15 +
.../apache/doris/service/FrontendServiceImpl.java | 54 +--
.../java/org/apache/doris/system/HeartbeatMgr.java | 34 +-
.../ExternalFileTableValuedFunction.java | 40 +-
.../org/apache/doris/catalog/ViewReplayTest.java | 51 +++
.../cloud/catalog/CloudEnvRouteCleanupTest.java | 145 +++++++
.../doris/cloud/catalog/CloudReplicaTest.java | 89 +++++
.../cloud/catalog/CloudTabletRebalancerTest.java | 144 +++++++
.../cloud/datasource/CloudInternalCatalogTest.java | 184 +++++++++
.../doris/cloud/load/CloudBrokerLoadJobTest.java | 35 ++
.../datasource/ExternalCatalogDropKindTest.java | 221 +++++++++++
.../iceberg/IcebergJdbcViewProbeTest.java | 144 +++++++
.../lakesoul/source/LakeSoulScanNodeTest.java | 110 ++++++
.../offset/s3/S3EventSourceOffsetProviderTest.java | 28 +-
.../job/offset/s3/S3SourceOffsetProviderTest.java | 29 ++
.../org/apache/doris/load/loadv2/LoadJobTest.java | 9 +
.../org/apache/doris/master/CheckpointTest.java | 276 +++++++++++++
.../nereids/load/NereidsStreamLoadTaskTest.java | 60 +++
.../ConstantPropagationFreshExprIdTest.java | 280 ++++++++++++++
.../rules/rewrite/ConstantPropagationTest.java | 154 +++++++-
.../rewrite/CteConsumerOutputsRefresherTest.java | 107 ++++++
.../nereids/rules/rewrite/ExprIdRewriterTest.java | 35 ++
.../trees/expressions/VariablePersistTest.java | 211 +++++++++-
.../trees/expressions/literal/JsonLiteralTest.java | 6 +-
.../service/FrontendServiceImplCloudTest.java | 70 ++++
.../doris/service/FrontendServiceImplTest.java | 102 +++++
.../doris/system/BackendHeartbeatHandlerTest.java | 309 +++++++++++++++
.../tablefunction/S3TableValuedFunctionTest.java | 48 ++-
fe/hive-udf/pom.xml | 66 ++++
fe/pom.xml | 79 +++-
.../tvf/test_json_uppercase_keys.out | 9 +
.../tvf/test_json_uppercase_keys/case_keys.jsonl | 2 +
.../data/nereids_p0/datatype/test_date_acquire.out | 60 ++-
.../test_sync_rollup_strict_sql_mode.out | 16 +
.../nereids_syntax_p1/mv/aggregate/agg_sync_mv.out | 26 +-
.../array_functions/array_flatten.out | 8 +
.../test_double_write_when_schema_change.out | 4 +-
.../iceberg/test_iceberg_initial_defaults.groovy | 2 +-
.../iceberg/test_iceberg_jdbc_catalog.groovy | 101 +++++
.../paimon/write/test_paimon_write_variant.groovy | 2 +-
.../write/test_paimon_write_variant_dml.groovy | 2 +-
.../write/test_paimon_write_variant_errors.groovy | 2 +-
.../write/test_paimon_write_variant_nested.groovy | 2 +-
.../test_paimon_write_variant_shredding.groovy | 2 +-
.../test_paimon_write_variant_table_modes.groovy | 2 +-
.../tvf/test_json_uppercase_keys.groovy | 49 +++
...test_bm25_score_parallel_scan_statistics.groovy | 78 ++++
.../test_streaming_insert_job_s3_event_aksk.groovy | 21 +-
.../nereids_p0/datatype/test_date_acquire.groovy | 71 ++++
.../test_sync_rollup_strict_sql_mode.groovy | 224 +++++++++++
.../mv/aggregate/agg_sync_mv.groovy | 11 +-
.../array_functions/array_flatten.groovy | 29 ++
.../test_materialized_view_load_open.groovy | 10 +-
.../test_double_write_when_schema_change.groovy | 6 +-
.../test_double_write_when_schema_change.groovy | 6 +-
132 files changed, 6203 insertions(+), 497 deletions(-)
create mode 100644 be/src/io/fs/packed_slice_location.h
create mode 100644 be/test/exprs/function/function_array_flatten_test.cpp
create mode 100644 be/test/runtime/fragment_mgr_log_test.cpp
create mode 100644 be/test/util/fragment_params_log_test.cpp
create mode 100644
fe/be-java-extensions/avro-scanner/src/test/java/org/apache/doris/avro/HDFSFileReaderTest.java
create mode 100644
fe/be-java-extensions/hadoop-hudi-scanner/src/test/java/org/apache/doris/hudi/HadoopHudiJniScannerLogTest.java
create mode 100644
fe/be-java-extensions/lakesoul-scanner/src/test/java/org/apache/doris/lakesoul/LakeSoulJniScannerTest.java
create mode 100644
fe/be-java-extensions/lakesoul-scanner/src/test/resources/log4j2-test.xml
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/ViewReplayTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/cloud/catalog/CloudEnvRouteCleanupTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/ExternalCatalogDropKindTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/iceberg/IcebergJdbcViewProbeTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/lakesoul/source/LakeSoulScanNodeTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/master/CheckpointTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/load/NereidsStreamLoadTaskTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationFreshExprIdTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/CteConsumerOutputsRefresherTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/service/FrontendServiceImplCloudTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/system/BackendHeartbeatHandlerTest.java
create mode 100644
regression-test/data/external_table_p0/tvf/test_json_uppercase_keys.out
create mode 100644
regression-test/data/external_table_p0/tvf/test_json_uppercase_keys/case_keys.jsonl
create mode 100644
regression-test/data/nereids_p0/variables_persist/test_sync_rollup_strict_sql_mode.out
create mode 100644
regression-test/suites/external_table_p0/tvf/test_json_uppercase_keys.groovy
create mode 100644
regression-test/suites/inverted_index_p0/test_bm25_score_parallel_scan_statistics.groovy
create mode 100644
regression-test/suites/nereids_p0/variables_persist/test_sync_rollup_strict_sql_mode.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
