This is an automated email from the ASF dual-hosted git repository.
morrySnow pushed a change to branch auto-pick-66977-branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 5775718b8d6 [improvement](cloud) Add dry-run mode for BE-to-MS RPC
rate limiting (#66977)
add a58bf4059a8 branch-4.1: [fix](be) Restore null literal return type
handling #66280 (#67189)
add 1d465f6e87d branch-4.1:[fix](mtmv) Avoid invalid slot cast in MV
null-reject compensation (#66613) (#67266)
add a4ea51fdb8b branch-4.1: [fix](user variable) Preserve TIMESTAMPTZ user
variable type #66833 (#67128)
add 2871c53ed1e branch-4.1:[fix](mtmv) Refresh MTMV after excluded trigger
tables change (#64041) (#67267)
add f72ab0ca13b branch-4.1: [feature](query) Support session level
force_forward_all_queries #66691 (#66768)
add 908e484f018 [fix](be) Preserve nested paths for lazy rowid fetch
(#67205)
add 5ace365e8c0 branch-4.1: [fix](cloud) Separate HTTP encoding for job
recycle and check keys #67243 (#67271)
add b87c1778607 branch-4.1: [fix](streamingjob) Prevent MySQL CDC data
loss on keepalive reconnect for non-GTID #66998 (#67269)
add 790e3207c6d branch-4.1: [fix](exec) Return Thrift DATETIME as naive
Arrow timestamp #67232 (#67299)
add 3073aea9d0a [fix](partition) handle rollback failure when adding
partition (#64650) (#67212)
add 2267b487300 branch-4.1: [fix](test) stabilize the flaky
shuffle_left_join regression test #65769 (#65839)
add 81e30885b90 branch-4.1: [fix](cloud) Skip versioned delete bitmap
cleanup for non-MoW tablets #67084 (#67287)
add f1d3604c395 branch-4.1:[feature](iceberg) Support Iceberg V3 default
values (#66538)
add 2c3ae38c344 branch-4.1: [fix](catalog) Support database properties for
Paimon and Iceberg (#67163)
add a10f932c33e [improvement](cloud) Add dry-run mode for BE-to-MS RPC
rate limiting (#66977)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5775718b8d6)
\
N -- N -- N refs/heads/auto-pick-66977-branch-4.1 (a10f932c33e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.licenserc.yaml | 1 +
.../exec/operator/memory_scratch_sink_operator.cpp | 3 +-
be/src/exec/rowid_fetcher.cpp | 24 +
be/src/exec/scan/access_path_parser.cpp | 25 +-
.../sink/writer/iceberg/viceberg_table_writer.cpp | 10 +-
.../sink/writer/iceberg/viceberg_table_writer.h | 2 +
be/src/exprs/function/function.cpp | 5 +
be/src/format/arrow/arrow_block_convertor.cpp | 13 +-
be/src/format/arrow/arrow_block_convertor.h | 4 +
be/src/format/arrow/arrow_row_batch.cpp | 5 +-
be/src/format/arrow/arrow_row_batch.h | 2 +-
be/src/format/orc/vorc_reader.cpp | 82 +-
be/src/format/orc/vorc_reader.h | 11 +-
be/src/format/parquet/vparquet_column_reader.cpp | 22 +-
be/src/format/parquet/vparquet_column_reader.h | 3 +
be/src/format/parquet/vparquet_group_reader.cpp | 16 +-
be/src/format/parquet/vparquet_reader.cpp | 39 +-
be/src/format/parquet/vparquet_reader.h | 7 +
be/src/format/table/equality_delete.cpp | 157 +-
be/src/format/table/iceberg_default_value.h | 520 +++++
.../iceberg_position_delete_sys_table_reader.cpp | 88 +-
be/src/format/table/iceberg_reader.cpp | 2021 +++++++++++++++-----
be/src/format/table/iceberg_reader.h | 77 +-
be/src/format/table/iceberg_scan_semantics.h | 6 +
be/src/format/table/table_format_reader.cpp | 794 ++++++--
be/src/format/table/table_format_reader.h | 194 +-
be/src/format/transformer/vorc_transformer.cpp | 291 ++-
be/src/format/transformer/vorc_transformer.h | 7 +
be/src/format_v2/column_data.h | 4 +
be/src/format_v2/column_mapper.cpp | 132 +-
be/src/format_v2/column_mapper.h | 11 +-
be/src/format_v2/table/iceberg_reader.cpp | 998 +++++++++-
be/src/format_v2/table/iceberg_reader.h | 28 +-
be/src/format_v2/table_reader.cpp | 365 +++-
be/src/format_v2/table_reader.h | 179 +-
be/src/storage/segment/column_reader.cpp | 1 -
be/test/exec/scan/access_path_parser_test.cpp | 66 +-
.../writer/iceberg/viceberg_table_writer_test.cpp | 54 +
.../function/simple_function_factory_test.cpp | 41 +-
be/test/format/parquet/parquet_read_lines.cpp | 148 ++
be/test/format/table/equality_delete_test.cpp | 176 ++
.../iceberg_reader_create_column_ids_test.cpp | 16 +-
.../format/table/iceberg/iceberg_reader_test.cpp | 886 ++++++++-
.../table/table_schema_change_helper_test.cpp | 583 +++++-
.../format/transformer/vorc_transformer_test.cpp | 494 ++++-
be/test/format_v2/column_mapper_test.cpp | 68 +-
be/test/format_v2/table/iceberg_reader_test.cpp | 1313 ++++++++++++-
be/test/format_v2/table_reader_test.cpp | 1738 ++++++++++-------
cloud/src/meta-service/http_encode_key.cpp | 3 +-
cloud/src/meta-store/keys.h | 1 +
cloud/src/recycler/recycler.cpp | 111 +-
cloud/src/recycler/recycler.h | 16 +-
cloud/test/http_encode_key_test.cpp | 12 +
cloud/test/recycler_test.cpp | 81 +-
.../java/CreateIcebergInitialDefaultFixtures.java | 457 +++++
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 1 +
.../org/apache/doris/analysis/AccessPathInfo.java | 2 +
.../main/java/org/apache/doris/catalog/MTMV.java | 47 +-
.../org/apache/doris/catalog/PartitionInfo.java | 2 +
.../org/apache/doris/datasource/ExternalUtil.java | 92 +-
.../org/apache/doris/datasource/FileScanNode.java | 32 +-
.../apache/doris/datasource/InternalCatalog.java | 68 +-
.../datasource/iceberg/IcebergMetadataOps.java | 30 +-
.../datasource/iceberg/IcebergTransaction.java | 264 ++-
.../doris/datasource/iceberg/IcebergUtils.java | 197 +-
.../iceberg/IcebergWriteSchemaContext.java | 789 ++++++++
.../iceberg/helper/IcebergWriterHelper.java | 21 +-
.../datasource/iceberg/source/IcebergScanNode.java | 1117 ++++++++++-
.../datasource/iceberg/source/IcebergSplit.java | 2 +
.../doris/datasource/paimon/PaimonMetadataOps.java | 36 +-
.../apache/doris/job/extensions/mtmv/MTMVTask.java | 5 +
.../org/apache/doris/nereids/StatementContext.java | 22 +-
.../nereids/analyzer/UnboundIcebergTableSink.java | 47 +-
.../glue/translator/PhysicalPlanTranslator.java | 33 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 12 +
.../post/materialize/MaterializeProbeVisitor.java | 20 +-
.../properties/ChildrenPropertiesRegulator.java | 9 +-
.../doris/nereids/rules/analysis/BindSink.java | 107 +-
.../mv/AbstractMaterializedViewRule.java | 4 +-
.../rules/expression/ExpressionNormalization.java | 2 +
.../rules/expression/ExpressionRuleType.java | 1 +
.../expression/rules/RewriteDefaultExpression.java | 113 ++
...IcebergMergeSinkToPhysicalIcebergMergeSink.java | 1 +
...IcebergTableSinkToPhysicalIcebergTableSink.java | 1 +
.../expressions/{BitNot.java => Default.java} | 43 +-
.../expressions/literal/VarBinaryLiteral.java | 13 +-
.../expressions/visitor/ExpressionVisitor.java | 5 +
.../trees/plans/commands/ExplainCommand.java | 18 +-
.../plans/commands/IcebergDmlCommandUtils.java | 55 +
.../trees/plans/commands/IcebergMergeCommand.java | 114 +-
.../trees/plans/commands/IcebergUpdateCommand.java | 53 +-
.../insert/IcebergInsertCommandContext.java | 13 +
.../commands/insert/IcebergMergeExecutor.java | 19 +-
.../commands/insert/InsertIntoTableCommand.java | 18 +-
.../insert/InsertOverwriteTableCommand.java | 36 +-
.../trees/plans/commands/insert/InsertUtils.java | 195 +-
.../plans/logical/LogicalIcebergMergeSink.java | 45 +-
.../plans/logical/LogicalIcebergTableSink.java | 40 +-
.../plans/physical/PhysicalIcebergMergeSink.java | 66 +-
.../plans/physical/PhysicalIcebergTableSink.java | 58 +-
.../plans/physical/PhysicalLazyMaterialize.java | 13 +-
.../org/apache/doris/planner/IcebergMergeSink.java | 119 +-
.../org/apache/doris/planner/IcebergTableSink.java | 92 +-
.../java/org/apache/doris/qe/ConnectContext.java | 3 +
.../java/org/apache/doris/qe/SessionVariable.java | 28 +
.../java/org/apache/doris/qe/StmtExecutor.java | 3 +-
.../apache/doris/datasource/ExternalUtilTest.java | 81 +
.../doris/datasource/InternalCatalogTest.java | 226 +++
.../iceberg/IcebergDDLAndDMLPlanTest.java | 533 +++++-
.../datasource/iceberg/IcebergMetadataOpTest.java | 106 +
.../iceberg/IcebergMetadataOpsValidationTest.java | 6 +-
.../datasource/iceberg/IcebergTransactionTest.java | 439 ++++-
.../doris/datasource/iceberg/IcebergUtilsTest.java | 118 +-
.../iceberg/IcebergWriteSchemaContextTest.java | 824 ++++++++
.../iceberg/helper/IcebergWriterHelperTest.java | 37 +
.../iceberg/source/IcebergScanNodeTest.java | 1769 ++++++++++++++++-
.../datasource/paimon/PaimonMetadataOpsTest.java | 118 ++
.../java/org/apache/doris/mtmv/MTMVTaskTest.java | 52 +
.../test/java/org/apache/doris/mtmv/MTMVTest.java | 105 +
.../materialize/MaterializeProbeVisitorTest.java | 128 ++
.../rules/analysis/UserVariableAnalysisTest.java | 17 +
.../exploration/mv/NullRejectInferenceTest.java | 47 +
.../trees/plans/commands/ExecuteCommandTest.java | 23 +
.../plans/commands/IcebergDmlCommandUtilsTest.java | 47 +
.../plans/commands/IcebergMergeCommandTest.java | 16 +
.../commands/insert/IcebergMergeExecutorTest.java | 3 +-
.../plans/commands/insert/InsertUtilsTest.java | 95 +-
.../apache/doris/planner/IcebergMergeSinkTest.java | 103 +
.../doris/qe/ForceForwardAllQueriesTest.java | 77 +
fs_brokers/cdc_client/pom.xml | 3 +-
.../shyiko/mysql/binlog/BinaryLogClient.java | 1477 ++++++++++++++
.../mysql/MySqlStreamingChangeEventSource.java | 17 +-
.../apache/doris/cdcclient/common/Constants.java | 2 +-
.../source/reader/mysql/MySqlSourceReader.java | 5 +-
.../BinaryLogClientTransactionReplayTest.java | 135 ++
.../MySqlBinaryLogClientKeepAliveITCase.java | 208 ++
gensrc/thrift/ExternalTableSchema.thrift | 13 +-
gensrc/thrift/PlanNodes.thrift | 3 +
.../test_array_zip_array_enumerate_uniq.out | 15 +
.../timestamptz/test_timestamptz_dst_fold.out | 7 +
.../iceberg/test_iceberg_initial_defaults.out | 318 +++
.../iceberg/test_iceberg_write_default.out} | 4 +-
.../write/test_iceberg_write_evolution_refs.out | 20 +-
.../data/mtmv_p0/test_create_mtmv_with_view.out | 3 +-
.../mtmv_p0/test_excluded_trigger_table_mtmv.out | 6 +
.../test_array_zip_array_enumerate_uniq.groovy | 20 +
.../timestamptz/test_timestamptz_dst_fold.groovy | 11 +
.../iceberg/test_gen_iceberg_by_api.groovy | 2 +-
.../iceberg/test_iceberg_initial_defaults.groovy | 643 +++++++
.../iceberg/test_iceberg_jdbc_catalog.groovy | 10 +-
.../iceberg/test_iceberg_write_default.groovy | 78 +
.../write/test_iceberg_write_evolution_refs.groovy | 12 +-
.../paimon/test_paimon_alter_properties.groovy | 9 +
.../paimon/test_paimon_jdbc_catalog.groovy | 9 +-
.../test_excluded_trigger_table_mtmv.groovy | 6 +-
.../distribute/shuffle_left_join.groovy | 18 +-
156 files changed, 22303 insertions(+), 2258 deletions(-)
create mode 100644 be/src/format/table/iceberg_default_value.h
create mode 100644
be/test/exec/sink/writer/iceberg/viceberg_table_writer_test.cpp
create mode 100644 be/test/format/table/equality_delete_test.cpp
create mode 100644
docker/thirdparties/docker-compose/iceberg/scripts/java/CreateIcebergInitialDefaultFixtures.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergWriteSchemaContext.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/RewriteDefaultExpression.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/{BitNot.java
=> Default.java} (54%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/InternalCatalogTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/iceberg/IcebergWriteSchemaContextTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/qe/ForceForwardAllQueriesTest.java
create mode 100644
fs_brokers/cdc_client/src/main/java/com/github/shyiko/mysql/binlog/BinaryLogClient.java
create mode 100644
fs_brokers/cdc_client/src/test/java/com/github/shyiko/mysql/binlog/BinaryLogClientTransactionReplayTest.java
create mode 100644
fs_brokers/cdc_client/src/test/java/org/apache/doris/cdcclient/itcase/MySqlBinaryLogClientKeepAliveITCase.java
create mode 100644
regression-test/data/external_table_p0/iceberg/test_iceberg_initial_defaults.out
copy regression-test/data/{datatype_p0/bitmap/test_bitmap_const.out =>
external_table_p0/iceberg/test_iceberg_write_default.out} (62%)
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_initial_defaults.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_write_default.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
