This is an automated email from the ASF dual-hosted git repository.
caiconghui pushed a change to branch stream_load_error
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 31b1755b73b [fix](stream_load) fix stream load may failed caused by
column name with keyword
add b3c00561118 [fix](clone) fix calculate merge missing version (#35627)
add a6e99fb6e7d [refactor](cancel) disable cancel instance for pipeline
(#35632)
add a62fe7c26ac [feat](thrift) add NereidsId and cardinality to TPlanNode
(#35482)
add 2c958fd7b72 [opt](mv) calc Projection cost according to the
expressions in project list (#35652)
add 7b5bc731efc [Improvement](executor)Support kill query in be (#35602)
add a8df1d7e145 [Test](regression-test): Normalize S3 http url (#35666)
add cc1fef18005 [regression-test](load) add something like $.tag.[a.b]
key's json case (#35134)
add 6272315c402 [improve](txn-insert) Add txn insert p2 case (#35439)
add 42e173b987a [fix](binlog) Add is temp for UpsertRecord (#35636)
add b5034c20d36 [Performance](opt) opt the scan performance and remove
unless code (#35604)
add f5647edd869 [chore](third-party) Add a workflow: Build Third Party
Libraries (macOS-arm64) (#35705)
add 3c18026c762 [feature](statistics)Support clear stale partition stats.
(#35661)
add 35b17ee0e12 [chore](ASF) Make the workflow Build Third Party Libraries
(macOS-arm64) required (#35706)
add 75629133932 [fix](log) print log of error status (#35687)
add dc729311370 [Feature](Prepared Statment) Implement in nereids planner
(#35318)
add a939b59dd4c [opt](mtmv) Improve the mv rewrite performance by
optimize code usage (#35674)
add 7f8901578bd [fix](regression-test) Fix s3 load profile test failed due
to css schema change (#35633)
add 99388a08b29 [fix](segment cache) estimate momory consumed by segment
(#35647)
add 23c925591a5 [Fix](trino-connector) add regression tests for
trino-connector accessing Kafka. (#35282)
add a11a590368d [fix](pipeline) Fix query hang up if limited rows is
reached (#35513)
add 50446688891 [thirdparty](compatibility) restore dragonbox for compile
compatibility (#35605)
add f53aee4c8f3 [Fix](tvf) Pass through user-defined properties (#35515)
add e008bcfda22 [fix](decimal256) fix coredump when enable decimal256 but
fallback to old planner (#35731)
add d1c1c49d53c [regression-test](case) forbid unstable case (#35738)
add dd9beff5b72 [fix](orc)fix orc reader missing column and filter missing
column. (#35583)
add 3beb92b3b00 [bugfix](iceberg)fix datetime conversion error and data
path error (#35708)
add adfcbe86ceb (Fix)[hive-writer] Fixed the issue when partition values
contain spaces when writing to s3. (#35645)
add 8c797025992 [Fix](inverted index) fix fast execute condition for vexpr
(#35673)
add bf5736a8106 [Feature](function) support array_split and
array_reverse_split functions (#35619)
add 5368bb19b07 [feature](be jvm monitor)append enable_jvm_monitor in
be.conf to control jvm monitor. (#35608)
add 2ba385385a1 [Bug](load) fix s3 load not display the progress info
(#35719)
add cfd8ac628fa [fix](stream_load) fix stream load may failed caused by
column name with keyword
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (31b1755b73b)
\
N -- N -- N refs/heads/stream_load_error (cfd8ac628fa)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 1 +
.github/workflows/build-thirdparty.yml | 60 ++++-
be/src/cloud/cloud_tablet.cpp | 30 +++
be/src/cloud/cloud_tablet.h | 7 +
be/src/common/config.cpp | 4 +
be/src/common/config.h | 4 +
be/src/common/status.h | 2 +
be/src/olap/base_tablet.cpp | 34 +--
be/src/olap/base_tablet.h | 6 +-
be/src/olap/primary_key_index.cpp | 2 +
be/src/olap/rowset/rowset.cpp | 7 +
be/src/olap/rowset/rowset.h | 2 +-
be/src/olap/rowset/segment_v2/column_reader.cpp | 11 +-
be/src/olap/rowset/segment_v2/column_reader.h | 2 +-
.../rowset/segment_v2/indexed_column_reader.cpp | 8 +
.../olap/rowset/segment_v2/indexed_column_reader.h | 2 +
.../olap/rowset/segment_v2/ordinal_page_index.cpp | 14 ++
be/src/olap/rowset/segment_v2/ordinal_page_index.h | 2 +
be/src/olap/rowset/segment_v2/segment.cpp | 20 +-
be/src/olap/rowset/segment_v2/segment.h | 1 +
be/src/olap/rowset/segment_v2/zone_map_index.cpp | 11 +
be/src/olap/rowset/segment_v2/zone_map_index.h | 2 +
be/src/olap/short_key_index.cpp | 16 ++
be/src/olap/short_key_index.h | 1 +
be/src/olap/tablet.cpp | 33 +++
be/src/olap/tablet.h | 7 +
be/src/olap/tablet_schema.cpp | 10 +
be/src/olap/tablet_schema.h | 4 +-
be/src/pipeline/exec/exchange_sink_buffer.cpp | 41 ++--
be/src/pipeline/exec/exchange_sink_buffer.h | 11 +-
be/src/pipeline/exec/exchange_sink_operator.cpp | 9 +-
be/src/pipeline/exec/exchange_sink_operator.h | 3 +
be/src/pipeline/exec/file_scan_operator.cpp | 3 +-
be/src/pipeline/exec/jdbc_scan_operator.cpp | 3 +-
be/src/pipeline/exec/olap_scan_operator.h | 5 +-
be/src/pipeline/exec/operator.cpp | 18 +-
be/src/pipeline/exec/operator.h | 13 ++
be/src/pipeline/exec/scan_operator.cpp | 1 -
be/src/pipeline/exec/scan_operator.h | 1 -
.../local_exchange_source_operator.cpp | 8 +
be/src/pipeline/local_exchange/local_exchanger.cpp | 25 +++
be/src/pipeline/local_exchange/local_exchanger.h | 4 +
be/src/pipeline/pipeline_fragment_context.h | 8 +
be/src/pipeline/pipeline_task.cpp | 12 +-
be/src/pipeline/pipeline_task.h | 6 +
be/src/runtime/external_scan_context_mgr.cpp | 9 +-
be/src/runtime/external_scan_context_mgr.h | 3 +-
be/src/runtime/fragment_mgr.cpp | 91 +++-----
be/src/runtime/fragment_mgr.h | 10 +-
be/src/service/backend_service.cpp | 37 ++-
be/src/util/jvm_metrics.cpp | 112 +++++++--
be/src/util/jvm_metrics.h | 16 +-
be/src/vec/columns/column_array.cpp | 12 +-
be/src/vec/columns/column_array.h | 18 +-
be/src/vec/core/block.cpp | 1 -
be/src/vec/exec/format/orc/vorc_reader.cpp | 18 +-
be/src/vec/exec/scan/pip_scanner_context.h | 17 +-
be/src/vec/exec/scan/scanner_context.cpp | 55 ++---
be/src/vec/exec/scan/scanner_context.h | 2 +-
be/src/vec/exec/scan/scanner_scheduler.cpp | 16 +-
be/src/vec/exprs/vectorized_fn_call.cpp | 3 +-
be/src/vec/exprs/vexpr.h | 1 +
be/src/vec/exprs/vliteral.h | 2 +
.../vec/functions/array/function_array_popback.cpp | 6 +-
.../functions/array/function_array_register.cpp | 2 +
.../vec/functions/array/function_array_split.cpp | 157 +++++++++++++
.../sink/writer/iceberg/viceberg_table_writer.cpp | 17 +-
.../hive/scripts/create_preinstalled_table.hql | 21 +-
docker/thirdparties/run-thirdparties-docker.sh | 2 +-
.../org/apache/doris/catalog/MysqlColType.java | 15 ++
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 1 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 1 +
.../org/apache/doris/analysis/DateLiteral.java | 19 +-
.../doris/analysis/LambdaFunctionCallExpr.java | 31 ++-
.../java/org/apache/doris/binlog/UpsertRecord.java | 4 +
.../doris/catalog/BuiltinScalarFunctions.java | 4 +
.../java/org/apache/doris/catalog/OlapTable.java | 4 +
.../org/apache/doris/catalog/TempPartitions.java | 4 +
.../org/apache/doris/common/profile/Profile.java | 17 +-
.../org/apache/doris/common/util/DebugUtil.java | 24 ++
.../apache/doris/common/util/ProfileManager.java | 7 +-
.../java/org/apache/doris/common/util/S3URI.java | 39 +++-
.../java/org/apache/doris/common/util/Util.java | 24 --
.../doris/datasource/iceberg/IcebergUtils.java | 3 +-
.../main/java/org/apache/doris/mtmv/MTMVCache.java | 20 +-
.../org/apache/doris/nereids/NereidsPlanner.java | 7 +
.../org/apache/doris/nereids/StatementContext.java | 26 +++
.../org/apache/doris/nereids/cost/CostModelV1.java | 5 +-
.../glue/translator/PhysicalPlanTranslator.java | 14 +-
.../glue/translator/PlanTranslatorContext.java | 10 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 21 ++
.../nereids/rules/analysis/ExpressionAnalyzer.java | 8 +
.../mv/AbstractMaterializedViewRule.java | 28 +--
.../mv/AsyncMaterializationContext.java | 5 +-
.../mv/InitMaterializationContextHook.java | 10 +-
.../exploration/mv/MaterializationContext.java | 41 ++--
.../nereids/rules/exploration/mv/StructInfo.java | 60 ++---
.../nereids/trees/expressions/Placeholder.java | 78 +++++++
.../functions/scalar/ArrayReverseSort.java | 2 +-
...rrayReverseSort.java => ArrayReverseSplit.java} | 48 ++--
.../{ArrayReverseSort.java => ArraySplit.java} | 48 ++--
.../nereids/trees/expressions/literal/Literal.java | 169 ++++++++++++++
.../expressions/visitor/ExpressionVisitor.java | 5 +
.../expressions/visitor/ScalarFunctionVisitor.java | 10 +
.../doris/nereids/trees/plans/PlaceholderId.java | 58 +++++
.../apache/doris/nereids/trees/plans/PlanType.java | 5 +-
.../trees/plans/commands/ExecuteCommand.java | 83 +++++++
.../trees/plans/commands/PrepareCommand.java | 120 ++++++++++
.../java/org/apache/doris/planner/PlanNode.java | 1 +
.../java/org/apache/doris/qe/ConnectContext.java | 13 ++
.../java/org/apache/doris/qe/ConnectProcessor.java | 7 +-
.../org/apache/doris/qe/MysqlConnectProcessor.java | 145 +++++++++---
.../org/apache/doris/qe/PrepareStmtContext.java | 12 +-
...tContext.java => PreparedStatementContext.java} | 31 +--
.../java/org/apache/doris/qe/SessionVariable.java | 4 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 107 +++++----
.../apache/doris/statistics/AnalysisManager.java | 2 +-
.../apache/doris/statistics/ColumnStatistic.java | 4 +-
.../org/apache/doris/statistics/Statistics.java | 9 +
.../apache/doris/statistics/StatisticsCleaner.java | 80 +++----
.../doris/statistics/StatisticsRepository.java | 68 +-----
.../tablefunction/HdfsTableValuedFunction.java | 7 +-
.../doris/tablefunction/S3TableValuedFunction.java | 14 +-
.../doris/transaction/DatabaseTransactionMgr.java | 6 +-
.../doris/transaction/PartitionCommitInfo.java | 13 +-
.../apache/doris/transaction/TransactionEntry.java | 4 +-
.../org/apache/doris/analysis/DateLiteralTest.java | 15 ++
.../apache/doris/common/util/DebugUtilTest.java | 45 ++++
.../org/apache/doris/common/util/S3URITest.java | 13 ++
.../org/apache/doris/common/util/UtilTest.java | 46 ----
.../property/PropertyPassThroughTest.java | 68 ++++++
gensrc/script/doris_builtins_functions.py | 37 +++
gensrc/thrift/PlanNodes.thrift | 1 +
regression-test/conf/regression-conf.groovy | 3 +
.../hive/test_hive_orc_add_column.out | 185 +++++++++++++++
.../iceberg/test_iceberg_filter.out | 54 +++++
.../kafka/test_trino_kafka_base.out | 43 ++++
regression-test/data/insert_p0/txn_insert.out | 25 +++
.../data/inverted_index_p0/test_index_rqg_bug2.out | 19 ++
.../data/load_p0/stream_load/test_json_load.out | 3 +
.../load_p0/stream_load/test_special_key_json.json | 1 +
.../data/prepared_stmt_p0/prepared_stmt.out | 53 +++--
.../array_functions/test_array_split.out | 60 +++++
.../org/apache/doris/regression/suite/Suite.groovy | 33 +++
.../cloud_p0/conf/regression-conf-custom.groovy | 3 +
.../cloud_p1/conf/regression-conf-custom.groovy | 1 +
regression-test/pipeline/external/conf/be.conf | 3 +
.../pipeline/external/conf/regression-conf.groovy | 3 +
regression-test/pipeline/p0/conf/be.conf | 3 +
.../pipeline/p0/conf/regression-conf.groovy | 3 +
regression-test/pipeline/p1/conf/be.conf | 3 +
.../clone_p0/test_clone_missing_version.groovy | 88 ++++++++
.../export_p0/export/test_show_export.groovy | 4 +-
.../outfile/csv/test_outfile_csv_array_type.groovy | 28 +--
.../csv/test_outfile_csv_complex_type.groovy | 14 +-
.../outfile/csv/test_outfile_csv_map_type.groovy | 36 +--
.../csv/test_outfile_csv_one_nested_type.groovy | 26 +--
.../outfile/csv/test_outfile_empty_data.groovy | 2 +-
.../orc/test_outfile_orc_one_nested_type.groovy | 26 +--
.../outfile/orc/test_outfile_orc_timestamp.groovy | 2 +-
.../test_outfile_expr_generate_col_name.groovy | 4 +-
.../parquet/test_outfile_parquet_array_type.groovy | 28 +--
.../test_outfile_parquet_complex_type.groovy | 14 +-
.../parquet/test_outfile_parquet_map_type.groovy | 36 +--
.../test_outfile_parquet_one_nested_type.groovy | 24 +-
.../suites/export_p0/test_export_data_types.groovy | 10 +-
.../suites/export_p0/test_export_orc.groovy | 2 +-
.../suites/export_p0/test_export_parquet.groovy | 2 +-
.../suites/export_p0/test_export_view.groovy | 2 +-
.../export_p0/test_outfile_file_suffix.groovy | 4 +-
.../export_p0/test_outfile_orc_array_type.groovy | 28 +--
.../export_p0/test_outfile_orc_complex_type.groovy | 14 +-
.../export_p0/test_outfile_orc_map_type.groovy | 36 +--
.../suites/export_p0/test_with_bom.groovy | 8 +-
.../export_p2/test_export_with_parallelism.groovy | 2 +-
.../suites/export_p2/test_export_with_s3.groovy | 2 +-
.../hive/test_hive_orc_add_column.groovy | 95 ++++++++
.../iceberg/test_iceberg_filter.groovy | 102 +++++++++
.../kafka/data/trino_kafka_basic_data.csv | 20 ++
...trino_kafka_base_db.trino_kafka_basic_data.json | 100 +++++++++
.../kafka/test_trino_kafka_base.groovy | 93 ++++++++
.../test_insert_from_tvf_with_common_user.groovy | 2 +-
.../external_table_p0/tvf/test_s3_tvf.groovy | 4 +-
.../tvf/test_s3_tvf_with_resource.groovy | 4 +-
regression-test/suites/insert_p0/txn_insert.groovy | 52 +++++
regression-test/suites/insert_p2/txn_insert.groovy | 1 +
.../txn_insert_concurrent_insert_duplicate.groovy | 170 ++++++++++++++
.../txn_insert_concurrent_insert_ud.groovy | 183 +++++++++++++++
.../txn_insert_concurrent_insert_unique.groovy | 170 ++++++++++++++
.../txn_insert_concurrent_insert_update.groovy | 195 ++++++++++++++++
.../inverted_index_p0/test_index_rqg_bug2.groovy | 106 +++++++++
.../load_p0/stream_load/test_json_load.groovy | 26 +++
.../suites/load_p0/tvf/test_tvf_error_url.groovy | 8 +-
.../suites/point_query_p0/test_point_query.groovy | 5 +-
.../suites/prepared_stmt_p0/prepared_stmt.groovy | 249 +++++++++------------
.../array_functions/test_array_split.groovy | 81 +++++++
.../query_profile/s3_load_profile_test.groovy | 10 +-
...on_stats.groovy => test_partition_stats.groovy} | 73 +++++-
thirdparty/build-thirdparty.sh | 16 ++
thirdparty/vars.sh | 7 +
200 files changed, 4583 insertions(+), 1061 deletions(-)
create mode 100644 be/src/vec/functions/array/function_array_split.cpp
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/Placeholder.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayReverseSort.java
=> ArrayReverseSplit.java} (50%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayReverseSort.java
=> ArraySplit.java} (52%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/PlaceholderId.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ExecuteCommand.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/PrepareCommand.java
copy fe/fe-core/src/main/java/org/apache/doris/qe/{PrepareStmtContext.java =>
PreparedStatementContext.java} (56%)
delete mode 100644
fe/fe-core/src/test/java/org/apache/doris/common/util/UtilTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/datasource/property/PropertyPassThroughTest.java
create mode 100644
regression-test/data/external_table_p0/hive/test_hive_orc_add_column.out
create mode 100644
regression-test/data/external_table_p0/iceberg/test_iceberg_filter.out
create mode 100644
regression-test/data/external_table_p0/trino_connector/kafka/test_trino_kafka_base.out
create mode 100644
regression-test/data/inverted_index_p0/test_index_rqg_bug2.out
create mode 100644
regression-test/data/load_p0/stream_load/test_special_key_json.json
create mode 100644
regression-test/data/query_p0/sql_functions/array_functions/test_array_split.out
create mode 100644
regression-test/suites/clone_p0/test_clone_missing_version.groovy
create mode 100644
regression-test/suites/external_table_p0/hive/test_hive_orc_add_column.groovy
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_filter.groovy
create mode 100644
regression-test/suites/external_table_p0/trino_connector/kafka/data/trino_kafka_basic_data.csv
create mode 100644
regression-test/suites/external_table_p0/trino_connector/kafka/table_desc/test_trino_kafka_base_db.trino_kafka_basic_data.json
create mode 100644
regression-test/suites/external_table_p0/trino_connector/kafka/test_trino_kafka_base.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_duplicate.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_ud.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_unique.groovy
create mode 100644
regression-test/suites/insert_p2/txn_insert_concurrent_insert_update.groovy
create mode 100644
regression-test/suites/inverted_index_p0/test_index_rqg_bug2.groovy
create mode 100644
regression-test/suites/query_p0/sql_functions/array_functions/test_array_split.groovy
rename regression-test/suites/statistics/{test_show_partition_stats.groovy =>
test_partition_stats.groovy} (60%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
