This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from 29777bc3a8f [fix](fe)reduce memory usage in alter (#32810) (#33474)
new 4079a7b6abf [fix](txn insert) Fix txn insert into values for sequence
column or column name is keyword (#33336)
new d667df2d06d [improvement](spill) avoid unnecessary spilling in hash
join build phase (#33277)
new 741d4ff97eb [fix](group commit) Fix syntax error when insert into
table which column names contain keyword (#33322)
new 159ebc76e7f [fix](npe) fix kafka be id npe (#33151)
new 4eee1a1f0d5 [fix](nereids) make runtime filter targets in fixed order
(#33191)
new 6462264e771 [Improvement](materialized-view) adjust priority of
materialized view match rule (#33305)
new 0d0a96d097c [fix](test) Fix auto partition dbgen product data (#33313)
new 045dd05f2ae [fix](Nereids): don't transpose agg and join if join is
mark join (#33312)
new 28acfaed2b8 [fix](pipeline)group by and output is empty (#33192)
new 0e262ba0e4e [improvement](spill) improve cancel of spill and improve
log printing (#33229)
new 53505e903ba [scan](improvement) Adjust parallel scanners num (#33223)
new ecbd92204d1 [fix](Nereids) variant push down not work on slot without
table (#33356)
new e2ad7149c39 [feature](debug point) Add handler to debug point (#33350)
new 75f497976c0 [opt](Nereids) auto fallback when query unsupport table
type (#33357)
new e998ce8293d [Improvement](runtime-filter) support sync join node build
side's size to init bloom runtime filter (#32180)
new ac6cefa10ab [Fix](schema change) Fix schema change fault when add
complex type column (#31824)
new b47348fd047 [Improvement](executor)cancel query when a query is queued
(#33339)
new c00e26e2764 [Fix]fix insert overwrite non-partition table null pointer
exception (#33205)
new 5732226be1d [fix](partial-update) remove unnecessary DECHEK on
IndexChannel::num_rows_filtered (#33160)
new 3a35f76f629 [bugfix](profile) should use backend ip:heartbeat port as
key during merge profile (#33368)
new 1684591ffd9 [chore](test) let some case suitable for legacy planner
and nereids (#33352)
new bcda1d80c2c [Bug](load) fix stream load file on hll type mv column
(#33373)
new 1d0076e3a30 [improve](serde) support complex type in write/read pb
serde (#33124)
new 35d5c9c8973 [fix](Nereids): add order for constraint test (#33323)
new 6bfeb0c7fb1 [feature](debug point) add macro DBUG_RUN_CALLBACK (#33407)
new 4b60265744a [feature](expr) add type check when expr prepare (#33330)
new d4271064bdd Fix alter column stats without min max value deserialize
failure. (#33406)
new 0f34cc0e88f [log](chore) print isBad in Replica::toString() (#33427)
new 0a2a0350082 [fix](merge-cloud) fix possible data leak when creating
tables (#33416)
new 4b1cb17b402 [fix](cases) Add check status timeout for backup/restore
cases (#32975)
The 30 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/exec/exec_node.cpp | 12 +-
be/src/exprs/bloom_filter_func.h | 4 +-
be/src/exprs/runtime_filter.cpp | 457 +++++++++++--------
be/src/exprs/runtime_filter.h | 25 +-
be/src/exprs/runtime_filter_slots.h | 167 +++----
be/src/pipeline/exec/file_scan_operator.h | 2 +
be/src/pipeline/exec/hashjoin_build_sink.cpp | 86 ++--
be/src/pipeline/exec/hashjoin_build_sink.h | 10 +-
be/src/pipeline/exec/join_build_sink_operator.cpp | 3 +-
be/src/pipeline/exec/join_build_sink_operator.h | 1 +
be/src/pipeline/exec/olap_scan_operator.cpp | 1 -
.../exec/partitioned_aggregation_sink_operator.cpp | 41 +-
.../exec/partitioned_aggregation_sink_operator.h | 9 +-
.../partitioned_aggregation_source_operator.cpp | 36 +-
.../exec/partitioned_hash_join_probe_operator.cpp | 94 ++--
.../exec/partitioned_hash_join_probe_operator.h | 12 +-
.../exec/partitioned_hash_join_sink_operator.cpp | 219 +++++++--
.../exec/partitioned_hash_join_sink_operator.h | 17 +
be/src/pipeline/exec/scan_operator.cpp | 7 +-
be/src/pipeline/exec/scan_operator.h | 2 +
be/src/pipeline/exec/spill_sort_sink_operator.cpp | 144 +++---
.../pipeline/exec/spill_sort_source_operator.cpp | 54 ++-
be/src/pipeline/pipeline_fragment_context.cpp | 4 +-
be/src/pipeline/pipeline_x/dependency.cpp | 28 +-
be/src/pipeline/pipeline_x/dependency.h | 37 +-
be/src/pipeline/pipeline_x/operator.cpp | 5 +
be/src/pipeline/pipeline_x/operator.h | 2 +
.../pipeline_x/pipeline_x_fragment_context.cpp | 28 +-
be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 13 +-
be/src/runtime/fragment_mgr.cpp | 159 ++++---
be/src/runtime/fragment_mgr.h | 4 +
be/src/runtime/runtime_filter_mgr.cpp | 95 +++-
be/src/runtime/runtime_filter_mgr.h | 32 +-
be/src/service/backend_options.cpp | 1 +
be/src/service/internal_service.cpp | 45 +-
be/src/service/internal_service.h | 10 +
be/src/util/brpc_client_cache.h | 4 +
be/src/util/debug_points.h | 51 +++
be/src/vec/columns/column_array.cpp | 3 +-
be/src/vec/columns/column_map.cpp | 6 +-
be/src/vec/core/columns_with_type_and_name.h | 2 +-
.../vec/data_types/serde/data_type_array_serde.cpp | 30 ++
.../vec/data_types/serde/data_type_array_serde.h | 8 +-
.../vec/data_types/serde/data_type_decimal_serde.h | 7 +-
.../vec/data_types/serde/data_type_ipv4_serde.cpp | 20 +
be/src/vec/data_types/serde/data_type_ipv4_serde.h | 3 +
.../vec/data_types/serde/data_type_ipv6_serde.cpp | 24 +
be/src/vec/data_types/serde/data_type_ipv6_serde.h | 3 +
.../vec/data_types/serde/data_type_jsonb_serde.cpp | 28 ++
.../vec/data_types/serde/data_type_jsonb_serde.h | 3 +
.../vec/data_types/serde/data_type_map_serde.cpp | 37 ++
be/src/vec/data_types/serde/data_type_map_serde.h | 8 +-
.../data_types/serde/data_type_nullable_serde.cpp | 16 +-
.../vec/data_types/serde/data_type_number_serde.h | 35 +-
.../vec/data_types/serde/data_type_string_serde.h | 8 +-
.../data_types/serde/data_type_struct_serde.cpp | 71 ++-
.../vec/data_types/serde/data_type_struct_serde.h | 20 +-
be/src/vec/exec/join/vhash_join_node.cpp | 6 +-
be/src/vec/exec/join/vhash_join_node.h | 12 +-
be/src/vec/exec/join/vjoin_node_base.cpp | 1 +
be/src/vec/exec/join/vjoin_node_base.h | 1 +
be/src/vec/exec/scan/vscanner.cpp | 7 +-
be/src/vec/exec/vaggregation_node.cpp | 4 +-
be/src/vec/exec/vaggregation_node.h | 2 +
be/src/vec/exprs/vexpr.cpp | 35 ++
be/src/vec/exprs/vexpr.h | 3 +
.../vec/runtime/shared_hash_table_controller.cpp | 11 +-
be/src/vec/runtime/shared_hash_table_controller.h | 20 +-
be/src/vec/sink/load_stream_stub.cpp | 5 +-
be/src/vec/sink/writer/vtablet_writer.h | 1 -
be/src/vec/spill/spill_stream.cpp | 14 +-
be/src/vec/spill/spill_stream.h | 4 +-
be/src/vec/spill/spill_stream_manager.cpp | 2 -
be/src/vec/spill/spill_writer.cpp | 18 +-
be/src/vec/spill/spill_writer.h | 5 +-
be/src/vec/utils/util.hpp | 19 +-
be/test/util/debug_points_test.cpp | 23 +
.../data_types/serde/data_type_serde_pb_test.cpp | 492 ++++++++++++++++++++-
.../org/apache/doris/analysis/AddColumnClause.java | 4 +
.../java/org/apache/doris/analysis/ColumnDef.java | 12 +-
.../apache/doris/analysis/ModifyColumnClause.java | 4 +
.../apache/doris/analysis/NativeInsertStmt.java | 5 +
.../java/org/apache/doris/catalog/OlapTable.java | 8 +
.../java/org/apache/doris/catalog/Replica.java | 2 +
.../doris/common/profile/ExecutionProfile.java | 19 +-
.../apache/doris/datasource/InternalCatalog.java | 2 +
.../apache/doris/datasource/kafka/KafkaUtil.java | 8 +-
.../src/main/java/org/apache/doris/load/Load.java | 9 +-
.../nereids/rules/analysis/BindExpression.java | 3 +-
.../doris/nereids/rules/analysis/BindRelation.java | 6 +
.../rules/exploration/TransposeAggSemiJoin.java | 4 +-
.../exploration/TransposeAggSemiJoinProject.java | 4 +-
.../mv/AbstractSelectMaterializedIndexRule.java | 25 +-
.../scalar/PushDownToProjectionFunction.java | 8 +-
.../trees/plans/physical/RuntimeFilter.java | 3 +-
.../apache/doris/planner/GroupCommitPlanner.java | 2 +-
.../org/apache/doris/planner/RuntimeFilter.java | 4 +
.../main/java/org/apache/doris/qe/Coordinator.java | 21 +-
.../java/org/apache/doris/qe/QeProcessorImpl.java | 2 +-
.../java/org/apache/doris/qe/SessionVariable.java | 11 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 27 +-
.../doris/resource/workloadgroup/QueueToken.java | 25 +-
.../doris/statistics/StatisticsRepository.java | 6 +-
.../main/java/org/apache/doris/system/Backend.java | 9 +
.../exploration/TransposeAggSemiJoinTest.java | 18 +
.../rewrite/TransposeSemiJoinAggProjectTest.java | 15 +
gensrc/proto/internal_service.proto | 30 +-
gensrc/proto/types.proto | 7 +
gensrc/thrift/PlanNodes.thrift | 2 +
.../data_model_p0/unique/test_unique_table.out | 2 +-
.../unique/test_unique_table_new_sequence.out | 4 +-
.../data/insert_p0/insert_group_commit_into.out | 8 +
.../routine_load_hll.out} | 16 +-
.../test | 0
.../mvljc/mvljc.out => test_base/test_base.out} | 18 +-
.../load.out | 0
.../data/nereids_hint_tpcds_p0/shape/query24.out | 2 +-
.../data/nereids_hint_tpcds_p0/shape/query64.out | 4 +-
.../insert_into_table/partial_update.out | 36 ++
.../data/nereids_syntax_p0/constraint.out | 8 +-
.../shape/query14.out | 2 +-
.../shape/query17.out | 2 +-
.../nereids_tpcds_shape_sf1000_p0/shape/query2.out | 2 +-
.../shape/query24.out | 2 +-
.../shape/query25.out | 2 +-
.../shape/query29.out | 2 +-
.../nereids_tpcds_shape_sf1000_p0/shape/query4.out | 2 +-
.../nereids_tpcds_shape_sf1000_p0/shape/query5.out | 10 +-
.../shape/query64.out | 2 +-
.../shape/query71.out | 4 +-
.../noStatsRfPrune/query5.out | 6 +-
.../noStatsRfPrune/query64.out | 4 +-
.../noStatsRfPrune/query65.out | 2 +-
.../noStatsRfPrune/query71.out | 4 +-
.../no_stats_shape/query14.out | 2 +-
.../no_stats_shape/query17.out | 2 +-
.../no_stats_shape/query2.out | 2 +-
.../no_stats_shape/query24.out | 2 +-
.../no_stats_shape/query25.out | 2 +-
.../no_stats_shape/query29.out | 2 +-
.../no_stats_shape/query4.out | 2 +-
.../no_stats_shape/query5.out | 10 +-
.../no_stats_shape/query64.out | 4 +-
.../no_stats_shape/query65.out | 2 +-
.../no_stats_shape/query71.out | 4 +-
.../rf_prune/query5.out | 6 +-
.../rf_prune/query64.out | 2 +-
.../rf_prune/query71.out | 4 +-
.../nereids_tpcds_shape_sf100_p0/shape/query17.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query2.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query24.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query25.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query29.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query4.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query5.out | 10 +-
.../nereids_tpcds_shape_sf100_p0/shape/query64.out | 2 +-
.../nereids_tpcds_shape_sf100_p0/shape/query71.out | 4 +-
.../nostats_rf_prune/q20-rewrite.out | 2 +-
.../nostats_rf_prune/q20.out | 2 +-
.../nostats_rf_prune/q5.out | 2 +-
.../rf_prune/q20-rewrite.out | 2 +-
.../nereids_tpch_shape_sf1000_p0/rf_prune/q20.out | 2 +-
.../shape/q20-rewrite.out | 2 +-
.../nereids_tpch_shape_sf1000_p0/shape/q20.out | 2 +-
.../data/nereids_tpch_shape_sf1000_p0/shape/q5.out | 2 +-
.../shape_no_stats/q20-rewrite.out | 2 +-
.../shape_no_stats/q20.out | 2 +-
.../shape_no_stats/q5.out | 2 +-
.../apache/doris/regression/suite/Syncer.groovy | 33 ++
.../account_p0/test_nereids_authentication.groovy | 14 +-
.../backup_restore/test_MoW_backup_restore.groovy | 48 +-
.../backup_restore/test_backup_restore.groovy | 8 +-
.../test_backup_restore_alias.groovy | 8 +-
.../test_backup_restore_bloom_filter.groovy | 8 +-
.../backup_restore/test_backup_restore_db.groovy | 8 +-
...t_backup_restore_diff_repo_same_snapshot.groovy | 16 +-
..._backup_restore_dup_without_default_keys.groovy | 8 +-
.../test_backup_restore_exclude.groovy | 8 +-
.../test_backup_restore_inverted_index.groovy | 8 +-
.../test_backup_restore_keep_on_local.groovy | 4 +-
.../test_backup_restore_multi_tables.groovy | 8 +-
...st_backup_restore_multi_tables_overwrite.groovy | 8 +-
.../test_backup_restore_partition.groovy | 12 +-
..._restore_reserve_dynamic_partition_false.groovy | 10 +-
...p_restore_reserve_dynamic_partition_true.groovy | 9 +-
.../test_backup_restore_table_auto_bucket.groovy | 8 +-
.../backup_restore/test_backup_restore_with.groovy | 8 +-
..._backup_restore_with_two_phase_fetch_opt.groovy | 8 +-
.../test_duplicate_backup_restore.groovy | 20 +-
.../test_restore_mix_exists_and_new_table.groovy | 8 +-
.../test_restore_to_new_table.groovy | 8 +-
.../test_unique_backup_restore.groovy | 16 +-
.../ccr_mow_syncer_p1/test_backup_restore.groovy | 10 +-
.../ccr_syncer_p1/test_backup_restore.groovy | 10 +-
.../correctness_p0/test_group_having_alias.groovy | 3 +
.../data_model_p0/unique/test_unique_table.groovy | 8 +-
.../unique/test_unique_table_new_sequence.groovy | 14 +-
.../unique/test_unique_table_sequence.groovy | 16 +-
.../insert_overwrite_legacy_unpartitioned.groovy} | 28 +-
.../insert_p0/insert_group_commit_into.groovy | 36 ++
.../inverted_index_p0/test_null_index.groovy | 5 -
.../routine_load_hll.groovy} | 29 +-
.../test_base.groovy} | 19 +-
.../load.groovy | 0
.../insert_into_table/partial_update.groovy | 37 ++
.../suites/nereids_p0/test_prune_tablet_mv.groovy | 2 +-
.../suites/nereids_syntax_p0/constraint.groovy | 14 +-
.../auto_partition/sql/multi_thread_load.groovy | 12 +
.../sql/stress_test_insert_into.groovy | 1 +
.../diff_data/stress_test_diff_date_list.groovy | 1 -
.../window_functions/test_ntile_function.groovy | 4 +-
.../test_alter_table_replace.groovy | 2 +-
.../test_schema_change_duplicate.groovy | 7 +
.../test_schema_change_unique.groovy | 8 +
.../test_schema_change_unique_mow.groovy | 7 +
215 files changed, 2814 insertions(+), 1236 deletions(-)
copy regression-test/data/mv_p0/{k1ap2spa/k1ap2spa.out =>
routine_load_hll/routine_load_hll.out} (54%)
copy regression-test/data/mv_p0/{routine_load_mapping =>
routine_load_hll}/test (100%)
copy regression-test/data/mv_p0/{where/mvljc/mvljc.out =>
test_base/test_base.out} (79%)
rename regression-test/data/{nereids_shape_check =>
nereids_broadcast_shuffle_join}/load.out (100%)
copy
regression-test/suites/{datatype_p0/decimalv3/test_decimalv3_key/test_decimalv3_key.groovy
=> insert_overwrite_p0/insert_overwrite_legacy_unpartitioned.groovy} (62%)
copy
regression-test/suites/mv_p0/{routine_load_mapping/routine_load_mapping.groovy
=> routine_load_hll/routine_load_hll.groovy} (59%)
copy regression-test/suites/mv_p0/{test_substr/test_substr.groovy =>
test_base/test_base.groovy} (75%)
rename regression-test/suites/{nereids_shape_check =>
nereids_broadcast_shuffle_join}/load.groovy (100%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
