This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a change to branch dev_join
in repository https://gitbox.apache.org/repos/asf/doris.git
omit 42f5e7dc9be fix some bug of tpch (#25931)
omit 7e5b252274a update
omit e47d9502fcb update
omit 20eff6d34cb fix tpch some bug of right semi/anti join (#25896)
omit 9489914c8a7 support right anti/semi/outer join (#25785)
omit 6d8e2192baf update dev_join to pre calculate bucket num (#25663)
omit fe26648c685 support batch size (#25629)
omit 0c94dcbcaff update rf
omit f645fd48046 merge block to single block on join/set node
add 8a436d8eccc [FIX](collectiontype) fix shrink char column in map/struct
(#25725)
add 20fa1eff65c [enhancement](Nereids) Filter no data partition after
partition prune (#25456)
add 4bda1650e12 [docs & fix](stats) Fix tablesample init failed and some
outdated contents in docs (#25603)
add a919ef618dd [fix](planner) Fix `select table tablet` not effective
(#25378)
add e8f479882da [pipelineX](local exchange) Add local exchange operator
(#25846)
add 01d5901356c [fix](Nereids) cte should support nested name reuse
(#25858)
add 8a8ae44eee8 [Fix](regression)Fix statistics related regression test
(#25888)
add f31c1d858a2 [fix](merge-on-write) fix duplicate key in schema change
(#25705)
add 3e21e4bdc24 [feature](CANCEL-ALTER-SYSTEM)decommission backend by ids
(#25441)
add d70af6b6bef [Fix](regression)es catalog adds prefixes or suffixes to
avoid mutual influence (#25826)
add 018abdb6792 [fix](fe-ut) fix npe of fe ut (#25904)
add 04a70ecaf01 [fix](info_db) do not fetch external catalog's info in
information_schema (#25844)
add e783ef716f4 [fix](multi-table) fix unknown source slot descriptor when
load multi table (#25762)
add 7f66be84d5c [fix](Outfile) Infer the column name if the column is
expression in `select into outfile` (#25854)
add 5c191e5b084 [fix](case) fix out file for test_struct_functions
(#25910)
add 0fb57a6db88 [fix](meta) add sync new image timeout (#25768)
add 0dcbc5ee18b [fix](backup) missing use_path_style properties for minio
(#25803)
add 77bea97d404 [test](sync)add sync after insert in test case (#25911)
add f3e6bcebd7e [github](action) re-add "Add Scope Labeler" and add new
label "meta-change" (#25907)
add 4434b3f32ea [fix](move-memtable) use pthread mutex in LoadStreamMgr
(#25882)
add a079ba763c7 [fix](tvf) fix 'frontends()' inconsistent with return
'show proc /fro… (#25885)
add 77f727e0a18 [chore](compaction) Print roswet size when compaction
finishes successfully (#25891)
add da4de17d5cd [improvement](function) improve date_trunc function
performance when timeunit is const (#25824)
add 6dd60c6ebbf [Enhance](BE) Add -Wshadow-field compile option to avoid
unexpected shadowing behavior (#25698)
add 78165a3f6b6 [refine](pipelineX) use finish dependency in task (#25881)
add ad1313cce67 [refactor](pipelineX) refine _build_side_pipelines (#25871)
add 31d2a9a4f52 [Enhancement](function) support fractions for
convert_tz(datetimev2) (#25915)
add f322ad277d5 [fix](nereids) PushdownAliasThroughJoin may lost required
columns from parent node (#25835)
add d7d284b4f89 [fix](Nereids) fix `''` and `""` in string literal (#25752)
add 6e1a4dbda23 [Fix](predicate pushdown) Common expression not acting on
any slot should not be pushed down (#25901)
add 005a36322e5 [opt](index compaction) optimize checks before index
compaction (#25486)
add 855956e503d [opt](Nereids) let column dist info check msg same with
legacy planner (#25925)
add 9aa9894b111 [fix](Nereids) create table default value error msg is
diff with lagacy planner (#25897)
add e7a3cb079b5 [Enhance](regression)docker hive s3 file address is
determined based on the configuration (#25905)
add af3e6f604ae [fix](nereids) prune partition bug in pattern ColA <> ColB
#25769
add d896debbe74 [fix](stats) Escape string when insert stats (#25815)
new f6435141d7a merge block to single block on join/set node
new 82501bda3f7 update rf
new feecd5e3711 support batch size (#25629)
new d9ea1a1723e update dev_join to pre calculate bucket num (#25663)
new 78f91914363 support right anti/semi/outer join (#25785)
new 7b8d22c91c7 fix tpch some bug of right semi/anti join (#25896)
new 29e3377c9b0 update
new 4e16520bfd8 update
new bc1110750a3 fix some bug of tpch (#25931)
new 8b3a79cb5fc update
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (42f5e7dc9be)
\
N -- N -- N refs/heads/dev_join (8b3a79cb5fc)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 10 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../workflows/labeler/scope-label-conf.yml | 14 +-
.../workflows/scope-label.yml | 31 +-
be/CMakeLists.txt | 1 +
be/src/exprs/runtime_filter_slots.h | 5 +-
be/src/io/fs/s3_file_writer.h | 1 -
be/src/olap/compaction.cpp | 52 +++-
be/src/olap/full_compaction.cpp | 3 +-
.../olap/rowset/segment_v2/inverted_index_cache.h | 7 +
.../rowset/segment_v2/inverted_index_reader.cpp | 7 +
.../rowset/segment_v2/inverted_index_writer.cpp | 7 +
be/src/olap/rowset/segment_v2/ngram_bloom_filter.h | 8 +
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 40 ++-
be/src/olap/rowset/segment_v2/segment_iterator.h | 1 +
be/src/olap/rowset_builder.cpp | 16 +-
be/src/olap/schema_change.cpp | 13 +-
be/src/olap/storage_engine.cpp | 7 +
be/src/olap/tablet.cpp | 32 +-
be/src/olap/tablet.h | 2 +-
be/src/olap/tablet_meta.cpp | 5 -
be/src/olap/task/engine_clone_task.cpp | 10 +
be/src/olap/task/engine_publish_version_task.cpp | 32 +-
be/src/pipeline/exec/es_scan_operator.h | 8 +
be/src/pipeline/exec/exchange_sink_operator.cpp | 15 +-
be/src/pipeline/exec/exchange_sink_operator.h | 1 -
be/src/pipeline/exec/jdbc_table_sink_operator.cpp | 5 -
be/src/pipeline/exec/jdbc_table_sink_operator.h | 2 -
.../exec/multi_cast_data_stream_source.cpp | 2 +
.../pipeline/exec/multi_cast_data_stream_source.h | 14 +-
be/src/pipeline/exec/olap_scan_operator.h | 3 -
be/src/pipeline/exec/olap_table_sink_operator.h | 6 -
.../pipeline/exec/partition_sort_source_operator.h | 4 +-
be/src/pipeline/exec/result_file_sink_operator.cpp | 6 -
be/src/pipeline/exec/result_file_sink_operator.h | 2 -
be/src/pipeline/exec/scan_operator.cpp | 17 +-
be/src/pipeline/exec/scan_operator.h | 4 -
.../exec/streaming_aggregation_sink_operator.h | 4 -
be/src/pipeline/pipeline_fragment_context.cpp | 14 +-
be/src/pipeline/pipeline_fragment_context.h | 2 +
be/src/pipeline/pipeline_x/dependency.h | 80 ++++-
.../local_exchange_sink_operator.cpp | 87 ++++++
.../local_exchange/local_exchange_sink_operator.h | 102 +++++++
.../local_exchange_source_operator.cpp | 52 ++++
.../local_exchange_source_operator.h | 67 +++++
be/src/pipeline/pipeline_x/operator.cpp | 15 +-
be/src/pipeline/pipeline_x/operator.h | 23 +-
.../pipeline_x/pipeline_x_fragment_context.cpp | 104 ++++---
.../pipeline_x/pipeline_x_fragment_context.h | 36 ++-
be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 25 +-
be/src/pipeline/pipeline_x/pipeline_x_task.h | 33 +-
be/src/runtime/load_stream_mgr.cpp | 4 +-
be/src/runtime/load_stream_mgr.h | 2 +-
be/src/runtime/plan_fragment_executor.cpp | 14 +-
be/src/runtime/plan_fragment_executor.h | 2 +
be/src/runtime/runtime_state.h | 3 +
be/src/util/s3_util.h | 2 +-
.../aggregate_function_collect.h | 6 +-
.../aggregate_function_hll_union_agg.h | 5 +-
.../aggregate_function_java_udaf.h | 8 +-
.../aggregate_function_reader_first_last.h | 6 +-
.../aggregate_functions/aggregate_function_rpc.h | 8 +-
.../aggregate_function_state_merge.h | 10 +-
.../aggregate_function_state_union.h | 10 +-
.../aggregate_function_window.h | 6 +-
be/src/vec/aggregate_functions/helpers.h | 14 +-
be/src/vec/columns/column.h | 2 +
be/src/vec/columns/column_map.cpp | 21 ++
be/src/vec/columns/column_map.h | 4 +-
be/src/vec/columns/column_nullable.h | 2 +
be/src/vec/columns/column_struct.cpp | 15 +
be/src/vec/columns/column_struct.h | 4 +
be/src/vec/columns/column_vector.cpp | 13 +-
be/src/vec/common/hash_table/hash_map.h | 7 +-
be/src/vec/common/hash_table/hash_map_context.h | 41 ++-
be/src/vec/exec/format/csv/csv_reader.cpp | 24 +-
be/src/vec/exec/format/csv/csv_reader.h | 6 +-
.../vec/exec/join/process_hash_table_probe_impl.h | 2 +-
be/src/vec/exec/join/vhash_join_node.h | 6 +-
be/src/vec/exec/scan/new_es_scan_node.h | 8 +
be/src/vec/exec/scan/new_es_scanner.cpp | 10 +-
be/src/vec/exec/scan/new_es_scanner.h | 1 -
be/src/vec/exec/scan/new_jdbc_scanner.cpp | 4 +-
be/src/vec/exec/scan/new_jdbc_scanner.h | 2 -
be/src/vec/exec/scan/new_odbc_scanner.cpp | 5 +-
be/src/vec/exec/scan/new_odbc_scanner.h | 2 -
be/src/vec/exec/scan/new_olap_scan_node.h | 3 -
be/src/vec/exec/scan/pip_scanner_context.h | 8 +-
be/src/vec/exec/scan/vscan_node.cpp | 10 +-
be/src/vec/exec/scan/vscan_node.h | 2 -
be/src/vec/exprs/vexpr.h | 16 +
be/src/vec/functions/function_convert_tz.h | 39 ++-
be/src/vec/functions/function_timestamp.cpp | 139 ++++-----
be/src/vec/runtime/partitioner.cpp | 56 +++-
be/src/vec/runtime/partitioner.h | 29 +-
be/src/vec/runtime/vdatetime_value.cpp | 53 ++++
be/src/vec/runtime/vdatetime_value.h | 4 +
be/src/vec/runtime/vorc_transformer.cpp | 11 +-
be/src/vec/sink/vdata_stream_sender.cpp | 9 +-
be/src/vec/sink/vdata_stream_sender.h | 7 +
be/src/vec/sink/vmemory_scratch_sink.h | 2 -
be/src/vec/sink/writer/vfile_result_writer.cpp | 2 +-
be/src/vec/sink/writer/vtablet_writer.cpp | 3 +-
be/test/vec/function/function_time_test.cpp | 96 ++++--
docker/thirdparties/custom_settings.env | 2 +
.../docker-compose/hive/scripts/hive-metastore.sh | 8 +-
.../kafka/scripts/multi_table_csv1.csv | 3 +
.../kafka/scripts/multi_table_json1.json | 3 +
docker/thirdparties/run-thirdparties-docker.sh | 8 +-
docs/en/docs/query-acceleration/statistics.md | 8 +-
.../CANCEL-ALTER-SYSTEM.md | 14 +
docs/zh-CN/docs/query-acceleration/statistics.md | 8 +-
.../CANCEL-ALTER-SYSTEM.md | 14 +
.../main/java/org/apache/doris/common/Config.java | 9 +
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 4 +-
.../java/org/apache/doris/alter/SystemHandler.java | 61 ++--
.../doris/analysis/CancelAlterSystemStmt.java | 51 +++-
.../java/org/apache/doris/analysis/SelectStmt.java | 3 +-
.../java/org/apache/doris/analysis/TypeDef.java | 11 +-
.../main/java/org/apache/doris/catalog/Env.java | 3 +-
.../datasource/property/S3ClientBEProperties.java | 13 +
.../property/constants/S3Properties.java | 2 +
.../java/org/apache/doris/nereids/CTEContext.java | 10 +-
.../doris/nereids/jobs/executor/Rewriter.java | 2 +
.../doris/nereids/parser/LogicalPlanBuilder.java | 4 +-
.../org/apache/doris/nereids/rules/RuleType.java | 1 +
.../doris/nereids/rules/analysis/AnalyzeCTE.java | 23 +-
.../rules/TryEliminateUninterestedPredicates.java | 22 ++
.../nereids/rules/rewrite/PruneEmptyPartition.java | 42 +++
.../rules/rewrite/PushdownAliasThroughJoin.java | 9 +-
.../doris/nereids/trees/expressions/Multiply.java | 6 +-
.../functions/ComputePrecisionForSum.java | 6 +-
.../trees/expressions/functions/agg/Avg.java | 6 +-
.../trees/plans/commands/info/DefaultValue.java | 27 --
.../commands/info/DistributionDescriptor.java | 2 +-
.../trees/plans/logical/LogicalOlapScan.java | 1 -
.../apache/doris/nereids/types/DecimalV3Type.java | 24 +-
.../org/apache/doris/planner/OlapScanNode.java | 21 +-
.../java/org/apache/doris/qe/SessionVariable.java | 9 +
.../apache/doris/service/FrontendServiceImpl.java | 41 ++-
.../apache/doris/statistics/BaseAnalysisTask.java | 2 +-
.../org/apache/doris/statistics/ColStatsData.java | 4 +-
.../FrontendsTableValuedFunction.java | 1 +
.../org/apache/doris/analysis/SelectStmtTest.java | 137 ++++++++-
.../org/apache/doris/catalog/ColumnTypeTest.java | 2 +-
.../doris/clone/TabletRepairAndBalanceTest.java | 15 +
.../datasource/property/PropertyConverterTest.java | 125 +++++++-
.../rules/rewrite/PruneOlapScanPartitionTest.java | 2 +
.../rules/rewrite/PruneOlapScanTabletTest.java | 2 -
.../rewrite/PushdownAliasThroughJoinTest.java | 10 +-
gensrc/thrift/PaloInternalService.thrift | 4 +-
.../data/correctness/test_pushdown_common_expr.out | 24 ++
regression-test/data/correctness_p0/test_avg.out | 6 +-
.../test_nestedtypes_csv_insert_into_with_s3.out | 2 +-
.../test_nestedtypes_json_insert_into_with_s3.out | 6 +-
regression-test/data/export/test_struct_export.out | Bin 469 -> 442 bytes
.../test_outfile_expr_generate_col_name.out | 267 ++++++++++++++++
.../hive/test_hive_statistic_timeout.out | 7 -
.../data/insert_p0/test_struct_insert.out | Bin 444 -> 417 bytes
.../load_p0/routine_load/test_routine_load.out | 14 +-
.../data/load_p0/stream_load/map_char_test.csv | 4 +
.../data/load_p0/stream_load/test_stream_load.out | 6 +
.../data/nereids_p0/cte/test_cte_name_reuse.out | 10 +
.../datetime_functions/test_date_function.out | 23 ++
.../one_row_relation.out} | Bin 170 -> 152 bytes
.../aggregate/aggregate_group_by_metric_type.out | Bin 170 -> 154 bytes
.../struct_functions/test_struct_functions.out | Bin 1379 -> 1297 bytes
regression-test/pipeline/external/conf/fe.conf | 1 +
.../correctness/test_pushdown_common_expr.groovy | 69 ++++-
.../suites/correctness_p0/test_avg.groovy | 3 +-
.../test_outfile_expr_generate_col_name.groovy | 260 ++++++++++++++++
.../external_table_p0/es/test_es_query.groovy | 18 +-
.../es/test_es_query_no_http_url.groovy | 30 +-
.../tvf/test_frontends_tvf.groovy | 2 +-
.../hive/test_hive_analyze_db.groovy | 308 +++++++++----------
.../hive/test_hive_partition_statistic.groovy | 2 +-
.../hive/test_hive_statistic.groovy | 334 ++++++++++-----------
.../hive/test_hive_statistic_cache.groovy | 288 +++++++++---------
.../hive/test_hive_statistic_clean.groovy | 115 +++----
.../hive/test_hive_statistic_timeout.groovy | 20 +-
.../load_p0/routine_load/test_routine_load.groovy | 89 ++++++
.../load_p0/stream_load/test_stream_load.groovy | 40 ++-
.../nereids_p0/cte/test_cte_name_reuse.groovy | 60 ++++
.../datetime_functions/test_date_function.groovy | 25 ++
.../nereids_syntax_p0/one_row_relation.groovy | 3 +
183 files changed, 3422 insertions(+), 1194 deletions(-)
copy docker/thirdparties/custom_settings.env =>
.github/workflows/labeler/scope-label-conf.yml (69%)
copy docker/thirdparties/custom_settings.env =>
.github/workflows/scope-label.yml (55%)
create mode 100644
be/src/pipeline/pipeline_x/local_exchange/local_exchange_sink_operator.cpp
create mode 100644
be/src/pipeline/pipeline_x/local_exchange/local_exchange_sink_operator.h
create mode 100644
be/src/pipeline/pipeline_x/local_exchange/local_exchange_source_operator.cpp
create mode 100644
be/src/pipeline/pipeline_x/local_exchange/local_exchange_source_operator.h
create mode 100644
docker/thirdparties/docker-compose/kafka/scripts/multi_table_csv1.csv
create mode 100644
docker/thirdparties/docker-compose/kafka/scripts/multi_table_json1.json
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PruneEmptyPartition.java
create mode 100644
regression-test/data/export_p0/outfile/outfile_expr/test_outfile_expr_generate_col_name.out
delete mode 100644
regression-test/data/external_table_p2/hive/test_hive_statistic_timeout.out
create mode 100644 regression-test/data/load_p0/stream_load/map_char_test.csv
create mode 100644 regression-test/data/nereids_p0/cte/test_cte_name_reuse.out
copy
regression-test/data/{query_p0/aggregate/aggregate_group_by_metric_type.out =>
nereids_syntax_p0/one_row_relation.out} (56%)
create mode 100644
regression-test/suites/export_p0/outfile/outfile_expr/test_outfile_expr_generate_col_name.groovy
create mode 100644
regression-test/suites/nereids_p0/cte/test_cte_name_reuse.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
