This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from 8198a31563d [fix](testcases) make all auto partition P2 cases
nonConcurrent (#31708) (#31821)
new a5b91276564 [refactor](jdbc catalog) split sqlserver jdbc executor
(#31679)
new e2ebf9d5662 [feature](Nereids) parallel output file (#31623)
new 33c356e6d82 fix in stats estimation (#31684)
new 3777ffb43f6 [enhancement](nereids)support null partition for list
partition (#31613)
new 25d1934289b [Feature](topn) support multiple topn filter on backend
(#31665)
new 0c71b666d76 [Bug](resource) fix npe on Resource read from json (#31723)
new a26b1eeee75 [fix](connections) fix connection hang after too many
connections (#31594)
new d7d8b4ee4a7 [fix](fe) Fix NPE in FrontendServiceImpl.loadTxnCommit if
table is dropped (#31715)
new 294cb1729d7 [fix](feut) fix TabletRepairAndBalanceTest (#31653)
new fa499cc2003 [Enhencement](env) Checking Master branch must use JDK17
(#31587)
new 31a03e7183c [fix](cloud) Fix regression case `test_nereids_row_policy`
run in cloud mode (#31726)
new 231768db0d5 [Performance](exec) Support runtime filter in <=> join
(#31754)
new cca07732616 [feature](function) round function defaults to rounding
normally
new 2a1c00f180b [docs](kafka-connector) Add doris-kafka-connector usage
documentation (#31729)
new 7c30cb20fd1 [Fix](partial update) Fix partial update load false when
schema includes auto increment column (#31725)
new 2c26a308d1f [fix](broker-load) fix broker load statement type
conversion failure (#31746)
new 5ed3c7eeef8 [opt](doc) update release verify (#31773)
new 7998da46919 [fix](cast) wrong result while cast const to double then
to string (#31657)
new ca59579bf69 fix](Nereids): support max_value in range partition
(#31721)
new 11903d29a1e [fix](jdbc catalog) fix close abort in sqlserver (#31718)
new 9d2e5f1c00b [feature](doris compose) doris compose use jdk 17 image
(#31775)
new 9541fb2602e [chore](ci) remove required of pipelinex (#31791)
new ad3308c8ab2 [fix](hive) support partition prune for
_HIVE_DEFAULT_PARTITION_ (#31736)
new d9943d03748 [chore](Nereids) change consumer rules name (#31777)
new a909bc36eec [fix](jdbc catalog) Change Druid Pool dependency to
version 1.2.5 (#31783)
new f2bece23d88 [chore](ci) add back required to test (#31795)
new 97640ee0e89 [test](leading) add leading tpc-ds regression test cases
(#31681)
new c98e93edc3c [chore](ci) add back pipelinex in master (#31799)
new 300f25c392c [chore](ci) remove required of pipelinex (#31806)
new 808563470f7 [pipelineX](debug) Refactor code and complete debug string
(#31733)
new cc8112273eb [fix](mtmv) Avoiding the occurrence of null pointers in
logs due to the deletion of MTMV (#31722)
new 7b51c4aacac [chore](fe) Speed up fe-core checkstyle (#31785)
new eea9b56f69f [fix](group commit) handle group commit create plan error
(#31757)
new 2d6e975d5a8 [fix](cast) fix wrong result while cast string to float
(#31781)
new 1434d3983bb [enhancement](test) Test the unique model by modify a key
type from TINYINT to other type (#31713)
new c43bc8349fc [fix](move-memtable) free resources when fail to open
stream (#31762)
new d94d2c65f66 [fix](Nereids) let OrToIn rewritten result have stable
order (#31731)
new 679c7657f6e [fix](cloud) Fix regression case `test_dynamic_partition`
run in cloud mode (#31803)
new 1d2d0bd411c [fix](update) Update set value should consider sequence
column (#31626)
new aba58b0f7b4 [test](Nereids) add grouping sets test (#31675)
new 2e9bd268cd8 [improvement](jdbc catalog) support sqlserver timestamp
type read (#31805)
new 0f1cbcc86aa [refactor](jdbc catalog) split postgresql jdbc Executor
(#31730)
new 3e1e6c8d764 [fix](cloud) Fix `version_p0` run in cloud mode (#31810)
new 9af64d848f8 [fix](pipelineX) fix error distribution in
DistinctStreamingAggOperatorX (#31804)
new 7f3a666facb [fix](planner) fix ifnull and nvl function with one
parameters exception message anbugious (#31808)
new 4f174c4fb99 [feature](function) Support for aggregate function
foreach combiner (#31526)
new 1dcb99519db [fix](move-memtable) use tuple desc from schema param for
delta writer (#31747)
The 47 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 1 -
be/src/exec/tablet_info.cpp | 12 +
be/src/exec/tablet_info.h | 2 +
be/src/exprs/bloom_filter_func.h | 11 +-
be/src/exprs/runtime_filter.cpp | 5 +
be/src/exprs/runtime_filter.h | 1 +
be/src/olap/delta_writer_v2.cpp | 6 +-
be/src/olap/iterators.h | 1 +
be/src/olap/partial_update_info.h | 3 +-
be/src/olap/rowset/beta_rowset_reader.cpp | 1 +
be/src/olap/rowset/rowset_reader_context.h | 1 +
be/src/olap/rowset/segment_v2/segment.cpp | 32 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 43 +-
be/src/olap/rowset/segment_v2/segment_writer.cpp | 28 +-
be/src/olap/rowset/segment_v2/segment_writer.h | 4 +-
.../rowset/segment_v2/vertical_segment_writer.cpp | 27 +-
.../rowset/segment_v2/vertical_segment_writer.h | 3 +
be/src/olap/rowset_builder.cpp | 4 +
be/src/olap/tablet_meta.cpp | 1 +
be/src/olap/tablet_reader.cpp | 9 +-
be/src/olap/tablet_reader.h | 1 +
be/src/olap/tablet_schema.cpp | 1 +
be/src/olap/tablet_schema.h | 11 +
be/src/olap/wal/wal_table.cpp | 38 +-
be/src/olap/wal/wal_table.h | 2 +-
.../exec/distinct_streaming_aggregation_operator.h | 2 +-
be/src/pipeline/exec/sort_sink_operator.cpp | 6 +-
be/src/pipeline/pipeline_fragment_context.cpp | 15 +-
be/src/pipeline/pipeline_x/dependency.cpp | 5 +-
.../pipeline_x/pipeline_x_fragment_context.cpp | 13 +-
be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 7 +-
be/src/runtime/fragment_mgr.cpp | 5 +-
be/src/runtime/group_commit_mgr.cpp | 22 +-
be/src/runtime/group_commit_mgr.h | 2 +-
be/src/runtime/query_context.h | 16 +-
be/src/runtime/runtime_state.cpp | 3 +
.../vec/aggregate_functions/aggregate_function.h | 2 +-
.../aggregate_function_approx_count_distinct.h | 2 +-
.../aggregate_functions/aggregate_function_avg.h | 2 +-
.../aggregate_function_avg_weighted.h | 2 +-
.../aggregate_function_binary.h | 2 +-
.../aggregate_functions/aggregate_function_bit.h | 2 +-
.../aggregate_function_bitmap.h | 4 +-
.../aggregate_function_bitmap_agg.h | 2 +-
.../aggregate_function_collect.h | 2 +-
.../aggregate_functions/aggregate_function_count.h | 4 +-
.../aggregate_function_count_by_enum.h | 2 +-
.../aggregate_functions/aggregate_function_covar.h | 2 +-
.../aggregate_function_distinct.h | 2 +-
.../aggregate_function_foreach.cpp | 64 ++
.../aggregate_function_foreach.h | 264 +++++++
.../aggregate_function_group_concat.h | 2 +-
.../aggregate_function_histogram.h | 2 +-
.../aggregate_function_hll_union_agg.h | 2 +-
.../aggregate_function_java_udaf.h | 2 +-
.../aggregate_functions/aggregate_function_map.h | 2 +-
.../aggregate_function_min_max.h | 2 +-
.../aggregate_function_min_max_by.h | 2 +-
.../aggregate_functions/aggregate_function_null.h | 4 +-
.../aggregate_function_orthogonal_bitmap.h | 2 +-
.../aggregate_function_percentile_approx.h | 10 +-
.../aggregate_function_product.h | 2 +-
.../aggregate_function_quantile_state.h | 2 +-
.../aggregate_function_reader_first_last.h | 2 +-
.../aggregate_function_retention.h | 2 +-
.../aggregate_functions/aggregate_function_rpc.h | 2 +-
.../aggregate_function_sequence_match.h | 2 +-
.../aggregate_function_simple_factory.cpp | 3 +
.../aggregate_function_simple_factory.h | 26 +-
.../aggregate_functions/aggregate_function_sort.h | 2 +-
.../aggregate_function_state_union.h | 2 +-
.../aggregate_function_stddev.h | 2 +-
.../aggregate_functions/aggregate_function_sum.h | 2 +-
.../aggregate_functions/aggregate_function_topn.h | 2 +-
.../aggregate_functions/aggregate_function_uniq.h | 2 +-
.../aggregate_function_window.h | 14 +-
.../aggregate_function_window_funnel.h | 2 +-
.../data_types/serde/data_type_nullable_serde.cpp | 2 +-
be/src/vec/exec/scan/new_olap_scanner.cpp | 6 +
be/src/vec/exec/vsort_node.cpp | 6 +-
be/src/vec/functions/round.h | 92 +--
be/src/vec/olap/vcollect_iterator.cpp | 4 +-
be/src/vec/runtime/vdata_stream_recvr.h | 14 +-
be/src/vec/sink/load_stream_stub.cpp | 4 +-
be/src/vec/sink/writer/vtablet_writer_v2.cpp | 2 +-
be/test/olap/wal/wal_manager_test.cpp | 2 -
bin/start_be.sh | 13 +-
bin/start_fe.sh | 12 +-
conf/fe.conf | 2 +-
docker/runtime/doris-compose/Dockerfile | 32 +-
docker/runtime/doris-compose/command.py | 28 +-
docker/runtime/doris-compose/utils.py | 28 +-
.../sqlserver/init/03-create-table.sql | 4 +
.../docker-compose/sqlserver/init/04-insert.sql | 2 +
.../community/release-and-verify/release-verify.md | 4 +-
docs/en/docs/ecosystem/doris-kafka-connector.md | 192 +++++
.../sql-functions/numeric-functions/round.md | 1 +
docs/sidebars.json | 1 +
.../community/release-and-verify/release-verify.md | 4 +-
docs/zh-CN/docs/ecosystem/doris-kafka-connector.md | 193 +++++
.../sql-functions/numeric-functions/round.md | 2 +-
env.sh | 86 ++-
.../org/apache/doris/jdbc/BaseJdbcExecutor.java | 13 +
.../org/apache/doris/jdbc/JdbcExecutorFactory.java | 4 +
.../org/apache/doris/jdbc/MySQLJdbcExecutor.java | 9 -
.../apache/doris/jdbc/PostgreSQLJdbcExecutor.java | 125 ++++
.../apache/doris/jdbc/SQLServerJdbcExecutor.java | 134 ++++
fe/fe-core/pom.xml | 9 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 2 +-
.../main/java/org/apache/doris/analysis/Expr.java | 1 +
.../org/apache/doris/analysis/FloatLiteral.java | 5 +
.../apache/doris/analysis/FunctionCallExpr.java | 2 +
.../org/apache/doris/analysis/OutFileClause.java | 2 +-
.../org/apache/doris/analysis/PartitionDesc.java | 3 -
.../org/apache/doris/analysis/PartitionValue.java | 21 +-
.../java/org/apache/doris/analysis/SlotRef.java | 2 +-
.../java/org/apache/doris/analysis/UpdateStmt.java | 14 +-
.../java/org/apache/doris/catalog/EsResource.java | 4 +
.../java/org/apache/doris/catalog/Function.java | 14 +
.../org/apache/doris/catalog/FunctionRegistry.java | 10 +-
.../java/org/apache/doris/catalog/HMSResource.java | 4 +
.../org/apache/doris/catalog/HdfsResource.java | 4 +
.../apache/doris/catalog/ListPartitionItem.java | 11 -
.../apache/doris/catalog/OdbcCatalogResource.java | 1 -
.../org/apache/doris/catalog/PartitionKey.java | 139 ++--
.../java/org/apache/doris/catalog/Resource.java | 3 +
.../java/org/apache/doris/catalog/S3Resource.java | 7 +-
.../org/apache/doris/catalog/SparkResource.java | 5 +
.../doris/datasource/hive/HiveMetaStoreCache.java | 2 -
.../jdbc/client/JdbcSQLServerClient.java | 1 +
.../apache/doris/job/extensions/mtmv/MTMVTask.java | 10 +-
.../org/apache/doris/load/loadv2/BulkLoadJob.java | 24 +-
.../org/apache/doris/mysql/AcceptListener.java | 107 +--
.../java/org/apache/doris/mysql/MysqlServer.java | 2 +-
.../glue/translator/ExpressionTranslator.java | 11 +
.../glue/translator/PhysicalPlanTranslator.java | 62 +-
.../doris/nereids/parser/LogicalPlanBuilder.java | 2 +
.../nereids/properties/RequestPropertyDeriver.java | 2 +-
.../org/apache/doris/nereids/rules/RuleType.java | 6 +-
.../rules/analysis/CollectJoinConstraint.java | 2 +-
.../rules/HiveDefaultPartitionEvaluator.java | 68 --
.../nereids/rules/expression/rules/OrToIn.java | 13 +-
.../rules/expression/rules/PartitionPruner.java | 9 +-
.../rules/expression/rules/SimplifyRange.java | 2 +-
.../rules/rewrite/CollectFilterAboveConsumer.java | 2 +-
.../rules/rewrite/CollectProjectAboveConsumer.java | 4 +-
.../doris/nereids/stats/FilterEstimation.java | 2 +-
...uilder.java => AggCombinerFunctionBuilder.java} | 34 +-
...UnionCombinator.java => ForEachCombinator.java} | 38 +-
.../functions/combinator/MergeCombinator.java | 4 +-
.../functions/combinator/StateCombinator.java | 4 +-
.../functions/combinator/UnionCombinator.java | 4 +-
.../trees/expressions/literal/FloatLiteral.java | 2 +-
.../nereids/trees/expressions/literal/Literal.java | 2 +
.../visitor/AggregateFunctionVisitor.java | 5 +
.../doris/nereids/trees/plans/algebra/Filter.java | 4 +-
.../trees/plans/commands/UpdateCommand.java | 13 +-
.../plans/commands/UpdateMvByPartitionCommand.java | 22 +-
.../trees/plans/commands/info/CreateTableInfo.java | 3 -
.../plans/commands/info/PartitionDefinition.java | 2 +-
.../trees/plans/physical/PhysicalFileSink.java | 17 +
.../org/apache/doris/planner/OlapTableSink.java | 14 +-
.../org/apache/doris/planner/OriginalPlanner.java | 39 +-
.../org/apache/doris/planner/ResultFileSink.java | 45 ++
.../apache/doris/service/FrontendServiceImpl.java | 10 +-
.../apache/doris/analysis/StmtRewriterTest.java | 12 +-
.../doris/clone/TabletRepairAndBalanceTest.java | 2 +-
.../doris/nereids/rules/rewrite/OrToInTest.java | 10 +
.../commands/UpdateMvByPartitionCommandTest.java | 56 ++
.../doris/planner/TableFunctionPlanTest.java | 6 +-
.../org/apache/doris/qe/OlapQueryCacheTest.java | 12 +-
fe/pom.xml | 2 +-
gensrc/proto/descriptors.proto | 1 +
gensrc/proto/olap_file.proto | 1 +
gensrc/thrift/Descriptors.thrift | 3 +-
gensrc/thrift/PaloInternalService.thrift | 1 +
gensrc/thrift/PlanNodes.thrift | 4 +
.../data/correctness/test_float_round_up.out | 13 +
.../test_distinct_streaming_agg_local_shuffle.out | 25 +
.../unique/auto_inc_partial_update3.csv | 5 +
.../unique/test_unique_table_auto_inc.out | 16 +
.../unique/test_unique_table_new_sequence.out | 20 +-
.../jdbc/test_sqlserver_jdbc_catalog.out | 7 +
.../fault_injection_p0/concurrency_update2.csv | 5 +
.../fault_injection_p0/concurrency_update3.csv | 5 +
..._partial_update_publish_conflict_with_error.out | 93 +--
.../data/function_p0/test_agg_foreach.out | 28 +
.../data/nereids_function_p0/scalar_function/D.out | 4 +-
.../data/nereids_function_p0/scalar_function/R.out | 4 +-
.../data/nereids_hint_tpcds_p0/shape/query1.out | 47 ++
.../data/nereids_hint_tpcds_p0/shape/query24.out | 65 ++
.../data/nereids_hint_tpcds_p0/shape/query64.out | 139 ++++
.../data/nereids_hint_tpcds_p0/shape/query67.out | 40 +
.../data/nereids_hint_tpcds_p0/shape/query72.out | 73 ++
.../data/nereids_hint_tpcds_p0/shape/query78.out | 65 ++
.../cast_function/test_cast_function.out | 6 +
.../test_grouping_sets_combination.out | 184 +++++
.../nereids_syntax_p0/list_partition_with_null.out | 15 +
regression-test/data/query_p0/join/test_join.out | 351 +++++++++
.../cast_function/test_cast_function.out | 6 +
.../sql_functions/math_functions/test_round.out | 24 +-
.../test_unique_model_schema_key_change.out | 50 ++
regression-test/data/update/test_update.out | 49 ++
.../issuesWithTheMostComments3.out | 2 +-
.../sql/issuesWithTheMostComments3.out | 2 +-
.../account_p0/test_nereids_row_policy.groovy | 20 +
.../test_float_round_up.groovy} | 26 +-
...est_distinct_streaming_agg_local_shuffle.groovy | 102 +++
.../unique/test_unique_table_auto_inc.groovy | 15 +-
.../suites/demo_p0/explain_action.groovy | 2 +-
.../suites/export_p0/test_outfile.groovy | 5 +-
.../jdbc/test_sqlserver_jdbc_catalog.groovy | 3 +
.../hive/test_hive_default_partition.groovy | 34 +-
...rtial_update_publish_conflict_with_error.groovy | 4 +-
.../suites/function_p0/test_agg_foreach.groovy | 95 +++
.../nereids_function_p0/scalar_function/N.groovy | 16 +-
.../ddl/gen_shape.py | 0
.../ddl/shape.tmpl | 0
.../suites/nereids_hint_tpcds_p0/load.groovy | 809 +++++++++++++++++++++
.../nereids_hint_tpcds_p0/shape/query1.groovy | 84 +++
.../nereids_hint_tpcds_p0/shape/query24.groovy | 143 ++++
.../nereids_hint_tpcds_p0/shape/query64.groovy | 279 +++++++
.../nereids_hint_tpcds_p0/shape/query67.groovy | 121 +++
.../nereids_hint_tpcds_p0/shape/query72.groovy | 91 +++
.../nereids_hint_tpcds_p0/shape/query78.groovy | 154 ++++
.../suites/nereids_p0/outfile/test_outfile.groovy | 5 +-
.../cast_function/test_cast_function.groovy | 2 +
.../test_grouping_sets_combination.groovy | 191 +++++
.../list_partition_with_null.groovy | 71 ++
.../test_dynamic_partition.groovy | 12 +
.../test_list_partition_datatype.groovy | 19 -
.../suites/query_p0/join/test_join.groovy | 19 +
.../cast_function/test_cast_function.groovy | 2 +
.../conditional_functions/test_ifnull.groovy | 9 +
.../conditional_functions/test_nvl.groovy | 8 +
.../test_unique_model_schema_key_change.groovy | 154 ++++
regression-test/suites/update/test_update.groovy | 41 +-
.../suites/version_p0/version_p0.groovy | 3 +-
238 files changed, 5885 insertions(+), 785 deletions(-)
create mode 100644
be/src/vec/aggregate_functions/aggregate_function_foreach.cpp
create mode 100644 be/src/vec/aggregate_functions/aggregate_function_foreach.h
create mode 100644 docs/en/docs/ecosystem/doris-kafka-connector.md
create mode 100644 docs/zh-CN/docs/ecosystem/doris-kafka-connector.md
create mode 100644
fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/PostgreSQLJdbcExecutor.java
create mode 100644
fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/SQLServerJdbcExecutor.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/HiveDefaultPartitionEvaluator.java
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/{AggStateFunctionBuilder.java
=> AggCombinerFunctionBuilder.java} (74%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/combinator/{UnionCombinator.java
=> ForEachCombinator.java} (67%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/UpdateMvByPartitionCommandTest.java
create mode 100644 regression-test/data/correctness/test_float_round_up.out
create mode 100644
regression-test/data/correctness_p0/test_distinct_streaming_agg_local_shuffle.out
create mode 100644
regression-test/data/data_model_p0/unique/auto_inc_partial_update3.csv
create mode 100644 regression-test/data/function_p0/test_agg_foreach.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query1.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query24.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query64.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query67.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query72.out
create mode 100644 regression-test/data/nereids_hint_tpcds_p0/shape/query78.out
create mode 100644
regression-test/data/nereids_rules_p0/grouping_sets/test_grouping_sets_combination.out
create mode 100644
regression-test/data/nereids_syntax_p0/list_partition_with_null.out
create mode 100644 regression-test/data/update/test_update.out
copy
regression-test/suites/{query_p0/sql_functions/conditional_functions/test_nvl.groovy
=> correctness/test_float_round_up.groovy} (66%)
create mode 100644
regression-test/suites/correctness_p0/test_distinct_streaming_agg_local_shuffle.groovy
create mode 100644 regression-test/suites/function_p0/test_agg_foreach.groovy
copy regression-test/suites/{nereids_tpcds_shape_sf1000_p0 =>
nereids_hint_tpcds_p0}/ddl/gen_shape.py (100%)
copy regression-test/suites/{nereids_tpcds_shape_sf1000_p0 =>
nereids_hint_tpcds_p0}/ddl/shape.tmpl (100%)
create mode 100644 regression-test/suites/nereids_hint_tpcds_p0/load.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query1.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query24.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query64.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query67.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query72.groovy
create mode 100644
regression-test/suites/nereids_hint_tpcds_p0/shape/query78.groovy
create mode 100644
regression-test/suites/nereids_rules_p0/grouping_sets/test_grouping_sets_combination.groovy
create mode 100644
regression-test/suites/nereids_syntax_p0/list_partition_with_null.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
