This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-1.2-unstable
in repository https://gitbox.apache.org/repos/asf/doris.git
omit 9de1fec6cc [temp](statistics) disable statistic tables
add cf5e2a2eb6 [fix](nereids) new statistics use wrong default selectivity
(#14233)
add 47326f951d [fix](nereids) count(*) reports npe when do filter
selectivity estimation (#14235)
add bdf7d2779a [fix](Nereids) aggregate always report has 1 row count
(#14236)
add 8dd2f8b349 [enhancement](nereids) set Ndv=rowCount if ndv is almost
equal to rowCount on ColumnStatisitics load (#14238)
add bea66e6a12 [fix](nereids) cannot generate RF on colocate join and
prune useful RF in RF prune (#14234)
add b0ff852d74 [opt](Nereids) right deep tree penalty adjust: use right
rowCount, not abs(left - right) (#14239)
add 37fdd011b4 [fix](fe-metric) Prometheus read format error #13831
(#13832)
add 93e5d8e660 [Vectorized](function) support bitmap_from_array function
(#14259)
add cffdeff4ec [fix](memory) Fix memory leak by calling boost::stacktrace
(#14269)
add 215a4c6e02 [Bug](BHJ) Fix wrong result when use broadcast hash join
for naaj (#14253)
add 89db3fee00 [feature-wip](MTMV)Add show statement for MTMV (#13786)
add 6cc5ae077e [Improvement](Sequence function) Capitalize const variables
(#14270)
add a3062c662c [feature-wip](statistics) support statistics injection and
show statistics (#14201)
add 5ae046b208 [bugfix](log) fix wrong print introduced by 49fecd2a6dae
#14266
add f86886f8f5 [Feature](function) Support array_compact function (#14141)
add 87544a017f [fuzztest](fe session variable) add fuzzy test config for
fe session variables. (#14272)
add e298696baf [Chore](env) add error information when DORIS_GCC_HOME not
set well (#14249)
add a45685d028 [fix](regression) concurrent regression cases may fail
#14271
add 333c6390ee [fix](be-ut) AddressSanitizer detects container-overflow
issues (#14255)
add 6d2e6d85d3 [enhancement](be)release memory in Node's close() method
(#14258)
add 5badd70db2 [fix](csv-reader) Fix core dump when load text into doris
with special delimiter (#14196)
add 70cc725649 [Vectorized](function) support
avg_weighted/percentile_array/topn_wei… (#14209)
add 9d70c531a3 [improvement](publish) fix publish timeout in cocurrent
load (#14231)
add 3ea9d3f2e1 [enhancement](array) support read list(Array) type from orc
file (#14132)
add 442b844b22 [regressiontest](delete)delete-where-in-test (#14036)
add 20634ab7e3 [feature-wip](multi-catalog) support partition&missing
columns in parquet lazy read (#14264)
add 6881989dd9 [Bug](jvm memory) Support multiple java version to get max
heap size (#14295)
add 47a6373e0a [feature](Nereids) support datev2 and datetimev2 type
(#14263)
add 943e014414 [enhancement](decommission) speed up decommission process
(#14028) (#14006)
add 3259fcb790 [typo](docs) fix docs kafka-load.md (#14313)
add 7182f14645 [improvement][fix](multi-catalog) speed up list partition
prune (#14268)
add 0bf6d1fd79 [typo](doc)Datax doris writer doc update (#14328)
add afc9065b51 [test](nereids) add filter estimation ut cases (#14293)
add a4d4fc8c02 datax doris writer doc fix (#14344)
add af462b07c7 [enhancement](explain) compress descriptor table explain
string (#14152)
add a9e53e5c86 [improvement](test) add conf for pipline (#14254)
add 6da2948283 [feature-wip](multi-catalog) support iceberg v2(step 1)
(#13867)
add dba19e591c [cherry-pick](scanner) using avg rowset to calculate batch
size instead of using total_bytes since it costs a lot of cpu (#14345)
add a382bb95e7 [fix](runtimefilter) fix heap-user-after-free of runtime
filter merge (#14362)
add 98956dfa19 [fix](statistics) statistics inaccurate after analyze same
table more than once (#14279)
add 44ee4386f7 [test](multi-catalog)Regression test for external hive orc
table (#13762)
add ccf4db394c [feature-wip](multi-catalog) Collect external table
statistics (#14160)
add 8fe5211df4 [improvement](multi-catalog)(cache) invalidate catalog
cache when refresh (#14342)
add d5af4f6558 [Neried](Profile) Add projection timer for neried (#14286)
add 50bfd99b59 [feature](join) support nested loop semi/anti join (#14227)
add 1a035e2073 [fix](profile)(AggNode) fix the GetResultsTime is always
zero (#14366)
add fb140d0180 [Enhancement](sequence-column) optimize the use of sequence
column (#13872)
add bd5a593403 [enhancement](memtracker) Use proc/meminfo MemAvailable to
control memory and optimize MemTracker log printing (#14335)
new 6ac4dd861b [temp](statistics) disable statistic tables
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9de1fec6cc)
\
N -- N -- N refs/heads/branch-1.2-unstable (6ac4dd861b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/common/config.h | 2 +-
be/src/common/status.cpp | 9 +-
be/src/exec/arrow/arrow_reader.cpp | 3 +
be/src/exec/arrow/arrow_reader.h | 1 +
be/src/exec/arrow/orc_reader.cpp | 2 +-
be/src/exec/broker_scanner.cpp | 2 +-
be/src/exec/exec_node.cpp | 5 +-
be/src/exec/exec_node.h | 1 +
be/src/exec/hash_table.cpp | 3 +-
be/src/exec/olap_scanner.cpp | 13 +-
be/src/exec/olap_scanner.h | 3 +
be/src/exec/text_converter.cpp | 164 +++++++-
be/src/exec/text_converter.h | 5 +
be/src/exprs/expr.cpp | 2 +-
be/src/olap/rowset/segment_v2/segment.cpp | 12 +-
be/src/olap/rowset/segment_v2/segment.h | 2 +
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 6 +-
be/src/olap/storage_engine.cpp | 37 +-
be/src/runtime/buffer_control_block.h | 2 +
be/src/runtime/fragment_mgr.cpp | 16 +
be/src/runtime/memory/mem_tracker_limiter.cpp | 50 ++-
be/src/runtime/memory/mem_tracker_limiter.h | 29 +-
be/src/runtime/runtime_filter_mgr.h | 4 +-
be/src/runtime/thread_context.h | 1 +
be/src/service/doris_main.cpp | 7 +-
be/src/service/internal_service.cpp | 3 +-
be/src/util/frame_of_reference_coding.cpp | 2 +-
be/src/util/mem_info.cpp | 86 ++--
be/src/util/mem_info.h | 19 +-
be/src/util/stack_util.cpp | 5 +
be/src/vec/CMakeLists.txt | 5 +-
.../aggregate_function_avg_weighted.cpp | 53 +++
.../aggregate_function_avg_weighted.h | 111 ++++++
.../aggregate_function_percentile_approx.cpp | 12 +-
.../aggregate_function_percentile_approx.h | 160 ++++++--
.../aggregate_function_sequence_match.cpp | 4 +-
.../aggregate_function_sequence_match.h | 10 +-
.../aggregate_function_simple_factory.cpp | 2 +
.../aggregate_function_topn.cpp | 58 +--
.../aggregate_functions/aggregate_function_topn.h | 73 +++-
be/src/vec/common/allocator.h | 16 +-
be/src/vec/common/pod_array.h | 2 +-
be/src/vec/exec/format/csv/csv_reader.cpp | 2 +-
be/src/vec/exec/format/generic_reader.h | 19 +
.../exec/format/parquet/vparquet_column_reader.cpp | 23 +-
.../exec/format/parquet/vparquet_column_reader.h | 10 +-
.../exec/format/parquet/vparquet_group_reader.cpp | 156 ++++++--
.../exec/format/parquet/vparquet_group_reader.h | 54 ++-
be/src/vec/exec/format/parquet/vparquet_reader.cpp | 230 ++++++++---
be/src/vec/exec/format/parquet/vparquet_reader.h | 41 +-
be/src/vec/exec/format/table/iceberg_reader.cpp | 168 ++++++++
be/src/vec/exec/format/table/iceberg_reader.h | 64 +++
.../exec/format/table/table_format_reader.cpp} | 19 +-
be/src/vec/exec/format/table/table_format_reader.h | 47 +++
be/src/vec/exec/join/vhash_join_node.cpp | 77 ++--
be/src/vec/exec/join/vhash_join_node.h | 14 +-
be/src/vec/exec/join/vjoin_node_base.cpp | 9 +-
be/src/vec/exec/join/vjoin_node_base.h | 8 +
be/src/vec/exec/join/vnested_loop_join_node.cpp | 73 +++-
be/src/vec/exec/join/vnested_loop_join_node.h | 7 +-
be/src/vec/exec/scan/new_es_scan_node.cpp | 2 +-
be/src/vec/exec/scan/new_es_scanner.cpp | 6 +-
be/src/vec/exec/scan/new_es_scanner.h | 2 +-
be/src/vec/exec/scan/new_jdbc_scanner.cpp | 2 +-
be/src/vec/exec/scan/new_jdbc_scanner.h | 4 +-
be/src/vec/exec/scan/vfile_scanner.cpp | 89 ++++-
be/src/vec/exec/scan/vfile_scanner.h | 1 +
be/src/vec/exec/scan/vscan_node.cpp | 1 +
be/src/vec/exec/vaggregation_node.cpp | 181 +++++----
be/src/vec/exec/vaggregation_node.h | 34 +-
be/src/vec/exec/vanalytic_eval_node.cpp | 20 +-
be/src/vec/exec/vanalytic_eval_node.h | 4 +-
be/src/vec/exec/vexcept_node.cpp | 4 +-
be/src/vec/exec/vintersect_node.cpp | 4 +-
be/src/vec/exec/vrepeat_node.cpp | 8 +-
be/src/vec/exec/vrepeat_node.h | 2 +
be/src/vec/exec/vset_operation_node.cpp | 51 ++-
be/src/vec/exec/vset_operation_node.h | 10 +-
be/src/vec/exec/vsort_node.cpp | 3 +-
.../functions/array/function_array_compact.cpp} | 18 +-
.../vec/functions/array/function_array_compact.h | 110 ++++++
.../functions/array/function_array_register.cpp | 2 +
be/src/vec/functions/function_bitmap.cpp | 82 +++-
be/src/vec/runtime/shared_hash_table_controller.h | 10 +
be/src/vec/sink/vresult_sink.cpp | 1 +
be/test/vec/exec/parquet/parquet_reader_test.cpp | 8 +-
be/test/vec/exec/parquet/parquet_thrift_test.cpp | 10 +-
.../docker-compose/hive/scripts/create.hql | 32 +-
.../docker-compose/hive/scripts/hive-metastore.sh | 2 +
...0032_9ch4x_5b6acedf-02e6-4b4e-9de2-ae050ddb7c35 | Bin 0 -> 36054 bytes
...0032_9ch4x_f45472a3-a935-4ff7-9340-b977d51ef3a2 | Bin 0 -> 37278 bytes
...0032_9ch4x_e5a56983-f7e1-4b66-8d7d-134cd8a3b5d4 | Bin 0 -> 35762 bytes
...0032_9ch4x_d3313b2c-f4da-468a-b3b0-7627b7dd6398 | Bin 0 -> 34950 bytes
...0032_9ch4x_91eb689b-3053-4630-9f72-7e068afd3302 | Bin 0 -> 34925 bytes
...0032_9ch4x_fbdd6369-e923-47ff-af40-d023687b7d88 | Bin 0 -> 37973 bytes
...0032_9ch4x_02ec7d8c-ecb8-49f4-ad32-6a0fa0915743 | Bin 0 -> 34718 bytes
...0032_9ch4x_311f132e-20d2-4e1a-82c1-c0edbc3e7903 | Bin 0 -> 31367 bytes
...0032_9ch4x_feb65e62-844e-4890-b326-3aa48ceb917f | Bin 0 -> 33435 bytes
...0032_9ch4x_2f09f727-20f2-4e4c-adec-6a624df8c9f5 | Bin 0 -> 37453 bytes
...0032_9ch4x_de0f05f9-2fd4-4bd6-886a-434ac3349ca1 | Bin 0 -> 34090 bytes
...0032_9ch4x_099556fa-095b-4a70-8718-912c59154582 | Bin 0 -> 32184 bytes
...0032_9ch4x_3340eb7c-fb57-4cc4-a3ed-fc55eb2674f9 | Bin 0 -> 36891 bytes
...0032_9ch4x_ef0c50af-d5c4-4375-ad11-106c64ca7903 | Bin 0 -> 36050 bytes
...0032_9ch4x_45fd0f90-ca2f-4f89-832b-5b99395e7081 | Bin 0 -> 34195 bytes
...0032_9ch4x_d7af1778-dd22-4e08-804d-be1e462fca25 | Bin 0 -> 36132 bytes
...0032_9ch4x_78760342-f3b8-415c-aaf5-be7a7ed38b4c | Bin 0 -> 36163 bytes
...0032_9ch4x_bbb15063-ee62-4fd3-9602-3e9ce04f72ef | Bin 0 -> 35750 bytes
docker/thirdparties/start-thirdparties-docker.sh | 2 +-
.../import/import-scenes/kafka-load.md | 2 +-
.../update-delete/sequence-column-manual.md | 36 +-
docs/en/docs/ecosystem/datax.md | 207 +++++++++-
.../docs/ecosystem/external-table/jdbc-of-doris.md | 7 +-
.../aggregate-functions/avg_weighted.md} | 43 +-
.../aggregate-functions/percentile.md | 2 +-
.../aggregate-functions/percentile_array.md | 52 +++
.../aggregate-functions/topn_array.md | 8 +-
.../{topn_array.md => topn_weighted.md} | 35 +-
.../sql-functions/array-functions/array_compact.md | 81 ++++
.../bitmap-functions/bitmap_from_array.md | 53 +++
.../Create/CREATE-TABLE.md | 8 +
.../{REFRESH-CATALOG.md => REFRESH.md} | 35 +-
docs/sidebars.json | 6 +-
.../import/import-scenes/kafka-load.md | 2 +-
.../update-delete/sequence-column-manual.md | 31 +-
docs/zh-CN/docs/ecosystem/datax.md | 208 +++++++++-
.../docs/ecosystem/external-table/jdbc-of-doris.md | 6 +
.../aggregate-functions/avg_weighted.md} | 44 +--
.../aggregate-functions/percentile.md | 2 +-
.../{percentile.md => percentile_array.md} | 39 +-
.../aggregate-functions/topn_array.md | 6 +-
.../{topn_array.md => topn_weighted.md} | 34 +-
.../sql-functions/array-functions/array_compact.md | 81 ++++
.../bitmap-functions/bitmap_from_array.md | 53 +++
.../Create/CREATE-TABLE.md | 8 +
.../{REFRESH-CATALOG.md => REFRESH.md} | 33 +-
env.sh | 12 +-
fe/fe-core/src/main/cup/sql_parser.cup | 52 ++-
.../apache/doris/alter/SchemaChangeHandler.java | 6 +
.../java/org/apache/doris/alter/SystemHandler.java | 24 +-
.../doris/analysis/AlterColumnStatsStmt.java | 10 +-
.../org/apache/doris/analysis/AnalyzeStmt.java | 15 +-
.../org/apache/doris/analysis/DataDescription.java | 4 +
.../org/apache/doris/analysis/DescriptorTable.java | 8 +-
.../java/org/apache/doris/analysis/InsertStmt.java | 7 +-
.../org/apache/doris/analysis/MVRefreshInfo.java | 10 +-
.../apache/doris/analysis/ShowColumnStatsStmt.java | 84 +++-
.../org/apache/doris/analysis/ShowMTMVJobStmt.java | 138 +++++++
.../apache/doris/analysis/ShowMTMVTaskStmt.java | 137 +++++++
.../org/apache/doris/analysis/SlotDescriptor.java | 16 +-
.../org/apache/doris/analysis/StmtRewriter.java | 30 +-
.../org/apache/doris/analysis/TupleDescriptor.java | 8 +-
.../apache/doris/catalog/CatalogRecycleBin.java | 50 +++
.../main/java/org/apache/doris/catalog/Env.java | 63 +--
.../java/org/apache/doris/catalog/FunctionSet.java | 21 +
.../java/org/apache/doris/catalog/OlapTable.java | 23 ++
.../org/apache/doris/catalog/RefreshManager.java | 2 +-
.../main/java/org/apache/doris/catalog/Table.java | 8 +
.../java/org/apache/doris/catalog/TableIf.java | 19 +
.../org/apache/doris/catalog/TableProperty.java | 10 +
.../doris/catalog/external/EsExternalTable.java | 2 +-
.../doris/catalog/external/ExternalDatabase.java | 3 +-
.../doris/catalog/external/ExternalTable.java | 11 +-
.../doris/catalog/external/HMSExternalTable.java | 95 +++--
.../main/java/org/apache/doris/common/Config.java | 74 ++--
.../apache/doris/common/util/PropertyAnalyzer.java | 15 +
.../org/apache/doris/datasource/CatalogMgr.java | 4 +-
.../apache/doris/datasource/ExternalCatalog.java | 3 +-
.../doris/datasource/ExternalMetaCacheMgr.java | 36 +-
.../doris/datasource/ExternalSchemaCache.java | 21 +-
.../apache/doris/datasource/InternalCatalog.java | 48 ++-
.../datasource/PooledHiveMetaStoreClient.java | 19 +
.../doris/datasource/hive/HiveMetaStoreCache.java | 100 ++++-
.../doris/external/elasticsearch/EsUtil.java | 6 +-
.../org/apache/doris/journal/JournalEntity.java | 16 +-
.../java/org/apache/doris/metric/MetricRepo.java | 2 +-
.../doris/metric/PrometheusMetricVisitor.java | 36 +-
.../java/org/apache/doris/monitor/jvm/JvmInfo.java | 24 +-
.../java/org/apache/doris/mtmv/MTMVJobFactory.java | 73 +++-
.../java/org/apache/doris/mtmv/MTMVJobManager.java | 41 +-
.../org/apache/doris/mtmv/MTMVTaskExecutor.java | 11 +-
.../apache/doris/mtmv/MTMVTaskExecutorPool.java | 4 +-
.../org/apache/doris/mtmv/MTMVTaskManager.java | 51 ++-
.../main/java/org/apache/doris/mtmv/MTMVUtils.java | 21 +
.../apache/doris/mtmv/metadata/ChangeMTMVJob.java | 3 +-
.../{AlterMTMVTask.java => ChangeMTMVTask.java} | 10 +-
.../org/apache/doris/mtmv/metadata/MTMVJob.java | 73 +++-
.../org/apache/doris/mtmv/metadata/MTMVTask.java | 75 +++-
.../apache/doris/nereids/cost/CostCalculator.java | 2 +-
.../glue/translator/PhysicalPlanTranslator.java | 42 +-
.../processor/post/RuntimeFilterPruner.java | 13 +-
.../expression/rewrite/ExpressionOptimization.java | 4 +-
.../rewrite/rules/SimplifyComparisonPredicate.java | 140 +++++++
.../doris/nereids/stats/ExpressionEstimation.java | 3 +
.../doris/nereids/stats/FilterEstimation.java | 2 +-
.../apache/doris/nereids/stats/JoinEstimation.java | 12 +-
.../doris/nereids/stats/StatsCalculatorV2.java | 1 +
.../trees/expressions/literal/DateLiteral.java | 45 ++-
.../trees/expressions/literal/DateTimeLiteral.java | 68 ++--
.../expressions/literal/DateTimeV2Literal.java | 89 +++++
.../trees/expressions/literal/DateV2Literal.java | 49 +++
.../apache/doris/nereids/types/DateTimeType.java | 4 +-
.../apache/doris/nereids/types/DateTimeV2Type.java | 4 +-
.../org/apache/doris/nereids/types/DateType.java | 4 +-
.../org/apache/doris/nereids/types/DateV2Type.java | 4 +-
.../doris/nereids/types/coercion/DateLikeType.java | 20 +-
.../doris/nereids/util/TypeCoercionUtils.java | 19 +
.../java/org/apache/doris/persist/EditLog.java | 26 +-
.../persist/StaleStatisticsRecordsDetector.java | 69 ----
.../org/apache/doris/planner/BrokerScanNode.java | 9 +-
.../apache/doris/planner/DistributedPlanner.java | 48 ++-
.../doris/planner/ListPartitionPrunerV2.java | 81 +++-
...{CrossJoinNode.java => NestedLoopJoinNode.java} | 40 +-
.../org/apache/doris/planner/OlapScanNode.java | 1 -
.../doris/planner/PartitionPrunerV2Base.java | 49 ++-
.../doris/planner/RangePartitionPrunerV2.java | 12 +-
.../apache/doris/planner/SingleNodePlanner.java | 13 +-
.../apache/doris/planner/StreamLoadPlanner.java | 2 +-
.../apache/doris/planner/StreamLoadScanNode.java | 9 +-
.../planner/external/ExternalFileScanNode.java | 8 +
.../doris/planner/external/HiveScanProvider.java | 44 ++-
.../apache/doris/planner/external/HiveSplit.java | 24 +-
.../planner/external/IcebergDeleteFileFilter.java | 76 ++++
.../planner/external/IcebergScanProvider.java | 91 ++++-
.../external/IcebergSplit.java} | 24 +-
.../doris/planner/external/LoadScanProvider.java | 12 +-
.../doris/planner/external/QueryScanProvider.java | 9 +-
.../doris/planner/external/TableFormatType.java | 25 +-
.../java/org/apache/doris/qe/ConnectContext.java | 4 +
.../main/java/org/apache/doris/qe/DdlExecutor.java | 3 +-
.../org/apache/doris/qe/MasterCatalogExecutor.java | 5 +-
.../java/org/apache/doris/qe/SessionVariable.java | 11 +
.../java/org/apache/doris/qe/ShowExecutor.java | 103 ++++-
.../apache/doris/service/FrontendServiceImpl.java | 32 +-
.../org/apache/doris/statistics/AnalysisJob.java | 30 +-
.../doris/statistics/AnalysisJobExecutor.java | 4 +-
.../AnalysisJobScheduler.java | 21 +-
.../doris/statistics/AnalysisJobWrapper.java | 2 +-
.../org/apache/doris/statistics/ColumnStat.java | 1 +
.../apache/doris/statistics/ColumnStatistic.java | 50 ++-
.../org/apache/doris/statistics/DeriveFactory.java | 4 +-
.../apache/doris/statistics/HMSAnalysisJob.java | 57 +++
.../apache/doris/statistics/HiveAnalysisJob.java | 201 ++++++++++
.../doris/statistics/IcebergAnalysisJob.java | 122 ++++++
...sDerive.java => NestedLoopJoinStatsDerive.java} | 6 +-
.../doris/statistics/StatisticConstants.java | 36 ++
.../statistics/StatisticStorageInitializer.java | 10 +-
.../apache/doris/statistics/StatisticalType.java | 2 +-
.../apache/doris/statistics/StatisticsCache.java | 11 +-
.../doris/statistics/StatisticsCacheLoader.java | 1 +
.../apache/doris/statistics/StatisticsManager.java | 36 --
.../doris/statistics/StatisticsRepository.java | 220 +++++++++++
.../doris/statistics/StatisticsTableCleaner.java | 5 +-
.../util/DBObjects.java} | 33 +-
.../statistics/{ => util}/StatisticsUtil.java | 64 ++-
.../doris/transaction/DatabaseTransactionMgr.java | 2 +-
fe/fe-core/src/main/jflex/sql_scanner.flex | 1 +
.../java/org/apache/doris/alter/AlterTest.java | 10 +
.../org/apache/doris/catalog/CreateTableTest.java | 38 ++
.../doris/cluster/DecommissionBackendTest.java | 92 ++++-
.../java/org/apache/doris/metric/MetricsTest.java | 15 +
.../org/apache/doris/mtmv/MTMVJobManagerTest.java | 57 ++-
.../apache/doris/mtmv/MTMVTaskExecutorTest.java | 4 +-
.../java/org/apache/doris/mtmv/MTMVUtilsTest.java | 7 +
.../org/apache/doris/mtmv/ShowMTMVJobStmtTest.java | 81 ++++
.../apache/doris/mtmv/ShowMTMVTaskStmtTest.java | 81 ++++
.../expression/rewrite/ExpressionRewriteTest.java | 76 ++++
.../doris/nereids/stats/FilterEstimationTest.java | 437 ++++++++++++++++++++-
.../java/org/apache/doris/planner/PlannerTest.java | 61 ++-
.../org/apache/doris/planner/QueryPlanTest.java | 2 +-
.../apache/doris/planner/StatisticDeriveTest.java | 2 +-
.../doris/planner/TableFunctionPlanTest.java | 60 ++-
.../ExtractCommonFactorsRuleFunctionTest.java | 18 +-
.../doris/statistics/AnalysisJobExecutorTest.java | 1 -
.../apache/doris/statistics/AnalysisJobTest.java | 2 +-
.../org/apache/doris/statistics/CacheTest.java | 1 +
.../apache/doris/utframe/TestWithFeService.java | 7 +
gensrc/script/doris_builtins_functions.py | 19 +
gensrc/thrift/PlanNodes.thrift | 23 +-
.../test_null_aware_left_anti_join.out | 2 +
.../unique/test_unique_table_new_sequence.out | 45 +++
.../data/delete_p0/test_delete_where_in.out | 21 +
regression-test/data/es_p0/test_es_query.out | 25 +-
.../external_catalog_p0/hive/test_hive_orc.out | 85 ++++
.../external_catalog_p0/hive/test_hive_parquet.out | 23 ++
.../load_p0/broker_load/simple_array_list_type.orc | Bin 0 -> 1692 bytes
.../data/load_p0/broker_load/test_array_load.out | 80 ++++
.../stream_load/test_txt_special_delimiter.csv | 1 +
.../stream_load/test_txt_special_delimiter.out | 7 +
.../data/query_p0/aggregate/aggregate.out | 9 +
.../join/test_nestedloop_semi_anti_join.out | 41 ++
.../array_functions/test_array_functions.out | 133 +++++++
.../test_array_functions_by_literal.out | 18 +
.../data/statistics/alter_col_stats.out | 11 +
regression-test/pipeline/p0/conf/be.conf | 69 ++++
regression-test/pipeline/p0/conf/external.json | 26 ++
regression-test/pipeline/p0/conf/fe.conf | 74 ++++
.../pipeline/p0/conf}/odbcinst.ini | 0
.../pipeline/p0/conf/regression-conf.groovy | 81 ++++
regression-test/pipeline/p1/conf/be.conf | 67 ++++
regression-test/pipeline/p1/conf/fe.conf | 75 ++++
.../pipeline/p1/conf}/odbcinst.ini | 0
.../pipeline/p1/conf/regression-conf.groovy | 46 +--
regression-test/suites/account_p0/test_role.groovy | 7 +-
.../test_null_aware_left_anti_join.groovy | 3 +
.../unique/test_unique_table_new_sequence.groovy | 124 ++++++
.../suites/delete_p0/test_delete_where_in.groovy | 99 +++++
regression-test/suites/es_p0/test_es_query.groovy | 19 +-
...st_hive_parquet.groovy => test_hive_orc.groovy} | 156 +++-----
.../hive/test_hive_parquet.groovy | 24 ++
.../load_p0/broker_load/test_array_load.groovy | 37 +-
.../stream_load/test_txt_special_delimiter.groovy | 58 +++
regression-test/suites/query/join/test_join.groovy | 346 +++++-----------
.../suites/query_p0/aggregate/aggregate.groovy | 3 +
.../join/test_nestedloop_semi_anti_join.groovy | 81 ++++
.../array_functions/test_array_functions.groovy | 11 +
.../test_array_functions_by_literal.groovy | 9 +
.../suites/statistics/alter_col_stats.groovy | 53 +++
.../tpch_sf1_p1/tpch_sf1/explain/test_q11.groovy | 4 +-
.../tpch_sf1_p1/tpch_sf1/explain/test_q22.groovy | 4 +-
run-fe-ut.sh | 4 +-
320 files changed, 8760 insertions(+), 2073 deletions(-)
create mode 100644
be/src/vec/aggregate_functions/aggregate_function_avg_weighted.cpp
create mode 100644
be/src/vec/aggregate_functions/aggregate_function_avg_weighted.h
create mode 100644 be/src/vec/exec/format/table/iceberg_reader.cpp
create mode 100644 be/src/vec/exec/format/table/iceberg_reader.h
copy be/src/{util/stack_util.cpp =>
vec/exec/format/table/table_format_reader.cpp} (70%)
create mode 100644 be/src/vec/exec/format/table/table_format_reader.h
copy be/src/{util/stack_util.cpp =>
vec/functions/array/function_array_compact.cpp} (70%)
create mode 100644 be/src/vec/functions/array/function_array_compact.h
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=desktops/p2_col=bigint_col/20221027_110128_00032_9ch4x_5b6acedf-02e6-4b4e-9de2-ae050ddb7c35
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=desktops/p2_col=boolean_col/20221027_110128_00032_9ch4x_f45472a3-a935-4ff7-9340-b977d51ef3a2
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=desktops/p2_col=float_col/20221027_110128_00032_9ch4x_e5a56983-f7e1-4b66-8d7d-134cd8a3b5d4
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=desktops/p2_col=int_col/20221027_110128_00032_9ch4x_d3313b2c-f4da-468a-b3b0-7627b7dd6398
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=desktops/p2_col=smallint_col/20221027_110128_00032_9ch4x_91eb689b-3053-4630-9f72-7e068afd3302
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=desktops/p2_col=tinyint_col/20221027_110128_00032_9ch4x_fbdd6369-e923-47ff-af40-d023687b7d88
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=phones/p2_col=bigint_col/20221027_110128_00032_9ch4x_02ec7d8c-ecb8-49f4-ad32-6a0fa0915743
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=phones/p2_col=boolean_col/20221027_110128_00032_9ch4x_311f132e-20d2-4e1a-82c1-c0edbc3e7903
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=phones/p2_col=float_col/20221027_110128_00032_9ch4x_feb65e62-844e-4890-b326-3aa48ceb917f
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=phones/p2_col=int_col/20221027_110128_00032_9ch4x_2f09f727-20f2-4e4c-adec-6a624df8c9f5
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=phones/p2_col=smallint_col/20221027_110128_00032_9ch4x_de0f05f9-2fd4-4bd6-886a-434ac3349ca1
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=phones/p2_col=tinyint_col/20221027_110128_00032_9ch4x_099556fa-095b-4a70-8718-912c59154582
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=tablets/p2_col=bigint_col/20221027_110128_00032_9ch4x_3340eb7c-fb57-4cc4-a3ed-fc55eb2674f9
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=tablets/p2_col=boolean_col/20221027_110128_00032_9ch4x_ef0c50af-d5c4-4375-ad11-106c64ca7903
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=tablets/p2_col=float_col/20221027_110128_00032_9ch4x_45fd0f90-ca2f-4f89-832b-5b99395e7081
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=tablets/p2_col=int_col/20221027_110128_00032_9ch4x_d7af1778-dd22-4e08-804d-be1e462fca25
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=tablets/p2_col=smallint_col/20221027_110128_00032_9ch4x_78760342-f3b8-415c-aaf5-be7a7ed38b4c
create mode 100644
docker/thirdparties/docker-compose/hive/scripts/orc/orc_all_types/p1_col=tablets/p2_col=tinyint_col/20221027_110128_00032_9ch4x_bbb15063-ee62-4fd3-9602-3e9ce04f72ef
copy
docs/en/docs/sql-manual/{sql-reference/Utility-Statements/REFRESH-CATALOG.md =>
sql-functions/aggregate-functions/avg_weighted.md} (59%)
create mode 100644
docs/en/docs/sql-manual/sql-functions/aggregate-functions/percentile_array.md
copy docs/en/docs/sql-manual/sql-functions/aggregate-functions/{topn_array.md
=> topn_weighted.md} (60%)
create mode 100644
docs/en/docs/sql-manual/sql-functions/array-functions/array_compact.md
create mode 100644
docs/en/docs/sql-manual/sql-functions/bitmap-functions/bitmap_from_array.md
rename
docs/en/docs/sql-manual/sql-reference/Utility-Statements/{REFRESH-CATALOG.md =>
REFRESH.md} (58%)
copy
docs/zh-CN/docs/sql-manual/{sql-reference/Utility-Statements/REFRESH-CATALOG.md
=> sql-functions/aggregate-functions/avg_weighted.md} (59%)
copy
docs/zh-CN/docs/sql-manual/sql-functions/aggregate-functions/{percentile.md =>
percentile_array.md} (52%)
copy
docs/zh-CN/docs/sql-manual/sql-functions/aggregate-functions/{topn_array.md =>
topn_weighted.md} (60%)
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/array-functions/array_compact.md
create mode 100644
docs/zh-CN/docs/sql-manual/sql-functions/bitmap-functions/bitmap_from_array.md
rename
docs/zh-CN/docs/sql-manual/sql-reference/Utility-Statements/{REFRESH-CATALOG.md
=> REFRESH.md} (60%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowMTMVJobStmt.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowMTMVTaskStmt.java
rename
fe/fe-core/src/main/java/org/apache/doris/mtmv/metadata/{AlterMTMVTask.java =>
ChangeMTMVTask.java} (90%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/SimplifyComparisonPredicate.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/DateTimeV2Literal.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/DateV2Literal.java
copy be/src/util/stack_util.cpp =>
fe/fe-core/src/main/java/org/apache/doris/nereids/types/coercion/DateLikeType.java
(70%)
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/persist/StaleStatisticsRecordsDetector.java
rename fe/fe-core/src/main/java/org/apache/doris/planner/{CrossJoinNode.java
=> NestedLoopJoinNode.java} (87%)
copy be/src/util/stack_util.cpp =>
fe/fe-core/src/main/java/org/apache/doris/planner/external/HiveSplit.java (69%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/external/IcebergDeleteFileFilter.java
copy
fe/fe-core/src/main/java/org/apache/doris/{statistics/StatisticConstants.java
=> planner/external/IcebergSplit.java} (65%)
copy be/src/util/stack_util.cpp =>
fe/fe-core/src/main/java/org/apache/doris/planner/external/TableFormatType.java
(70%)
rename fe/fe-core/src/main/java/org/apache/doris/{persist =>
statistics}/AnalysisJobScheduler.java (85%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/HMSAnalysisJob.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/HiveAnalysisJob.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/IcebergAnalysisJob.java
rename
fe/fe-core/src/main/java/org/apache/doris/statistics/{CrossJoinStatsDerive.java
=> NestedLoopJoinStatsDerive.java} (89%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/statistics/StatisticsRepository.java
copy fe/fe-core/src/main/java/org/apache/doris/{nereids/types/DateType.java =>
statistics/util/DBObjects.java} (61%)
rename fe/fe-core/src/main/java/org/apache/doris/statistics/{ =>
util}/StatisticsUtil.java (76%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mtmv/ShowMTMVJobStmtTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/mtmv/ShowMTMVTaskStmtTest.java
create mode 100644
regression-test/data/data_model_p0/unique/test_unique_table_new_sequence.out
create mode 100644 regression-test/data/delete_p0/test_delete_where_in.out
create mode 100644
regression-test/data/external_catalog_p0/hive/test_hive_orc.out
create mode 100644
regression-test/data/load_p0/broker_load/simple_array_list_type.orc
create mode 100644
regression-test/data/load_p0/stream_load/test_txt_special_delimiter.csv
create mode 100644
regression-test/data/load_p0/stream_load/test_txt_special_delimiter.out
create mode 100644
regression-test/data/query_p0/join/test_nestedloop_semi_anti_join.out
create mode 100644 regression-test/data/statistics/alter_col_stats.out
create mode 100644 regression-test/pipeline/p0/conf/be.conf
create mode 100644 regression-test/pipeline/p0/conf/external.json
create mode 100644 regression-test/pipeline/p0/conf/fe.conf
copy {conf => regression-test/pipeline/p0/conf}/odbcinst.ini (100%)
create mode 100644 regression-test/pipeline/p0/conf/regression-conf.groovy
create mode 100644 regression-test/pipeline/p1/conf/be.conf
create mode 100644 regression-test/pipeline/p1/conf/fe.conf
copy {conf => regression-test/pipeline/p1/conf}/odbcinst.ini (100%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/types/DateTimeType.java
=> regression-test/pipeline/p1/conf/regression-conf.groovy (52%)
create mode 100644
regression-test/suites/data_model_p0/unique/test_unique_table_new_sequence.groovy
create mode 100644 regression-test/suites/delete_p0/test_delete_where_in.groovy
copy regression-test/suites/external_catalog_p0/hive/{test_hive_parquet.groovy
=> test_hive_orc.groovy} (54%)
create mode 100644
regression-test/suites/load_p0/stream_load/test_txt_special_delimiter.groovy
create mode 100644
regression-test/suites/query_p0/join/test_nestedloop_semi_anti_join.groovy
create mode 100644 regression-test/suites/statistics/alter_col_stats.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
