This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a change to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from 1d076635ba4 [fix](map)fix log message in tablet_schema (#38254)
new 058c55ec8cb [fix](array_range) fix array_range func for large param
which should return error (#38284)
new 2aeb3b5af4a [enhance](RateLimit) Add bvar to monitor object storage
rate limit sleep time and failure time (#38294)
new 3b19178d6cc [Fix](recycler) recycler need index_suffix and inverted
index storage format to delete idx file (#38306)
new c68fab2dd6e [fix](load) fix no error url for stream load (#38325)
new 2938366dce2 [Opt](load) don't print stack when some errors occur for
stream load (#38332)
new c9d873d3779 [enhancement](cloud) unify scanner thread pool as LOCAL in
cloud mode (#38372)
new 9fbc89bbad4 [enhencement](trino-connector) trino-connector supports
push down projection to connectors (#37874)
new 872ff51b31e [chore](fe) Log the cloud/local mode (#38531)
new 1c850866bb7 [fix](test)Fix unstable index compaction fault injection
case (#38501)
new 1e13f0d6fb9 [fix](simdjson) fix simdjson reader will parse given root
twice when data is object array (#38490)
new 2a37230a654 [fix](cloud) Ensure that the eror log has been uploaded to
S3 before returning an error URL to the user (#38359)
new 309a8e29d17 [fix](cloud) Do not execute test_publish_timeout
regression case in cloud mode (#38383)
new 26640f114fa [fix](bash) Make sure that the `prof_prefix` defined in
`JEMALLOC_CONF` is not overwritten in start_be.sh (#38354)
new 879a6577bfe [Feature](multi-catalog) Add memory tracker for orc
reader/writer and arrow parquet writer。 (#37234)
The 14 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/http/action/stream_load.cpp | 29 +--
be/src/io/fs/azure_obj_storage_client.cpp | 90 ++++++++--
be/src/io/fs/s3_file_reader.cpp | 12 --
be/src/io/fs/s3_obj_storage_client.cpp | 70 ++++++--
be/src/runtime/exec_env.h | 13 ++
be/src/runtime/exec_env_init.cpp | 12 +-
be/src/runtime/runtime_state.cpp | 48 +++--
be/src/runtime/runtime_state.h | 4 +-
.../runtime/stream_load/stream_load_executor.cpp | 4 +-
be/src/util/faststring.h | 2 +-
be/src/util/s3_util.cpp | 40 +++--
be/src/util/s3_util.h | 22 +--
be/src/util/slice.h | 2 +-
be/src/vec/common/allocator.cpp | 90 ++++++----
be/src/vec/common/allocator.h | 165 +++++++++++++++--
be/src/vec/common/allocator_fwd.h | 4 +-
be/src/vec/common/hash_table/phmap_fwd_decl.h | 2 +-
be/src/vec/exec/format/json/new_json_reader.cpp | 3 +-
be/src/vec/exec/format/json/new_json_reader.h | 1 +
.../format/orc/orc_memory_pool.h} | 41 +++--
be/src/vec/exec/format/orc/vorc_reader.cpp | 2 +
.../vec/exec/format/parquet/arrow_memory_pool.cpp | 74 ++++++++
be/src/vec/exec/format/parquet/arrow_memory_pool.h | 199 +++++++++++++++++++++
be/src/vec/exec/scan/new_olap_scanner.cpp | 4 +
.../vec/functions/array/function_array_range.cpp | 31 +++-
be/src/vec/runtime/vorc_transformer.cpp | 22 +--
be/src/vec/runtime/vparquet_transformer.cpp | 15 +-
bin/start_be.sh | 4 +-
cloud/src/recycler/azure_obj_client.cpp | 24 ++-
cloud/src/recycler/recycler.cpp | 74 +++++---
cloud/src/recycler/recycler.h | 2 +
cloud/src/recycler/s3_accessor.cpp | 50 ++++--
cloud/src/recycler/s3_accessor.h | 22 +++
cloud/src/recycler/s3_obj_client.cpp | 26 ++-
cloud/src/recycler/util.h | 15 +-
cloud/test/recycler_test.cpp | 52 ++++--
common/cpp/s3_rate_limiter.cpp | 28 +--
common/cpp/s3_rate_limiter.h | 6 +-
.../src/main/java/org/apache/doris/DorisFE.java | 7 +
.../source/TrinoConnectorScanNode.java | 43 +++--
.../load_p0/stream_load/load_object_array_json.out | 4 +
.../load_p0/stream_load/test_json_object_array.csv | 1 +
.../test_index_compaction_fault_injection.groovy | 7 +-
.../test_index_compaction_dup_keys.groovy | 2 +-
.../test_index_compaction_null.groovy | 6 +-
.../test_index_compaction_unique_keys.groovy | 3 +-
...dex_compaction_with_multi_index_segments.groovy | 6 +-
...onpath.groovy => load_object_array_json.groovy} | 52 +++---
.../stream_load/test_publish_timeout.groovy | 3 +
.../load_p0/stream_load/test_stream_load.groovy | 40 +++++
.../scalar_function/Array.groovy | 29 +++
.../array_functions/test_array_functions.groovy | 22 +++
52 files changed, 1184 insertions(+), 345 deletions(-)
copy be/src/vec/{columns/column_nothing.h =>
exec/format/orc/orc_memory_pool.h} (53%)
create mode 100644 be/src/vec/exec/format/parquet/arrow_memory_pool.cpp
create mode 100644 be/src/vec/exec/format/parquet/arrow_memory_pool.h
create mode 100644
regression-test/data/load_p0/stream_load/load_object_array_json.out
create mode 100644
regression-test/data/load_p0/stream_load/test_json_object_array.csv
copy
regression-test/suites/load_p0/stream_load/{load_json_with_jsonpath.groovy =>
load_object_array_json.groovy} (59%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
