This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch auto-pick-59056-branch-4.0
in repository https://gitbox.apache.org/repos/asf/doris.git
from 7c28cdd1204 branch-4.0: [Feature](Streaming Job) Extend streaming job
to support MySQL synchronization #58898 (#59228)
add c8b8cb95ea5 [Feature](iceberg) Add manifest-level cache for Iceberg
tables to reduce I/O and parsing overhead (#59056)
No new revisions were added by this update.
Summary of changes:
fe/check/checkstyle/suppressions.xml | 3 +
.../datasource/iceberg/IcebergExternalCatalog.java | 36 +
.../datasource/iceberg/IcebergMetadataCache.java | 16 +
.../doris/datasource/iceberg/IcebergUtils.java | 16 +
.../iceberg/cache/ContentFileEstimator.java | 194 +++++
.../iceberg/cache/IcebergManifestCache.java | 91 +++
.../iceberg/cache/IcebergManifestCacheLoader.java | 118 +++
.../iceberg/cache/ManifestCacheKey.java} | 32 +-
.../iceberg/cache/ManifestCacheValue.java | 65 ++
.../datasource/iceberg/source/IcebergScanNode.java | 196 ++++-
.../metastore/AbstractIcebergProperties.java | 62 ++
.../java/org/apache/iceberg/DeleteFileIndex.java | 906 +++++++++++++++++++++
.../iceberg/test_iceberg_manifest_cache.out | 21 +
.../iceberg/test_iceberg_manifest_cache.groovy | 119 +++
14 files changed, 1853 insertions(+), 22 deletions(-)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/cache/ContentFileEstimator.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/cache/IcebergManifestCache.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/cache/IcebergManifestCacheLoader.java
copy
fe/fe-core/src/main/java/org/apache/doris/{nereids/metrics/EventEnhancer.java
=> datasource/iceberg/cache/ManifestCacheKey.java} (61%)
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/cache/ManifestCacheValue.java
create mode 100644
fe/fe-core/src/main/java/org/apache/iceberg/DeleteFileIndex.java
create mode 100644
regression-test/data/external_table_p0/iceberg/test_iceberg_manifest_cache.out
create mode 100644
regression-test/suites/external_table_p0/iceberg/test_iceberg_manifest_cache.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
