aokolnychyi commented on code in PR #11481:
URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831735172
##########
data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java:
##########
@@ -146,6 +151,26 @@ private <T> Iterable<T> materialize(CloseableIterable<T>
iterable) {
@Override
public PositionDeleteIndex loadPositionDeletes(
Iterable<DeleteFile> deleteFiles, CharSequence filePath) {
+ if (containsDVs(deleteFiles)) {
+ DeleteFile dv = Iterables.getOnlyElement(deleteFiles);
+ validateDV(dv, filePath);
+ return readDV(dv); // TODO: support caching entire DV files
Review Comment:
A bit of context about how caching works for V2 deletes. If we estimate the
content of entire file to fit into the cache (its in-memory representation), we
read the entire file and cache bitmaps for each data file we read. We can do
similar stuff for Puffin. I need to explore the performance impact of not
knowing the footer size upfront.
Any early feedback is welcome!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
