szehon-ho commented on code in PR #11555:
URL: https://github.com/apache/iceberg/pull/11555#discussion_r1902289753
##########
core/src/main/java/org/apache/iceberg/util/ContentFileUtil.java:
##########
@@ -60,28 +69,63 @@ public static CharSequence referencedDataFile(DeleteFile
deleteFile) {
return deleteFile.referencedDataFile();
}
- int pathId = MetadataColumns.DELETE_FILE_PATH.fieldId();
- Type pathType = MetadataColumns.DELETE_FILE_PATH.type();
-
Map<Integer, ByteBuffer> lowerBounds = deleteFile.lowerBounds();
- ByteBuffer lowerPathBound = lowerBounds != null ? lowerBounds.get(pathId)
: null;
+ ByteBuffer lowerPathBound = lowerBounds != null ? lowerBounds.get(PATH_ID)
: null;
if (lowerPathBound == null) {
return null;
}
Map<Integer, ByteBuffer> upperBounds = deleteFile.upperBounds();
- ByteBuffer upperPathBound = upperBounds != null ? upperBounds.get(pathId)
: null;
+ ByteBuffer upperPathBound = upperBounds != null ? upperBounds.get(PATH_ID)
: null;
if (upperPathBound == null) {
return null;
}
if (lowerPathBound.equals(upperPathBound)) {
- return Conversions.fromByteBuffer(pathType, lowerPathBound);
+ return Conversions.fromByteBuffer(PATH_TYPE, lowerPathBound);
} else {
return null;
}
}
+ /**
+ * Replace file_path reference for a delete file entry
+ *
+ * @param deleteFile delete file whose entry will be replaced
+ * @param sourcePrefix source prefix which will be replaced
+ * @param targetPrefix target prefix which will replace it
+ * @return metrics for the new delete file entry
+ */
+ public static Metrics replacePathBounds(
+ DeleteFile deleteFile, String sourcePrefix, String targetPrefix) {
+ Preconditions.checkArgument(
+ deleteFile.content() == FileContent.POSITION_DELETES,
+ "Only position delete files supported");
+
+ Map<Integer, ByteBuffer> lowerBounds = deleteFile.lowerBounds();
+ ByteBuffer lowerPathBound = lowerBounds != null ? lowerBounds.get(PATH_ID)
: null;
+ if (lowerPathBound == null) {
+ return metricsWithoutPathBounds(deleteFile);
+ }
+
+ Map<Integer, ByteBuffer> upperBounds = deleteFile.upperBounds();
+ ByteBuffer upperPathBound = upperBounds != null ? upperBounds.get(PATH_ID)
: null;
+ if (upperPathBound == null) {
+ return metricsWithoutPathBounds(deleteFile);
+ }
+
+ if (lowerPathBound.equals(upperPathBound)) {
Review Comment:
Yea if I understand it correctly, the existing logic doesnt do filtering if
the two bounds arent equal:, see just above this method:
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/util/ContentFileUtil.java#L81
I believe the reasoning is, its not even worth it (as paths are so random).
So for simplicity I just remove these metrics if the two bounds are not the
same, what do you think? Or let me know if you prefer I just change both paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
