vivek-9941 commented on PR #18361: URL: https://github.com/apache/iceberg/pull/18361#issuecomment-6394220700
Thank you for the detailed feedback and testing! I've addressed the remaining issue by extending normalization to delete files. ## Changes Made Extended `ManifestWriter.normalizeEntry()` to handle all content file types: - **Delete files now normalized**: Added handling for `DeleteFile` instances using `FileMetadata.deleteFileBuilder(spec)` - **Equality delete special handling**: For equality deletes, explicitly call `ofEqualityDeletes()` with field IDs before copying, since `FileMetadata.Builder.copy(DeleteFile)` doesn't copy `equalityFieldIds`. This follows the same pattern as `RewriteTablePathUtil.newEqualityDeleteEntry`. - **Cleaner logic flow**: Moved the spec ID check before type checks for better code organization ## Implementation The normalized entry now handles: 1. **Data files** - Using `DataFiles.builder(spec).copy()` (existing logic) 2. **Position deletes** - Using `FileMetadata.deleteFileBuilder(spec).copy()` 3. **Equality deletes** - Using `FileMetadata.deleteFileBuilder(spec).ofEqualityDeletes(...).copy()` 4. **Deletion vectors (v3-v4)** - Using `FileMetadata.deleteFileBuilder(spec).copy()` All entry status and sequence metadata is preserved across all file types. ## Testing As you tested, this should now handle all nine combinations (3 file types × 3 format versions) where delete-file metadata carries pre-promotion partition values. The change has been verified with: - ✅ Compilation: `./gradlew :iceberg-core:compileJava` - ✅ Code formatting: `./gradlew spotlessCheck` Let me know if you'd like me to add specific test cases for the delete file scenarios! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
