wombatu-kun opened a new pull request, #16498: URL: https://github.com/apache/iceberg/pull/16498
## Summary Closes #16493. DeleteOrphanFilesSparkAction.filteredCompareToFileList() previously scoped a user-supplied compareToFileList to the action's location field using a raw files.col(FILE_PATH).startsWith(location) filter. When location lacks a trailing path separator (the production-typical shape for storage URIs like s3://bucket/table returned by Table.location()), that filter also accepts sibling paths such as s3://bucket/table-backup/.... Files in those sibling directories then entered the orphan candidate set and could be deleted. This PR normalizes the prefix to directory form via `LocationUtil.stripTrailingSlash(location) + "/"` before the startsWith filter, so an entry equal to the location path itself is now out of scope as well; the `DeleteOrphanFiles.location(String)` javadoc now states that the location is interpreted as a directory. The same `+ "/"` shape is already used in SnapshotTableSparkAction.doExecute() to prevent identical sibling-prefix collisions, so this aligns the orphan-files action with that existing precedent. The fix targets spark/v4.2, the latest Spark version; v3.5, v4.0 and v4.1 will follow in a backport PR once this is merged (DeleteOrphanFilesSparkAction.java is byte-identical across all four trees, so the backport is mechanical). The directory-listing path (listedFileDS()) is unaffected: both the Hadoop listing and prefix listing are already bounded to the location directory, the latter because FileSystemWalker.listDirRecursivelyWithFileIO appends "/" to the location before calling listPrefix. --- **AI Disclosure** - Model: Claude Opus 4.7, Claude Opus 5 - Platform/Tool: Claude Code - Human Oversight: fully reviewed - Prompt Summary: Fix DeleteOrphanFilesSparkAction matching sibling prefixes when the location lacks a trailing slash. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
