mattfaltyn opened a new pull request, #2288: URL: https://github.com/apache/iceberg-go/pull/2288
Fixes #2287. ## Problem Parquet scan planning can return multiple byte-range tasks for one source file. Compaction counted and packed those ranges as independent files, then read only each range while scheduling the entire source file for replacement. Partial-progress commits could therefore publish one range's rows and fail on a later group, leaving unread rows absent from the active snapshot. ## Change - Collapse split tasks by source path before compaction counts, candidate selection and bin packing. Preserve per-file delete and lineage metadata without mutating caller tasks. - Reject incomplete ranges, duplicate files and residual row filters in direct compaction worker groups before I/O. Keep the existing zero-range shorthand for complete files. - Document the complete-file input contract and add planner, executor and end-to-end atomic/partial-progress regressions. Tasks passed to the planner come from one coherent scan plan. The fix preserves the exported API; compaction reads complete source files, with concurrency across files. ## Validation The new row-preservation regression fails on the base: the split partial-progress case returns an error after committing only 1 of 12 rows; the unsplit controls pass. The regression must preserve all twelve rows with this change. Local checks and current-head CI results will be recorded here before this draft is marked ready for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
