waterWang opened a new pull request, #17835: URL: https://github.com/apache/iceberg/pull/17835
ContentFileParser serialized neither the data sequence number nor the file sequence number of a content file, and the REST ContentFile schema did not define fields for them. Any file scan task that crosses the REST scan planning API therefore arrived with null sequence numbers on its data file and delete files. Since delete files apply only to data files with an equal or smaller data sequence number, engines consuming those tasks could not determine delete applicability at read time, breaking merge-on-read for tables with delete files. Changes: - `ContentFileParser.toJson`: serialize `data-sequence-number` and `file-sequence-number` when present - `ContentFileParser.fromJson`: deserialize both fields and set them on the constructed file objects - `rest-catalog-open-api.yaml`: add `data-sequence-number` and `file-sequence-number` to the ContentFile schema - Tests: round-trip tests for data files, delete files, and backward-compatible null defaults Fixes #17833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
