twuebi opened a new pull request, #1296: URL: https://github.com/apache/iceberg-go/pull/1296
The candidate planner only counted positional and equality delete files toward delete pressure, so a right-sized data file accumulating file-scoped deletes (a deletion vector, or a path-scoped positional delete) was never rewritten — its dead rows persisted as read amplification indefinitely. Add a DeleteRatioThreshold (default 0.3, mirroring Java's delete-ratio-threshold). A file is forced into compaction when its file-scoped deletes shadow at least that fraction of its rows. Following Java's BinPackRewriteFilePlanner.tooHighDeleteRatio, the deleted-row count sums the record counts of file-scoped deletes only — deletion vectors and path-scoped positional deletes (those carrying a referenced data file), identified by isFileScoped (= referenced data file present, matching ContentFileUtil.isFileScoped) — capped at the data file's record count. Equality deletes and partition-scoped positional deletes are not file-scoped and stay under the existing count rule, to which deletion vectors are now also added. Unlike Java, a threshold of 0 disables the rule so a zero-valued Config stays valid; the default remains 0.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
