vbhanuchander-lang commented on PR #17645: URL: https://github.com/apache/iceberg/pull/17645#issuecomment-5518538221
@anoopj your three comments were all addressed in `3873922` on 14 Aug — the description was rewritten to list all four sites, the test was replaced with one that actually fails when `VectorizedArrowReader` is reverted (the previous one pinned Arrow's own sizing contract rather than this reader's use of it, exactly as you said), and the method names dropped the `test` prefix. Would you take another look? Re-verified against `main` (`8ea7d00`) today: still `MERGEABLE`, checks green, and the test still fails with the production change reverted. For anyone picking this up cold, the decision is narrow: every `setInitialCapacity(int)` argument in Arrow is a **value count**, and this reader was passing `batchSize * AVERAGE_VARIABLE_WIDTH_RECORD_SIZE` — a byte count — so it over-reserves by that factor. `BaseVariableWidthVector` has a `setInitialCapacity(int, double)` overload for the density case, which is what the fix uses. Closes #17305. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
