Baunsgaard commented on PR #16937: URL: https://github.com/apache/iceberg/pull/16937#issuecomment-4790082849
## Native fixture writes don't move CI time I tried extending the native-write idea to other action suites to cut Spark CI wall-time. The gains are real but too small to matter. **fixture writes aren't the bottleneck** (the rewrite action + Spark read-back dominate). **Local A/B (JUnit `testsuite` wall time):** - `TestRewritePositionDeleteFilesAction`: 42.1s → 42.1s (**0%**, reverted) — few large writes, native costs the same. - `TestRewriteTablePathsAction`: 203s → 193s (~5%, noisy) — many tiny 1-row writes, the best case, still only ~10s. **CI (this push vs. previous, `spark-tests core` jobs):** deltas swing from −416s to +511s per job with no consistent direction; total 45m19s → 49m9s. A ~10s saving (≈0.4% of a ~2200–2700s job) is fully swamped by run-to-run variance. **Conclusion:** native writes are a fine local-test cleanup but not a clear win in CI. I am therefore closing the PR, since it is not bringing the benefits wanted! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
