andygrove opened a new pull request, #205: URL: https://github.com/apache/datafusion-site/pull/205
Draft blog post for the upcoming Comet 1.1.0 release. Built from the 333 commits on `apache/main` since the `1.0.0` tag (2026-08-04), 37 distinct authors, plus the `iceberg-writes.md`, `tuning.md`, and `spark-versions.md` user guides on main. **Native Iceberg writes (experimental)** is the headline and gets roughly 40% of the post: the AQE problem, the split-operator plan (`IcebergWrite` / `IcebergCommit`), the iceberg-rust native write, fidelity-by-design (manifest metrics re-derived on the JVM with iceberg-java's own code; eligibility as an allowlist), failure/abort semantics, and the accepted divergences. Then Iceberg reads (V3 deletion vectors, native system functions), native Parquet writes on Spark 4.0+, Celeborn native RSS, performance, memory observability, expanded coverage, and deprecations. ## Open items before this can merge - [ ] **Date is a placeholder.** Currently `2026-09-25` in the filename and frontmatter — update once the release date is set. - [ ] **Spark 3.4 removal.** The 1.0.0 post promised removal in 1.1.0, but main still builds and publishes 3.4 binaries and `spark-versions.md` now says "removal in a future release." Written here as still-deprecated-not-removed to match the code. JDK 11 *was* removed (#5897), so that half landed. - [ ] **No performance numbers.** The 1.0.0 post cited the AWS Labs EKS benchmark and 0.17.0 cited "TPC-DS 1TB ~9% faster"; there is nothing comparable here yet. The Iceberg write benchmark (#6038) landed the harness, not results. TPC-DS numbers vs 1.0.0, or native-vs-iceberg-java write timings, would strengthen both the Performance section and the Iceberg pitch. - [ ] **Changelog link** points at `docs/source/changelog/1.1.0.md`, which does not exist yet. - [ ] **Celeborn framing.** Shipped as an unlabeled 8-part series with no dedicated user-guide page, so it is described plainly with its requirements rather than labeled experimental. Confirm whether it should carry an experimental label. - [ ] Commit/contributor counts are from `git log 1.0.0..apache/main` as of 2026-09-22 and will need refreshing at release time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
