alexanderbianchi opened a new pull request, #3587: URL: https://github.com/apache/iceberg-rust/pull/3587
## Which issue does this PR close? Related to #3276 and #3277. No issue is closed by this draft. ## What changes are included in this PR? Delete-file loading currently consumes Parquet streams on the I/O runtime, so footer parsing, decompression, page decoding, and delete-filter construction compete with storage requests. Run the caching delete loader on the configured CPU runtime instead. Add a private runtime-dispatched file reader that opens storage readers and executes byte-range reads on the I/O runtime. Awaiting these reads resumes Parquet processing on its caller's runtime. Dropping a read cancels its spawned storage request. The shared Parquet opening path uses this adapter; data-file decoding placement is addressed in a follow-up draft. Puffin deletion-vector blob loading stays on I/O workers; vector deserialization and validation run in the CPU parse phase. Existing delete caching and concurrency limits are preserved. This adds task dispatch per storage read. No throughput benchmark has been run. ## Are these changes tested? - `cargo test -p iceberg --lib arrow::`: 285 passed. - New tests exercise separate CPU/I/O runtimes, storage error propagation, cancellation of an outstanding storage read, and delete loading that requires the CPU runtime to make progress. - `cargo clippy -p iceberg --lib --tests -- -D warnings`: passed. - `cargo fmt --all -- --check`: passed. ## AI Disclosure Implementation, tests, and this description were prepared with OpenAI Codex assistance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
