alexanderbianchi opened a new pull request, #3514: URL: https://github.com/apache/iceberg-rust/pull/3514
## Which issue does this PR close? Follows up on the [reader-options discussion in #2671](https://github.com/apache/iceberg-rust/pull/2671#pullrequestreview-5596949417) and supports [apache/datafusion-iceberg#67](https://github.com/apache/datafusion-iceberg/pull/67). Does not close #2671; the DataFusion integration continues in its own repository. ## What changes are included in this PR? An eager scan can retain file tasks and discard `TableScan`, but currently cannot retain that scan's reader settings independently of its planning state. `TableScan::reader_options()` captures a small serializable `TableScanReaderOptions` value. Its `apply_to(ArrowReaderBuilder)` method is also used by `to_arrow()`, so normal scans and separately constructed readers share the settings mapping. ```rust let options = scan.reader_options(); let tasks: Vec<_> = scan.plan_files().await?.try_collect().await?; drop(scan); let reader = options .apply_to(ArrowReaderBuilder::new(local_file_io, local_runtime)) .build(); ``` The value captures batch size, data-file concurrency, row-group filtering, row selection, and bloom filtering. Projection, predicates, and deletes remain in file tasks. Local Parquet I/O tuning survives applying the options; callers can then override execution settings such as per-partition concurrency. Build one reader and clone it across partitions to share delete loading. This enables #67 to preserve non-default reader settings in a follow-up. It adds no eager planner or DataFusion dependency and does not change the task wire format. ## Are these changes tested? Regression coverage serializes the settings, drops the scan, and reads filtered tasks with position deletes using fresh local I/O/runtime. It compares results with `to_arrow()` and checks batch sizing. A separate test covers resetting an unset batch size and preserving local Parquet I/O tuning. Passed: 2,031 iceberg library tests (one existing ignored test), 88 documentation tests, crate Clippy with all targets/features and `-D warnings`, formatting, and `git diff --check`. Regenerated and inspected `iceberg/public-api.txt`. ## AI Disclosure Implemented with Codex assistance. Review focus: the boundary between scan settings and local I/O tuning, serialized-option API evolution, and preserving the reader default when the captured batch size is unset. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
