comphead opened a new pull request, #6649: URL: https://github.com/apache/datafusion-comet/pull/6649
## Which issue does this PR close? Closes #6124. ## Rationale for this change Native Iceberg scans fail on OpenDAL's hard-wired 10s per-I/O timeout (`io operation timeout reached`), and nothing in Comet could change it. apache/iceberg-rust#3263 made the timeout configurable through the `opendal.io-timeout-ms` FileIO property. ## What changes are included in this PR? - Bump `iceberg` and `iceberg-storage-opendal` from `bb1e4a4` to `af1da4c`, the merge commit of apache/iceberg-rust#3263 ([22 commits](https://github.com/apache/iceberg-rust/compare/bb1e4a4...af1da4c)). - Add `spark.comet.iceberg.ioTimeout` (default `30s`). The native Iceberg scan and write pass it to iceberg-rust as `opendal.io-timeout-ms`. - Add `opendal.` to the native `STORAGE_PROPERTY_PREFIXES` so the property reaches `FileIO`. The merged upstream key is `opendal.io-timeout-ms`, not the `client.io-timeout-ms` the issue expected, so this is needed. - API updates the bump requires: `FileWrite::close` now returns `FileMetadata`, and `UnboundPartitionField` fields are private. ## How are these changes tested? - New native unit test `io_timeout_reaches_the_file_io`: the key matches iceberg-rust's `OPENDAL_IO_TIMEOUT_MS` and passes the storage-prefix filter. - Every native Iceberg scan and write now carries the property, so the existing Iceberg suites run with it set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
