alamb opened a new pull request, #23170: URL: https://github.com/apache/datafusion/pull/23170
## Which issue does this PR close? - Related to https://github.com/apache/datafusion/pull/21882#issuecomment-4793942224. ## Rationale for this change PR #21882 adds custom spill file support. This PR adds a small example showing how users can back spill files with an `ObjectStore`, using a local object store for a runnable example while keeping the implementation applicable to remote stores. ## What changes are included in this PR? - Adds a `data_io` example that implements `TempFileFactory`, `SpillFile`, and `SpillWriter` over `Arc<dyn ObjectStore>`. - Wires the example into the `data_io` example runner and README. - Enables the `fs` feature for the examples' `object_store` dependency so the example can use `LocalFileSystem`. ## Are these changes tested? Yes: ```bash cargo fmt --all cargo run -p datafusion-examples --example data_io -- object_store_spill git diff --check ``` ## Are there any user-facing changes? Yes. This adds a new example for configuring ObjectStore-backed spill files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
