timsaucer opened a new pull request, #1677: URL: https://github.com/apache/datafusion-python/pull/1677
# Which issue does this PR close? Related to #1612. This PR does not close it, but provides the FFI query planner plumbing that a `datafusion-distributed` integration can build on. This is part 1 of 3 in the split of #1672 (part 2: composable extension codecs; part 3: atomic `SessionContext.with_extensions`). # Rationale for this change Extension libraries (for example distributed execution engines) need to supply their own `QueryPlanner` to a `SessionContext` without compiling against the `datafusion-python` crate. This PR exposes the query planner over the FFI boundary, following the same PyCapsule pattern used for table providers and catalogs. # What changes are included in this PR? - `SessionContext.with_query_planner(planner)` installs a planner exported via a `__datafusion_query_planner__` PyCapsule, preserving existing session state and codec settings. - `SessionContext.__datafusion_query_planner__()` exports the current planner so another planner can wrap it as an explicit fallback (a session holds exactly one planner; layering is explicit delegation). - A `RuntimeAwareQueryPlanner` adapter binds foreign planners to the Tokio runtime owned by datafusion-python. - New example crate `datafusion-ffi-query-planner-example` demonstrating a real three-library plan exchange (host, provider library, planner library as separate cdylibs), including session config transfer via `SessionConfig.with_extension`. - New `docs/source/contributor-guide/ffi.md` sections covering the query planner capsule protocol and the three-library setup. # Are there any user-facing changes? New public APIs: `SessionContext.with_query_planner` and `SessionContext.__datafusion_query_planner__`. A new example crate ships under `examples/`. No breaking changes to existing APIs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
