andygrove opened a new pull request, #2496: URL: https://github.com/apache/datafusion-ballista/pull/2496
> **Stacked on #2416.** GitHub cannot base a PR on a fork branch, so all of #2416 shows here until it merges. Review only the last commit, `feat: add a Python ADBC example and TPC-H runner for Flight SQL`. # Which issue does this PR close? Part of #2298. Does not close it. # Rationale for this change #2416 adds an Arrow Flight SQL frontend to the scheduler. These two scripts exercise it from the client side through the generic `adbc_driver_flightsql` driver alone, with no Ballista client library, which is the path a JDBC/ODBC or BI-tool client takes. They were split out of #2416 to keep that PR focused on the frontend. # What changes are included in this PR? - `examples/python/adbc_flight_sql.py` registers the repo's sample CSV, runs a distributed aggregate over it, and lists table types through catalog introspection. `examples/README.md` gains a section on running it. - `benchmarks/tpch_adbc.py` runs the 22 TPC-H queries through the frontend. It mirrors the Rust harness where the two must agree: the same table list, the same `find_path` resolution for object-store URLs, the same statement splitting, and the same "last SELECT or WITH is the answer" rule that q15 needs. It reports `plan+exec` (`GetFlightInfo`) and `fetch` (`DoGet`) separately, because `GetFlightInfo` blocks for the whole query until `PollFlightInfo` is implemented, and that is what `--timeout` has to be sized against. `benchmarks/README.md` documents it. - The Flight SQL user guide links to both. Both scripts drop each table before creating it, so they can be re-run against a scheduler that has already served them. # How are these changes tested? The files are unchanged from #2416. Its manual testing ran all 22 TPC-H queries over `adbc_driver_flightsql` against a scheduler and executor on TPC-H SF100 Parquet, twice on the same scheduler without a restart. Both scripts pass `ruff check` and `ruff format --check` at the version CI pins. CI only runs ruff for changes under `python/`, so it does not check them on this PR. # Are there any user-facing changes? A new Python example and benchmark script, with README and user guide sections for them. No Rust changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
