wirybeaver opened a new pull request, #2389: URL: https://github.com/apache/datafusion-ballista/pull/2389
## Dependencies - Stacked on #2388; review the second commit (`feat: add Hudi read integration`) for this PR's own diff. - Uses the DataFusion 55 compatibility work in apache/hudi-rs#698, pinned to its current commit until that change is merged and released. ## Which issue does this PR close? Proves the extension seam proposed for #1241 with a real open-table-format read. ## What changes are included? Adds an independently built `integrations/hudi` crate rather than adding Hudi to Ballista core or the main workspace. - implements `BallistaTableProviderCodec` for `HudiDataSource` - transports only the base URI and caller-supplied options, then reconstructs the provider on the scheduler - provides helpers to install the codec and register a Hudi table - uses hudi-rs's standard Parquet `DataSourceExec` path for COW reads, so no Hudi physical node is required - adds one focused CI job for the independent crate Scope is read-only Parquet copy-on-write. MOR custom execution, writes, dynamic loading, and broad fallback machinery are intentionally excluded. ## Tests - `cd integrations/hudi && cargo clippy --all-targets --no-deps -- -D warnings` - `cd integrations/hudi && cargo test --test distributed_read` The test runs a projected and filtered Hudi COW query through a two-worker Ballista standalone cluster and verifies the resulting rows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
