wirybeaver commented on issue #1241: URL: https://github.com/apache/datafusion-ballista/issues/1241#issuecomment-5631267444
@milenkovicm datafusion-comet already support Delta-lake in the same way: add a contrib build gate with a Table format specific provider interface and no dependency; the implementation will be in a different crate to avoid the release of the main datafusion-comet. I have a commit to support Lance in datafusion-comet that is close to get merged following the same pattern https://github.com/apache/datafusion-comet/pull/5728 Thus, I think datafusion-ballista can adopt the same paradigm for any Table format support. The PR #2388 is a demonstration using Hudi. I can use Lance if you have any concern on the Hudi rust, though hudi rust will be in a different crate and not impact the major ballista release. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
