alamb commented on issue #21301: URL: https://github.com/apache/datafusion/issues/21301#issuecomment-5639239144
My thinking on this issue has evolved: I personally believe we should leave `datafusion-variant` outside the core datafusion repo, and use it as an experiment / proving grounds for "best practices for ecosystem integration". It seems like what `datafusion-variant` currently lacks is maintainer bandwidth to push it forward, for example there are PRs such as https://github.com/datafusion-contrib/datafusion-variant/pull/57 which have been stuck for a while. I suspect @friendlymatthew has been busy with other work. The fact that @sdf-jkl has stepped up and wants to help I think is a great opportunity. As @adriangb pointed out to me in a call, `datafusion-variant` is relatively immature compared to the rest of the `datafusion` core crates and thus is not clear that putting `datafusion-variant` in the main DataFusion repo would actually speed up its development. The only open question in my mind is if it is ok that the variant integration stays in [datafusion-contrib](https://github.com/datafusion-contrib), which will limit some contributors compared to if it were in `apache/datafusion-variant` (e.g. Apple employees are restricted in what they can contribute to) My proposed plan of action: 1. Add @sdf-jkl as a committer to `datafusion-variant` (see https://github.com/datafusion-contrib/datafusion-variant/issues/65) 2. See how far we can get with an integration in that repository 3. If there is some reason putting the code in another repo makes sense, we can revist that. cc @Samyak2 and @bharath-techie who I think have also expressed interest in this code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
