wirybeaver commented on issue #4632: URL: https://github.com/apache/datafusion-comet/issues/4632#issuecomment-4720972645
Hi @andygrove, could you take a look at this proposal when you have a chance? The issue body includes an ASCII architecture sketch for the proposed Lance native read path. The intended boundary is: Spark/lance-spark remains the planning authority and exposes `LanceScan.nativeScanPlan()`, while Comet consumes that descriptor through optional `contrib-lance` support and executes the assigned fragments with Rust Lance. There is a draft prototype in #4633, with the companion Lance Spark descriptor PR at lance-format/lance-spark#624. I would especially appreciate your feedback on whether this fits Comet's contrib/native scan patterns, and on the open Arrow C Data packaging/classpath blocker before this could move beyond draft. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
