adriangb commented on PR #21480: URL: https://github.com/apache/datafusion/pull/21480#issuecomment-4215673477
One thing I wonder is if we can avoid per-file rewrites, pruning predicate creation, etc. by caching these things. In most systems you might have 2-3 different physical schemas (different writers, etc.) but often times just 1 so re-doing the exact same work across many files is wasteful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
