zhuqi-lucas commented on PR #21956: URL: https://github.com/apache/datafusion/pull/21956#issuecomment-4456649095
Thanks @adriangb, you're right. Routing through `try_pushdown_sort` matches what that channel is for, decouples the reorder from TopK (so it helps any ORDER BY on a sorted source, not just LIMIT-K), and lets the `sort_options`/`fetch` fields on `DynamicFilterPhysicalExpr` go away — along with the `find_dynamic_filter` walk under AND/wrappers and the `with_fetch`/`create_filter` ordering coupling, which were the parts I was least happy about anyway. Can't think of a case where the sort metadata reaches parquet through the dynamic filter but not through `try_pushdown_sort` — same SortExec → DataSource shape, dynamic filter just happens to be what SortExec attaches to its predicate. Let me refactor along the lines you sketched. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
