stuhood commented on issue #23436: URL: https://github.com/apache/datafusion/issues/23436#issuecomment-6279636564
> * The modulo-arithmetic for `Hash` let's us support it pretty transparently in DFD in a pretty elegant way. `Range` is fundamentally different with sorted split points. Distributing the data and implementing shuffles on `Range` partitioned data requires different algorithms. If we add `KeyPartitioning` into the mix, we may want a different shuffle implementation again. That's all to say - I wonder if an `Enum` actually is the right way to capture this difference. @jayshrivastava : How does https://github.com/apache/datafusion/issues/25483 play into your thinking? It feels like that issue presents a pretty elegant approach to unifying re-partitions under `Hash` and `Range` (and effectively: how partitions map to one another), which aligns with how shuffles would be executed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
