jerqi commented on issue #172: URL: https://github.com/apache/incubator-uniffle/issues/172#issuecomment-1229465774
> For your thought, do I need to submit a data-locality shuffle patch about Spark firstly? As I know, we need to change the Spark codebase about `MapOutputTracker.getPreferredLocationsForShuffle` and `ShuffleRowRDD.getPreferredLocations` . When enable RSS, the perferred locations could be gotten from the dep.shuffleHandle(RssShuffleHandle of Uniffle partitionToServers vars). If I'm wrong, plz point out it. Thanks > > I still think this is a necessary feature for co-location deployment. Besides this has been implemented by other RSS project. Not only the spark patch, but also we need to modify the strategy of coordinator. I know the Bytedance RSS implement this feature, if the cluster is big enough, the feature will be useless. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
