martin-g commented on code in PR #1578:
URL:
https://github.com/apache/datafusion-ballista/pull/1578#discussion_r3161295444
##########
ballista/core/src/execution_plans/shuffle_reader.rs:
##########
@@ -94,19 +96,49 @@ impl ShuffleReaderExec {
partition,
metrics: ExecutionPlanMetricsSet::new(),
properties,
- work_dir: None, // to be updated at the executor side
+ work_dir: None, // to be updated at the executor side
+ client_pool: None, // to be updated at the executor side
})
}
/// changes work dir where shuffle files are located
- pub fn change_work_dir(&self, work_dir: String) -> Self {
+ pub fn with_work_dir(&self, work_dir: String) -> Self {
Self {
stage_id: self.stage_id,
schema: self.schema.clone(),
partition: self.partition.clone(),
metrics: self.metrics.clone(),
properties: self.properties.clone(),
work_dir: Some(work_dir),
+ client_pool: self.client_pool.clone(),
+ }
+ }
+ /// creates new shuffle reader with client pool
+ pub fn with_client_pool(&self, client_pool: Arc<dyn BallistaClientPool>)
-> Self {
+ Self {
+ stage_id: self.stage_id,
+ schema: self.schema.clone(),
+ partition: self.partition.clone(),
+ metrics: self.metrics.clone(),
+ properties: self.properties.clone(),
+ work_dir: self.work_dir.clone(),
+ client_pool: Some(client_pool),
+ }
+ }
+ /// creates new shuffle reader with client pool and work dir
+ pub fn with_client_pool_and_work_dir(
Review Comment:
Is this method really needed ?
The user could do it in two steps:
`with_client_pool(client_pool).with_work_dir(work_dir)`
At the moment `fn with_client_pool()` is not used anywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
