andygrove commented on code in PR #2038:
URL:
https://github.com/apache/datafusion-ballista/pull/2038#discussion_r3588128164
##########
ballista/core/proto/ballista.proto:
##########
@@ -419,9 +429,14 @@ message ExecutorSpecification {
repeated ExecutorResource resources = 1;
}
+// Virtual cores this executor has been assigned at runtime — analogous to
+// YARN vcores (yarn.nodemanager.resource.cpu-vcores) or Spark's
+// spark.executor.cores. Not physical `nproc`: may over- or under-subscribe.
+// One task per executor drives this many DataFusion partitions in parallel
+// through one plan-Arc.
message ExecutorResource {
oneof resource {
- uint32 task_slots = 1;
Review Comment:
If it's not too much trouble, I think it would be good to go ahead with a
standalone PR to do this rename to `vcores`. I personally think it makes more
sense than the current naming, because it allows for more flexibility in the
execution model, including for the proposed approach in the current PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
