avantgardnerio commented on code in PR #2038:
URL:
https://github.com/apache/datafusion-ballista/pull/2038#discussion_r3587377630
##########
.github/workflows/tpch.yml:
##########
@@ -111,7 +111,7 @@ jobs:
--bind-port 50051 \
--scheduler-host 127.0.0.1 \
--scheduler-connect-timeout-seconds 10 \
- --concurrent-tasks 4 \
+ --vcores 4 \
Review Comment:
Because it only represented concurrent tasks in the old model, where 1 task
consumed 1 partition, which is generally modeled to use 1 core (unless
deliberately overridden to over/under subscribe).
In the new model, the scheduler will attempt to assign as many partitions as
possible to a single task. So the `task_slot` / `concurrent-tasks` was no
longer semantically correct.
I needed a new term. I felt `vcores` was the most clear / concise. It
roughly represents number of cores, but not really due to over/under
subscription, kubernetes / docker, hyperthreading - thus the `v`.
I'm happy to call it whatever we decide as long as the new term is
semantically correct / unambiguous (the old `partitions` turned into 3 things
in the new model).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
