[
https://issues.apache.org/jira/browse/SPARK-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972703#comment-16972703
]
Imran Rashid commented on SPARK-29762:
--------------------------------------
The scenario I was thinking is:
application is submitted with app-wide resource requirements for 4 cores & 2
gpus executors, but not for tasks. (I guess they specify
"spark.executor.resource.gpu" but not "spark.task.resource.gpu".).
One taskset is submitted with no task resource requirements, because it doesn't
need any gpus.
Another taskset is submitted which does require 1 cpu & 1 gpu per task.
The user does it this way because they don't want all of their tasks to use
gpus, but they know enough of them will need gpus that they'd rather just
request gpus upfront, than scale up & down two different types of executors.
> GPU Scheduling - default task resource amount to 1
> --------------------------------------------------
>
> Key: SPARK-29762
> URL: https://issues.apache.org/jira/browse/SPARK-29762
> Project: Spark
> Issue Type: Story
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: Thomas Graves
> Priority: Major
>
> Default the task level resource configs (for gpu/fpga, etc) to 1. So if the
> user specifies the executor resource then to make it more user friendly lets
> have the task resource config default to 1. This is ok right now since we
> require resources to have an address. It also matches what we do for the
> spark.task.cpus configs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
