Hi
I believe there is no nice way to prevent preemption from specific queue.
Only thing we can do is disabling preemption completely.
<property>
<name>yarn.scheduler.fair.preemption</name>
<value>true</value>
</property>
From Hadoop 2.9.0 or 3.0.0-alpha1 we will be able to use allowPreemptionFrom
property to forbid preemption from the queue.
<queue name=“highPriority">
<allowPreemptionFrom>false</allowPreemptionFrom>
</queue>
https://issues.apache.org/jira/browse/YARN-4462
<https://issues.apache.org/jira/browse/YARN-4462>
Thanks
Kai Sasaki
> On May 15, 2017, at 16:17, wuchang <[email protected]> wrote:
>
> Below is my fair-scheduler.xml file:
>
> <allocations>
> <queue name="highPriority">
> <minResources>100000 mb, 30 vcores</minResources>
> <maxResources>500000 mb, 100 vcores</maxResources>
> <weight>0.35</weight>
> <minSharePreemptionTimeout>20</minSharePreemptionTimeout>
> <fairSharePreemptionTimeout>25</fairSharePreemptionTimeout>
> <fairSharePreemptionThreshold>0.8</fairSharePreemptionThreshold>
> </queue>
> <queue name="default">
> <minResources>25000 mb, 20 vcores</minResources>
> <maxResources>225000 mb, 70 vcores</maxResources>
> <weight>0.14</weight>
> <minSharePreemptionTimeout>20</minSharePreemptionTimeout>
> <fairSharePreemptionTimeout>25</fairSharePreemptionTimeout>
> <fairSharePreemptionThreshold>0.5</fairSharePreemptionThreshold>
> <maxAMShare>-1.0f</maxAMShare>
> </queue>
> <queue name="ep">
> <minResources>100000 mb, 30 vcores</minResources>
> <maxResources>600000 mb, 100 vcores</maxResources>
> <weight>0.42</weight>
> <minSharePreemptionTimeout>20</minSharePreemptionTimeout>
> <fairSharePreemptionTimeout>25</fairSharePreemptionTimeout>
> <fairSharePreemptionThreshold>0.8</fairSharePreemptionThreshold>
> <maxAMShare>-1.0f</maxAMShare>
> </queue>
> <queue name="vip">
> <minResources>15000 mb, 20 vcores</minResources>
> <maxResources>120000 mb, 30 vcores</maxResources>
> <weight>0.09</weight>
> <minSharePreemptionTimeout>20</minSharePreemptionTimeout>
> <fairSharePreemptionTimeout>25</fairSharePreemptionTimeout>
> <fairSharePreemptionThreshold>0.8</fairSharePreemptionThreshold>
> <maxAMShare>-1.0f</maxAMShare>
> </queue>
> </allocations>
>
> Since the root.ep queue and root.highPriority queue is so important that
> anytime I don’t want them to be preempted.I know that if the resource of them
> is preempted , their container maybe killed and thus it may take much more
> time for applications running on them to finish.
> So , Any solutions?
