9uapaw commented on a change in pull request #3330:
URL: https://github.com/apache/hadoop/pull/3330#discussion_r698268877
##########
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
##########
@@ -123,17 +123,25 @@ Configuration
| Property | Description |
|:---- |:---- |
-| `yarn.scheduler.capacity.<queue-path>.capacity` | Queue *capacity* in
percentage (%) as a float (e.g. 12.5) OR as absolute resource queue minimum
capacity. The sum of capacities for all queues, at each level, must be equal to
100. However if absolute resource is configured, sum of absolute resources of
child queues could be less than it's parent absolute resource capacity.
Applications in the queue may consume more resources than the queue's capacity
if there are free resources, providing elasticity. |
-| `yarn.scheduler.capacity.<queue-path>.maximum-capacity` | Maximum queue
capacity in percentage (%) as a float OR as absolute resource queue maximum
capacity. This limits the *elasticity* for applications in the queue. 1) Value
is between 0 and 100. 2) Admin needs to make sure absolute maximum capacity >=
absolute capacity for each queue. Also, setting this value to -1 sets maximum
capacity to 100%. |
+| `yarn.scheduler.capacity.<queue-path>.capacity` | Queue *capacity* in
percentage (%) as a float (e.g. 12.5), weight as a float with the postfix *w*
(e.g. 2.0w) or as absolute resource queue minimum capacity. When using
percentage values the sum of capacities for all queues, at each level, must be
equal to 100. If absolute resource is configured, sum of absolute resources of
child queues could be less than its parent absolute resource capacity.
Applications in the queue may consume more resources than the queue's capacity
if there are free resources, providing elasticity. |
+| `yarn.scheduler.capacity.<queue-path>.maximum-capacity` | Maximum queue
capacity in percentage (%) as a float (when the *capacity* property is defined
with either percentages or weights) or as absolute resource queue maximum
capacity. This limits the *elasticity* for applications in the queue. 1) Value
is between 0 and 100. 2) Admin needs to make sure absolute maximum capacity >=
absolute capacity for each queue. Also, setting this value to -1 sets maximum
capacity to 100%. |
| `yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent` | Each
queue enforces a limit on the percentage of resources allocated to a user at
any given time, if there is demand for resources. The user limit can vary
between a minimum and maximum value. The former (the minimum value) is set to
this property value and the latter (the maximum value) depends on the number of
users who have submitted applications. For e.g., suppose the value of this
property is 25. If two users have submitted applications to a queue, no single
user can use more than 50% of the queue resources. If a third user submits an
application, no single user can use more than 33% of the queue resources. With
4 or more users, no user can use more than 25% of the queues resources. A value
of 100 implies no user limits are imposed. The default is 100. Value is
specified as a integer. |
| `yarn.scheduler.capacity.<queue-path>.user-limit-factor` | User limit factor
provides a way to control the max amount of resources that a single user can
consume. It is the multiple of the queue's capacity. By default this is set to
1 which ensures that a single user can never take more than the queue's
configured capacity irrespective of how idle the cluster is. Increasing it
means a single user can use more than the minimum capacity of the cluster,
while decreasing it results in lower maximum resources. Setting this to -1 will
disable the feature. Value is specified as a float. Note: using the flexible
auto queue creation
(yarn.scheduler.capacity.\<queue-path\>.auto-queue-creation-v2) with weights
will automatically set this property to -1, as the dynamic queues will be
created with the hardcoded weight of 1 and in idle cluster scenarios they
should be able to use more resources than calculated. |
| `yarn.scheduler.capacity.<queue-path>.maximum-allocation-mb` | The per queue
maximum limit of memory to allocate to each container request at the Resource
Manager. This setting overrides the cluster configuration
`yarn.scheduler.maximum-allocation-mb`. This value must be smaller than or
equal to the cluster maximum. |
| `yarn.scheduler.capacity.<queue-path>.maximum-allocation-vcores` | The per
queue maximum limit of virtual cores to allocate to each container request at
the Resource Manager. This setting overrides the cluster configuration
`yarn.scheduler.maximum-allocation-vcores`. This value must be smaller than or
equal to the cluster maximum. |
| `yarn.scheduler.capacity.<queue-path>.user-settings.<user-name>.weight` |
This floating point value is used when calculating the user limit resource
values for users in a queue. This value will weight each user more or less than
the other users in the queue. For example, if user A should receive 50% more
resources in a queue than users B and C, this property will be set to 1.5 for
user A. Users B and C will default to 1.0. |
- * Resource Allocation using Absolute Resources configuration
+ * Configuring Resource Allocation
- `CapacityScheduler` supports configuration of absolute resources instead of
providing Queue *capacity* in percentage. As mentioned in above configuration
section for `yarn.scheduler.capacity.<queue-path>.capacity` and
`yarn.scheduler.capacity.<queue-path>.max-capacity`, administrator could
specify an absolute resource value like `[memory=10240,vcores=12]`. This is a
valid configuration which indicates 10GB Memory and 12 VCores.
+ `CapacityScheduler` supports three different resource allocation
configuration modes: percentage values (*relative mode*), weights and absolute
resources.
+
+ Relative mode provides a way to describe queue's resources as a fraction of
its parent's resources. For example if *capacity* is set as 50.0, users queue
has 50% of its parent, root's resources set as minimum capacity.
Review comment:
The last sentence is hard to understand, could you rephrase it please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
