brumi1024 commented on a change in pull request #3330:
URL: https://github.com/apache/hadoop/pull/3330#discussion_r698356952
##########
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
##########
@@ -681,6 +689,55 @@ list of current scheduling edit policies as a comma
separated string in `yarn.re
|:---- |:---- |
| `yarn.resourcemanager.monitor.capacity.queue-management.monitoring-interval`
| Time in milliseconds between invocations of this
QueueManagementDynamicEditPolicy policy. Default value is 1500 |
+* Parent queue configuration for **flexible** dynamic leaf queue auto-creation
and management
+
+The `Flexible Dynamic Queue Auto-Creation and Management` feature allows a
**ParentQueue** to auto-create both parent and leaf queues. Such parent queues
can have other pre-configured queues co-existing with the auto-created queues.
The auto-created queues will have weights as *capacity* so the pre-configured
queues under the parent must be configured the same way. The
`CapacityScheduler` supports the following parameters to enable auto-creation
of queues
+
+| Property | Description |
+|:---- |:---- |
+| `yarn.scheduler.capacity.<queue-path>.auto-queue-creation-v2.enabled` |
*Mandatory* parameter: Indicates to the `CapacityScheduler` that flexible auto
queue creation needs to be enabled for the specified parent queue. Boolean
value expected. The default value is *false*, i.e. auto leaf queue creation is
not enabled in *ParentQueue* by default. |
+| `yarn.scheduler.capacity.<queue-path>.auto-queue-creation-v2.max-queues` |
*Optional* parameter: Limits the number of dynamic queues created under a
parent queue. Integer value expected. The default value is *1000*. |
+
+
+* Configuring **flexible** `Auto-Created Leaf Queues` with `CapacityScheduler`
+
+The parent queue which has the flexible auto queue creation enabled supports
the configuration of dynamically created leaf and parent queues through
template parameters. The auto-created queues support all of the leaf queue
configuration parameters except for **Queue ACL**, **Absolute Resource**
configurations. Queue ACLs are currently inherited from the parent queue i.e
they are not configurable on the leaf queue template
+
+| Property | Description |
+|:---- |:---- |
+|
`yarn.scheduler.capacity.<queue-path>.auto-queue-creation-v2.template.<queue-property>`
| *Optional* parameter: Specifies a queue property (like capacity,
maximum-capacity, user-limit-factor, maximum-am-resource-percent ... - Refer
**Queue Properties** section) inherited by the auto-created **parent** and
**leaf** queues. |
+|
`yarn.scheduler.capacity.<queue-path>.auto-queue-creation-v2.leaf-template.<queue-property>`
| *Optional* parameter: Specifies a queue property inherited by auto-created
**leaf** queues. |
+|
`yarn.scheduler.capacity.<queue-path>.auto-queue-creation-v2.parent-template.<queue-property>`
| *Optional* parameter: Specifies a queue property inherited by auto-created
**parent** queues. |
+
+Using the following example configuration snippet will instruct the
`CapacityScheduler` to:
+* enable the flexible auto queue creation for root.parent
+* create **all** of the dynamic queues below root.parent with 80% as the
maximum capacity, because of the wildcard queue path (root.parent.*)
Review comment:
This line is for the property
yarn.scheduler.capacity.root.parent.*.auto-queue-creation-v2.template.maximum-capacity,
which has a simple template tag, not the parent-template. Doesn't this apply
to all of the queues, not just the parents?
Or do you mean that this overrides the static queues as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
