[
https://issues.apache.org/jira/browse/HADOOP-12916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15214591#comment-15214591
]
ASF GitHub Bot commented on HADOOP-12916:
-----------------------------------------
Github user arp7 commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/86#discussion_r57606818
--- Diff:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/CallQueueManager.java
---
@@ -49,25 +57,68 @@
private final AtomicReference<BlockingQueue<E>> putRef;
private final AtomicReference<BlockingQueue<E>> takeRef;
+ private RpcScheduler scheduler;
+
public CallQueueManager(Class<? extends BlockingQueue<E>> backingClass,
+ Class<? extends RpcScheduler> schedulerClass,
boolean clientBackOffEnabled, int maxQueueSize, String namespace,
Configuration conf) {
+ int priorityLevels = parseNumLevels(namespace, conf);
+ this.scheduler = createScheduler(schedulerClass, priorityLevels,
+ namespace, conf);
BlockingQueue<E> bq = createCallQueueInstance(backingClass,
- maxQueueSize, namespace, conf);
+ priorityLevels, maxQueueSize, namespace, conf);
this.clientBackOffEnabled = clientBackOffEnabled;
this.putRef = new AtomicReference<BlockingQueue<E>>(bq);
this.takeRef = new AtomicReference<BlockingQueue<E>>(bq);
LOG.info("Using callQueue " + backingClass);
}
+ private static <T extends RpcScheduler> T createScheduler(
+ Class<T> theClass, int priorityLevels, String ns, Configuration
conf) {
+ // Used for custom, configurable scheduler
+ try {
+ Constructor<T> ctor = theClass.getDeclaredConstructor(int.class,
+ String.class, Configuration.class);
+ return ctor.newInstance(priorityLevels, ns, conf);
+ } catch (RuntimeException e) {
--- End diff --
See HDFS-9478 which is fixing exception handling when constructing
callqueue instances. We could use a similar fix for createScheduler.
> Allow different Hadoop IPC Call Queue throttling policies with FCQ/BackOff
> --------------------------------------------------------------------------
>
> Key: HADOOP-12916
> URL: https://issues.apache.org/jira/browse/HADOOP-12916
> Project: Hadoop Common
> Issue Type: Improvement
> Components: ipc
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
> Attachments: HADOOP-12916.00.patch, HADOOP-12916.01.patch,
> HADOOP-12916.02.patch, HADOOP-12916.03.patch, HADOOP-12916.04.patch
>
>
> Currently back off policy from HADOOP-10597 is hard coded to base on whether
> call queue is full. This ticket is open to allow flexible back off policies
> such as moving average of response time in RPC calls of different priorities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
