Repository: spark Updated Branches: refs/heads/master 5eef1e6c6 -> 5bf2889bf
[SPARK-16138] Try to cancel executor requests only if we have at least 1 ## What changes were proposed in this pull request? Adding additional check to if statement ## How was this patch tested? I built and deployed to internal cluster to observe behaviour. After the change the invalid logging is gone: ``` 16/06/22 08:46:36 INFO yarn.YarnAllocator: Driver requested a total number of 1 executor(s). 16/06/22 08:46:36 INFO yarn.YarnAllocator: Canceling requests for 1 executor container(s) to have a new desired total 1 executors. 16/06/22 08:46:36 INFO yarn.YarnAllocator: Driver requested a total number of 0 executor(s). 16/06/22 08:47:36 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 1. ``` Author: Peter Ableda <[email protected]> Closes #13850 from peterableda/patch-2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5bf2889b Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5bf2889b Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/5bf2889b Branch: refs/heads/master Commit: 5bf2889bfcfd776e7ee1369443a0474421a800bd Parents: 5eef1e6 Author: Peter Ableda <[email protected]> Authored: Thu Jun 23 09:00:31 2016 -0500 Committer: Tom Graves <[email protected]> Committed: Thu Jun 23 09:00:31 2016 -0500 ---------------------------------------------------------------------- .../main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/5bf2889b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---------------------------------------------------------------------- diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala index 1b80071..dbdac33 100644 --- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala +++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala @@ -352,7 +352,7 @@ private[yarn] class YarnAllocator( logInfo(s"Submitted container request (host: ${hostStr(request)}, capability: $resource)") } - } else if (missing < 0) { + } else if (numPendingAllocate > 0 && missing < 0) { val numToCancel = math.min(numPendingAllocate, -missing) logInfo(s"Canceling requests for $numToCancel executor container(s) to have a new desired " + s"total $targetNumExecutors executors.") --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
