Repository: spark
Updated Branches:
refs/heads/branch-2.1 f94646415 -> 7c23bd49e
[SPARK-19432][CORE] Fix an unexpected failure when connecting timeout
## What changes were proposed in this pull request?
When connecting timeout, `ask` may fail with a confusing message:
```
17/02/01 23:15:19 INFO Worker: Connecting to master ...
java.lang.IllegalArgumentException: requirement failed: TransportClient has not
yet been set.
at scala.Predef$.require(Predef.scala:224)
at
org.apache.spark.rpc.netty.RpcOutboxMessage.onTimeout(Outbox.scala:70)
at
org.apache.spark.rpc.netty.NettyRpcEnv$$anonfun$ask$1.applyOrElse(NettyRpcEnv.scala:232)
at
org.apache.spark.rpc.netty.NettyRpcEnv$$anonfun$ask$1.applyOrElse(NettyRpcEnv.scala:231)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:138)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:136)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
```
It's better to provide a meaningful message.
## How was this patch tested?
Jenkins
Author: Shixiong Zhu <[email protected]>
Closes #16773 from zsxwing/connect-timeout.
(cherry picked from commit 8303e20c45153f91e585e230caa29b728a4d8c6c)
Signed-off-by: Shixiong Zhu <[email protected]>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7c23bd49
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7c23bd49
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7c23bd49
Branch: refs/heads/branch-2.1
Commit: 7c23bd49e826fc2b7f132ffac2e55a71905abe96
Parents: f946464
Author: Shixiong Zhu <[email protected]>
Authored: Wed Feb 1 21:39:21 2017 -0800
Committer: Shixiong Zhu <[email protected]>
Committed: Wed Feb 1 21:39:30 2017 -0800
----------------------------------------------------------------------
core/src/main/scala/org/apache/spark/rpc/netty/Outbox.scala | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/7c23bd49/core/src/main/scala/org/apache/spark/rpc/netty/Outbox.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/rpc/netty/Outbox.scala
b/core/src/main/scala/org/apache/spark/rpc/netty/Outbox.scala
index 6c090ad..a7b7f58 100644
--- a/core/src/main/scala/org/apache/spark/rpc/netty/Outbox.scala
+++ b/core/src/main/scala/org/apache/spark/rpc/netty/Outbox.scala
@@ -56,7 +56,7 @@ private[netty] case class RpcOutboxMessage(
content: ByteBuffer,
_onFailure: (Throwable) => Unit,
_onSuccess: (TransportClient, ByteBuffer) => Unit)
- extends OutboxMessage with RpcResponseCallback {
+ extends OutboxMessage with RpcResponseCallback with Logging {
private var client: TransportClient = _
private var requestId: Long = _
@@ -67,8 +67,11 @@ private[netty] case class RpcOutboxMessage(
}
def onTimeout(): Unit = {
- require(client != null, "TransportClient has not yet been set.")
- client.removeRpcRequest(requestId)
+ if (client != null) {
+ client.removeRpcRequest(requestId)
+ } else {
+ logError("Ask timeout before connecting successfully")
+ }
}
override def onFailure(e: Throwable): Unit = {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
