Repository: spark
Updated Branches:
refs/heads/master cdd53b762 -> 0d7aac99d
[SPARK-9641] [DOCS] spark.shuffle.service.port is not documented
Document spark.shuffle.service.{enabled,port}
CC sryza tgravescs
This is pretty minimal; is there more to say here about the service?
Author: Sean Owen <[email protected]>
Closes #7991 from srowen/SPARK-9641 and squashes the following commits:
3bb946e [Sean Owen] Add link to docs for setup and config of external shuffle
service
2302e01 [Sean Owen] Document spark.shuffle.service.{enabled,port}
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0d7aac99
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0d7aac99
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0d7aac99
Branch: refs/heads/master
Commit: 0d7aac99da660cc42eb5a9be8e262bd9bd8a770f
Parents: cdd53b7
Author: Sean Owen <[email protected]>
Authored: Thu Aug 6 19:29:42 2015 +0100
Committer: Sean Owen <[email protected]>
Committed: Thu Aug 6 19:29:42 2015 +0100
----------------------------------------------------------------------
docs/configuration.md | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/0d7aac99/docs/configuration.md
----------------------------------------------------------------------
diff --git a/docs/configuration.md b/docs/configuration.md
index 24b6063..c60dd16 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -474,6 +474,25 @@ Apart from these, the following properties are also
available, and may be useful
</td>
</tr>
<tr>
+ <td><code>spark.shuffle.service.enabled</code></td>
+ <td>false</td>
+ <td>
+ Enables the external shuffle service. This service preserves the shuffle
files written by
+ executors so the executors can be safely removed. This must be enabled if
+ <code>spark.dynamicAllocation.enabled</code> is "true". The external
shuffle service
+ must be set up in order to enable it. See
+ <a href="job-scheduling.html#configuration-and-setup">dynamic allocation
+ configuration and setup documentation</a> for more information.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.shuffle.service.port</code></td>
+ <td>7337</td>
+ <td>
+ Port on which the external shuffle service will run.
+ </td>
+</tr>
+<tr>
<td><code>spark.shuffle.sort.bypassMergeThreshold</code></td>
<td>200</td>
<td>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
