This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new ab5109e24e1 [SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS`
to override JVM memory setting
ab5109e24e1 is described below
commit ab5109e24e142efd4e26a0dba2cd0e1e5a788b3d
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Jan 26 16:39:17 2023 -0800
[SPARK-42201][BUILD] `build/sbt` should allow `SBT_OPTS` to override JVM
memory setting
### What changes were proposed in this pull request?
This PR aims to fix a bug which `build/sbt` doesn't allow JVM memory
setting via `SBT_OPTS`.
### Why are the changes needed?
`SBT_OPTS` is supposed to be used in this way in the community.
https://github.com/apache/spark/blob/e30bb538e480940b1963eb14c3267662912d8584/appveyor.yml#L54
However, `SBT_OPTS` memory setting like the following is ignored because `
-Xms4096m -Xmx4096m -XX:ReservedCodeCacheSize=512m` is injected by default
after `SBT_OPTS`. We should switch the order.
```
$ SBT_OPTS="-Xmx6g" build/sbt package
```
https://github.com/apache/spark/blob/e30bb538e480940b1963eb14c3267662912d8584/build/sbt-launch-lib.bash#L124
### Does this PR introduce _any_ user-facing change?
No. This is a dev-only change.
### How was this patch tested?
Manually run the following.
```
$ SBT_OPTS="-Xmx6g" build/sbt package
```
While running the above command, check the JVM options.
```
$ ps aux | grep java
dongjoon 36683 434.3 3.1 418465456 1031888 s001 R+ 1:11PM
0:19.86 /Users/dongjoon/.jenv/versions/temurin17/bin/java -Xms4096m -Xmx4096m
-XX:ReservedCodeCacheSize=512m -Xmx6g -jar build/sbt-launch-1.8.2.jar package
```
Closes #39758 from dongjoon-hyun/SPARK-42201.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 66ec1eb630a4682f5ad2ed2ee989ffcce9031608)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
build/sbt-launch-lib.bash | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/build/sbt-launch-lib.bash b/build/sbt-launch-lib.bash
index 8fb6672bddc..01ba6b929f9 100755
--- a/build/sbt-launch-lib.bash
+++ b/build/sbt-launch-lib.bash
@@ -183,8 +183,8 @@ run() {
# run sbt
execRunner "$java_cmd" \
- ${SBT_OPTS:-$default_sbt_opts} \
$(get_mem_opts $sbt_mem) \
+ ${SBT_OPTS:-$default_sbt_opts} \
${java_opts} \
${java_args[@]} \
-jar "$sbt_jar" \
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
