dacort commented on code in PR #407: URL: https://github.com/apache/incubator-livy/pull/407#discussion_r1227180190
########## dev/docker/livy-dev-base/Dockerfile: ########## Review Comment: We need to update the main project README with this new location. Either update the `cd incubator-livy/dev/docker` and `docker run` commands or just the `docker build` command with the extra directory. ########## dev/docker/build-images.sh: ########## Review Comment: `chmod 755` this file. ########## dev/docker/README.md: ########## @@ -0,0 +1,116 @@ +# Livy with standalone Spark Cluster +## Pre-requisite +Following steps use Ubuntu as development environment but most of the instructions can be modified to fit another OS as well. +* Install wsl if on windows, instructions available [here](https://ubuntu.com/tutorials/install-ubuntu-on-wsl2-on-windows-11-with-gui-support) +* Install docker engine, instructions available [here](https://docs.docker.com/engine/install/ubuntu/) +* Install docker-compose, instructions available [here](https://docs.docker.com/compose/install/) + +## Standalone cluster using docker-compose +### Build the current livy branch and copy it to appropriate folder +``` +$ mvn clean package -Pscala-2.12 -Pspark3 -DskipITs -DskipTests Review Comment: We may want to call out the important of `clean` here. I had previously packaged with scala-2.11 and my `livy-dev-server` wouldn't start because of a version conflict with the below error message: ``` Exception in thread "main" java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V ``` ########## pom.xml: ########## @@ -1180,10 +1179,11 @@ <profile> <id>spark3</id> <properties> - <spark.version>3.0.0</spark.version> + <spark.version>3.2.3</spark.version> Review Comment: Are we OK bumping the version of Spark here? I like it, just want to make sure it doesn't break anything... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
