Repository: zeppelin Updated Branches: refs/heads/master 75441540f -> a8ea9e1a2
[ZEPPELIN-1386] Docker images for running Apache Zeppelin releases ### What is this PR for? This PR is for making docker images for zeppelin releases. It contains a script for building image for each release. Another script is used for publishing images to zeppelin Dockerhub account. This repo, https://github.com/mfelgamal/zeppelin-dockers, is a demonstration of this PR. It contains zeppelin-base image and an image for each zeppelin release. ### What type of PR is it? [Feature] ### Todos - Review Comments - Documentation ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1386 ### How should this be tested? - run create_release script or publish_release script. ### Screenshots (if appropriate) ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? yes Author: mahmoudelgamal <mahmoudf.elga...@gmail.com> Author: mfelgamal <mahmoudf.elga...@gmail.com> Author: Mahmoud Elgamal <mahmoudf.elga...@gmail.com> Author: 1ambda <1am...@gmail.com> Closes #1538 from mfelgamal/zeppelin-dockers and squashes the following commits: cc8493f [Mahmoud Elgamal] Merge pull request #3 from 1ambda/fix/remove-startzeppelinsh d48ecef [1ambda] fix: Remove start-zeppelin.sh b64c680 [mahmoudelgamal] Remove gcc and g++ for decreasing the size 1f093d4 [mahmoudelgamal] Add script start-zeppelin to zeppelin-base d2c744e [mahmoudelgamal] add scala to zeppelin-base fd23970 [mahmoudelgamal] remove bash erorr message. e1d4b77 [mahmoudelgamal] add R and python to zeppelin-base e731cb4 [mahmoudelgamal] Add java-cacerts to zeppelin-base e642309 [mahmoudelgamal] Add documentation and some modifications 231a414 [mahmoudelgamal] Add zeppelin-base image ac06f3a [mahmoudelgamal] Make docker image for zeppelin release 48d0a01 [mfelgamal] Merge pull request #1 from apache/master Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/a8ea9e1a Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/a8ea9e1a Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/a8ea9e1a Branch: refs/heads/master Commit: a8ea9e1a263c07428f4882da46983f879c62138a Parents: 7544154 Author: mahmoudelgamal <mahmoudf.elga...@gmail.com> Authored: Tue Dec 13 12:34:39 2016 +0200 Committer: Alexander Bezzubov <b...@apache.org> Committed: Wed Dec 14 12:59:51 2016 +0900 ---------------------------------------------------------------------- dev/create_release.sh | 28 +++++++++-- dev/publish_release.sh | 11 ++++- docs/install/docker.md | 70 ++++++++++++++++++++++++++++ scripts/docker/zeppelin-base/Dockerfile | 42 +++++++++++++++++ 4 files changed, 146 insertions(+), 5 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a8ea9e1a/dev/create_release.sh ---------------------------------------------------------------------- diff --git a/dev/create_release.sh b/dev/create_release.sh index 272713b..ece5863 100755 --- a/dev/create_release.sh +++ b/dev/create_release.sh @@ -33,14 +33,28 @@ if [[ $# -ne 2 ]]; then usage fi -if [[ -z "${GPG_PASSPHRASE}" ]]; then - echo "You need GPG_PASSPHRASE variable set" - exit 1 -fi +for var in GPG_PASSPHRASE DOCKER_USERNAME; do + if [[ -z "${!var}" ]]; then + echo "You need ${var} variable set" + exit 1 + fi +done RELEASE_VERSION="$1" GIT_TAG="$2" +function build_docker_base() { + # build base image + docker build -t ${DOCKER_USERNAME}/zeppelin-base:latest "${BASEDIR}/../scripts/docker/zeppelin-base" +} +function build_docker_image() { + # build release image + echo "FROM ${DOCKER_USERNAME}/zeppelin-base:latest + RUN mkdir /usr/local/zeppelin/ + ADD zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME} /usr/local/zeppelin/" > "Dockerfile" + docker build -t ${DOCKER_USERNAME}/zeppelin-release:"${RELEASE_VERSION}" . +} + function make_source_package() { # create source package cd ${WORKING_DIR} @@ -97,10 +111,16 @@ function make_binary_release() { mv "zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME}.tgz.md5" "${WORKING_DIR}/" mv "zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME}.tgz.sha512" "${WORKING_DIR}/" + # build docker image if binary_release_name 'all' + if [[ $1 = "all" ]]; then + build_docker_image + fi + # clean up build dir rm -rf "${WORKING_DIR}/zeppelin-${RELEASE_VERSION}-bin-${BIN_RELEASE_NAME}" } +build_docker_base git_clone make_source_package make_binary_release all "-Pspark-2.0 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pr -Pscala-2.11" http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a8ea9e1a/dev/publish_release.sh ---------------------------------------------------------------------- diff --git a/dev/publish_release.sh b/dev/publish_release.sh index fd1083a..f945f6e 100755 --- a/dev/publish_release.sh +++ b/dev/publish_release.sh @@ -30,7 +30,7 @@ if [[ $# -ne 2 ]]; then usage fi -for var in GPG_PASSPHRASE ASF_USERID ASF_PASSWORD; do +for var in GPG_PASSPHRASE ASF_USERID ASF_PASSWORD DOCKER_USERNAME DOCKER_PASSWORD DOCKER_EMAIL; do if [[ -z "${!var}" ]]; then echo "You need ${var} variable set" exit 1 @@ -67,6 +67,14 @@ function curl_error() { fi } +function publish_to_dockerhub() { + # publish images + docker login --username="${DOCKER_USERNAME}" --password="${DOCKER_PASSWORD}" --email="${DOCKER_EMAIL}" + docker push ${DOCKER_USERNAME}/zeppelin-base:latest + docker push ${DOCKER_USERNAME}/zeppelin-release:"${RELEASE_VERSION}" + +} + function publish_to_maven() { cd "${WORKING_DIR}/zeppelin" @@ -153,5 +161,6 @@ function publish_to_maven() { } git_clone +publish_to_dockerhub publish_to_maven cleanup http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a8ea9e1a/docs/install/docker.md ---------------------------------------------------------------------- diff --git a/docs/install/docker.md b/docs/install/docker.md new file mode 100644 index 0000000..c7b6f69 --- /dev/null +++ b/docs/install/docker.md @@ -0,0 +1,70 @@ +--- +layout: page +title: "Apache Zeppelin Releases Docker Images" +description: "This document contains instructions about making docker containers for Zeppelin. It mainly provides guidance into how to create, publish and run docker images for zeppelin releases." +group: install +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> +{% include JB/setup %} + +# Apache Zeppelin Releases Docker Images + +<div id="toc"></div> + +## Overview +This document contains instructions about making docker containers for Zeppelin. It mainly provides guidance into how to create, publish and run docker images for zeppelin releases. + +## Quick Start +### Installing Docker +You need to [install docker](https://docs.docker.com/engine/installation/) on your machine. + +### Creating and Publishing Zeppelin docker image +* In order to be able to create and/or publish an image, you need to set the **DockerHub** credentials `DOCKER_USERNAME, DOCKER_PASSWORD, DOCKER_EMAIL` variables as environment variables. + +* To create an image for some release use : +`create_release.sh <release-version> <git-tag>`. +* To publish the created image use : +`publish_release.sh <release-version> <git-tag>` + +### Running a Zeppelin docker image + +* To start Zeppelin, you need to pull the zeppelin release image: +``` +docker pull ${DOCKER_USERNAME}/zeppelin-release:<release-version> + +docker run --rm -it -p 7077:7077 -p 8080:8080 ${DOCKER_USERNAME}/zeppelin-release:<release-version> -c bash +``` +* Then a docker container will start with a Zeppelin release on path : +`/usr/local/zeppelin/` + +* Run zeppelin inside docker: +``` +/usr/local/zeppelin/bin/zeppelin.sh +``` + +* To Run Zeppelin in daemon mode +Mounting logs and notebooks zeppelin to folders on your host machine + +``` +docker run -p 7077:7077 -p 8080:8080 --privileged=true -v $PWD/logs:/logs -v $PWD/notebook:/notebook \ +-e ZEPPELIN_NOTEBOOK_DIR='/notebook' \ +-e ZEPPELIN_LOG_DIR='/logs' \ +-d ${DOCKER_USERNAME}/zeppelin-release:<release-version> \ +/usr/local/zeppelin/bin/zeppelin.sh +``` + + +* Zeppelin will run at `http://localhost:8080`. + http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a8ea9e1a/scripts/docker/zeppelin-base/Dockerfile ---------------------------------------------------------------------- diff --git a/scripts/docker/zeppelin-base/Dockerfile b/scripts/docker/zeppelin-base/Dockerfile new file mode 100644 index 0000000..3084d3e --- /dev/null +++ b/scripts/docker/zeppelin-base/Dockerfile @@ -0,0 +1,42 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +FROM alpine:3.4 +MAINTAINER Apache Software Foundation <d...@zeppelin.apache.org> + +ENV JAVA_HOME /usr/lib/jvm/java-1.7-openjdk +ENV PATH $PATH:$JAVA_HOME/bin + +RUN apk add --update bash curl openjdk7-jre wget ca-certificates python build-base make gcc g++ java-cacerts openssl && \ + rm /usr/lib/jvm/java-1.7-openjdk/jre/lib/security/cacerts && \ + ln -s /etc/ssl/certs/java/cacerts /usr/lib/jvm/java-1.7-openjdk/jre/lib/security/cacerts && \ + curl --silent \ + --location https://github.com/sgerrand/alpine-pkg-R/releases/download/3.3.1-r0/R-3.3.1-r0.apk --output /var/cache/apk/R-3.3.1-r0.apk && \ + apk add --update --allow-untrusted /var/cache/apk/R-3.3.1-r0.apk && \ + curl --silent \ + --location https://github.com/sgerrand/alpine-pkg-R/releases/download/3.3.1-r0/R-dev-3.3.1-r0.apk --output /var/cache/apk/R-dev-3.3.1-r0.apk && \ + apk add --update --allow-untrusted /var/cache/apk/R-dev-3.3.1-r0.apk && \ + R -e "install.packages('knitr', repos = 'http://cran.us.r-project.org')" && \ + apk del curl build-base make gcc g++ && \ + rm -rf /var/cache/apk/* + +RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.1.3/dumb-init_1.1.3_amd64 +RUN chmod +x /usr/local/bin/dumb-init + +# ports for zeppelin +EXPOSE 8080 7077 + +ENTRYPOINT ["/usr/local/bin/dumb-init", "--"]
