Repository: kylin Updated Branches: refs/heads/1.x-staging 1e0a79ef1 -> 16e5466c0
minor, fix some style and typo issues for website Project: http://git-wip-us.apache.org/repos/asf/kylin/repo Commit: http://git-wip-us.apache.org/repos/asf/kylin/commit/16e5466c Tree: http://git-wip-us.apache.org/repos/asf/kylin/tree/16e5466c Diff: http://git-wip-us.apache.org/repos/asf/kylin/diff/16e5466c Branch: refs/heads/1.x-staging Commit: 16e5466c008b35ee2ff1f511f1fc1f0b47e7725e Parents: 1e0a79e Author: lidongsjtu <don...@ebay.com> Authored: Mon Dec 28 22:23:13 2015 +0800 Committer: lidongsjtu <don...@ebay.com> Committed: Mon Dec 28 22:23:37 2015 +0800 ---------------------------------------------------------------------- website/_dev/about_temp_files.md | 20 ++++++++++---------- website/_dev/dev_env.md | 18 +++++++++--------- website/_docs/howto/howto_cleanup_storage.md | 9 ++------- website/community/poweredby.md | 2 +- 4 files changed, 22 insertions(+), 27 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kylin/blob/16e5466c/website/_dev/about_temp_files.md ---------------------------------------------------------------------- diff --git a/website/_dev/about_temp_files.md b/website/_dev/about_temp_files.md index b3fba83..b46a764 100644 --- a/website/_dev/about_temp_files.md +++ b/website/_dev/about_temp_files.md @@ -6,24 +6,24 @@ permalink: /development/about_temp_files.html --- As we reviewed the code we found that Kylin left lots of garbage files in: -1. Local file system of the CLI -2. HDFS -3. Local file system of the hadoop nodes. -A ticked was opened to track this issue: -https://issues.apache.org/jira/browse/KYLIN-926 +* Local file system of the CLI +* HDFS +* Local file system of the hadoop nodes. + +A ticket was opened to track this issue: +[https://issues.apache.org/jira/browse/KYLIN-926](https://issues.apache.org/jira/browse/KYLIN-926) For future developments, please: -1. Whenever you want to create temp files at Local, choose + +* Whenever you want to create temp files at Local, choose File.createTempFile or use the folder: BatchConstants.CFG_KYLIN_LOCAL_TEMP_DIR(/tmp/kylin), do not randomly use another folder in /tmp, it will end up a mess, and look unprofessional. - -2. Whenever you create temp files at Local, remember to delete it after +* Whenever you create temp files at Local, remember to delete it after using it. It's best to use FileUtils.forceDelete, as it also works for deleting folders. Try avoid deleteOnExit, in case Kylin exits abnormally. - -3. Whenever you want to create files in HDFS, try to create it under +* Whenever you want to create files in HDFS, try to create it under kylin.hdfs.working.dir or BatchConstants.CFG_KYLIN_HDFS_TEMP_DIR, and remember to delete it after it is no longer useful. Try avoid throwing everything into hdfs:///tmp and leave it as garbage. http://git-wip-us.apache.org/repos/asf/kylin/blob/16e5466c/website/_dev/dev_env.md ---------------------------------------------------------------------- diff --git a/website/_dev/dev_env.md b/website/_dev/dev_env.md index 5722ac4..ec0d3fa 100644 --- a/website/_dev/dev_env.md +++ b/website/_dev/dev_env.md @@ -46,13 +46,13 @@ ln -s /root/apache-maven-3.2.5/bin/mvn /usr/bin/mvn First clone the Kylin project to your local: {% highlight Groff markup %} - git clone https://github.com/apache/kylin.git +git clone https://github.com/apache/kylin.git {% endhighlight %} Install Kylin artifacts to the maven repo {% highlight Groff markup %} - mvn clean install -DskipTests +mvn clean install -DskipTests {% endhighlight %} ### Modify local configuration @@ -75,15 +75,15 @@ Run a end-to-end cube building test, these special test cases will populate some It might take a while (maybe one hour), please keep patient. {% highlight Groff markup %} - mvn test -Dtest=org.apache.kylin.job.BuildCubeWithEngineTest -DfailIfNoTests=false -Dhdp.version=<hdp-version> -P sandbox +mvn test -Dtest=org.apache.kylin.job.BuildCubeWithEngineTest -DfailIfNoTests=false -Dhdp.version=<hdp-version> -P sandbox - mvn test -Dtest=org.apache.kylin.job.BuildIIWithEngineTest -DfailIfNoTests=false -Dhdp.version=<hdp-version> -P sandbox +mvn test -Dtest=org.apache.kylin.job.BuildIIWithEngineTest -DfailIfNoTests=false -Dhdp.version=<hdp-version> -P sandbox {% endhighlight %} Run other tests, the end-to-end cube building test is exclueded {% highlight Groff markup %} - mvn test -fae -P sandbox +mvn test -fae -P sandbox {% endhighlight %} ### Launch Kylin Web Server @@ -91,15 +91,15 @@ Run other tests, the end-to-end cube building test is exclueded Copy server/src/main/webapp/WEB-INF to webapp/app/WEB-INF {% highlight Groff markup %} - cp -r server/src/main/webapp/WEB-INF webapp/app/WEB-INF +cp -r server/src/main/webapp/WEB-INF webapp/app/WEB-INF {% endhighlight %} Download JS for Kylin web GUI. `npm` is part of `Node.js`, please search about how to install it on your OS. {% highlight Groff markup %} - cd webapp - npm install -g bower - bower --allow-root install +cd webapp +npm install -g bower +bower --allow-root install {% endhighlight %} In IDE, launch `org.apache.kylin.rest.DebugTomcat` with working directory set to the /server folder. (By default Kylin server will listen on 7070 port; If you want to use another port, please specify it as a parameter when run `DebugTomcat) http://git-wip-us.apache.org/repos/asf/kylin/blob/16e5466c/website/_docs/howto/howto_cleanup_storage.md ---------------------------------------------------------------------- diff --git a/website/_docs/howto/howto_cleanup_storage.md b/website/_docs/howto/howto_cleanup_storage.md index 192bc2a..ef35d68 100644 --- a/website/_docs/howto/howto_cleanup_storage.md +++ b/website/_docs/howto/howto_cleanup_storage.md @@ -11,18 +11,13 @@ Kylin will generate intermediate files in HDFS during the cube building; Besides automated garbage collection, it might not cover all cases; You can do an offline storage cleanup periodically: Steps: - -1. Check which resources can be cleanup, this will not remove anything: +1. Check which resources can be cleanup, this will not remove anything: {% highlight Groff markup %} hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-(version).jar org.apache.kylin.job.hadoop.cube.StorageCleanupJob --delete false {% endhighlight %} - Here please replace (version) with the specific Kylin jar version in your installation; - -2. You can pickup 1 or 2 resources to check whether they're no longer be referred; Then add the "--delete true" option to start the cleanup: - +2. You can pickup 1 or 2 resources to check whether they're no longer be referred; Then add the "--delete true" option to start the cleanup: {% highlight Groff markup %} hbase org.apache.hadoop.util.RunJar ${KYLIN_HOME}/lib/kylin-job-(version).jar org.apache.kylin.job.hadoop.cube.StorageCleanupJob --delete true {% endhighlight %} - On finish, the intermediate HDFS location and HTables will be dropped; \ No newline at end of file http://git-wip-us.apache.org/repos/asf/kylin/blob/16e5466c/website/community/poweredby.md ---------------------------------------------------------------------- diff --git a/website/community/poweredby.md b/website/community/poweredby.md index bda15c8..2dcd70d 100644 --- a/website/community/poweredby.md +++ b/website/community/poweredby.md @@ -28,6 +28,6 @@ __Companies & Organizations__ * [Baidu Map](http://map.baidu.com/) (_NASDAQ: BIDU_) * Apache Kylin is used for analysis Baidu Map's huge amounts of data about users, channels and etc, it performs quite fast with its own pre-calculation and index technologies along with the cubes. * [NetEase](http://www.163.com/) (_NASDAQ: NTES_) - * Apache has been introduced as an OLAP engine for the data platform at NetEase, powering data analysis for various products & business with great performance. + * Apache Kylin has been introduced as an OLAP engine for the data platform at NetEase, powering data analysis for various products & business with great performance.
