This is an automated email from the ASF dual-hosted git repository. xxyu pushed a commit to branch doc5.0 in repository https://gitbox.apache.org/repos/asf/kylin.git
The following commit(s) were added to refs/heads/doc5.0 by this push: new 6a9543f165 Update screenshots, refine how to contribute doc 6a9543f165 is described below commit 6a9543f1659f412a1319387f3b0a5fe0b691baa4 Author: XiaoxiangYu <x...@apache.org> AuthorDate: Tue Sep 6 17:17:41 2022 +0800 Update screenshots, refine how to contribute doc --- website/docs/development/how_to_contribute.md | 45 ++++++---- .../docs/development/how_to_debug_kylin_in_ide.md | 99 ++++++++++++--------- website/docs/development/how_to_package.md | 23 ++++- website/docs/development/how_to_write_doc.md | 39 ++++++-- .../images/IDEA_Install_Scala_plugin.png | Bin 0 -> 333636 bytes .../development/images/IDEA_Notify_Install_SDK.png | Bin 0 -> 406049 bytes .../images/IDEA_RELOAD_ALL_MAVEN_PROJECT.png | Bin 0 -> 474625 bytes .../docs/development/images/IDEA_SANDBOX_INIT.png | Bin 0 -> 490586 bytes .../docs/development/images/OPEN_KYLIN_PROJECT.png | Bin 0 -> 187986 bytes .../development/images/PROJECT_BUILD_SUCCEED.png | Bin 0 -> 1005844 bytes .../docs/development/images/RUN_KYLIN_IN_IDEA.png | Bin 0 -> 23851 bytes .../development/images/SPARDER_SUCCEED_IN_IDE.png | Bin 0 -> 304300 bytes .../docs/development/images/how-to-debug-05.png | Bin 63470 -> 0 bytes .../docs/development/images/how-to-debug-06.png | Bin 934419 -> 0 bytes website/sidebars.js | 4 + 15 files changed, 144 insertions(+), 66 deletions(-) diff --git a/website/docs/development/how_to_contribute.md b/website/docs/development/how_to_contribute.md index 83cd5142de..27f5dcdaf3 100644 --- a/website/docs/development/how_to_contribute.md +++ b/website/docs/development/how_to_contribute.md @@ -40,9 +40,9 @@ Want to know what do different role(like contributor, committer and PMC member) ::: ### Overall steps -1. Fork [Apache Kylin Repo](https://github.com/apache/kylin) to your repository. -2. Clone the fork repo to your local. -3. Create a new development branch locally. +1. [Fork Kylin Repo to your repository](#fork_repo) (https://github.com/apache/kylin). +2. [Clone the fork repo to your local](#clone_repo). +3. [Create a new development branch locally](#create_branch). 4. [Setup development environment](how_to_debug_kylin_in_ide.md) 5. [Pick or Create a JIRA](#open_issue), describe the feature/enhancement/bug. 6. [Discuss with others in mailing list](#discuss_proposal) or issue comments, make sure the proposed changes fit in with what others are doing and have planned for the project. @@ -54,21 +54,32 @@ Want to know what do different role(like contributor, committer and PMC member) 9. [Create a pull request](#open_pull_request) for you code change. 10. If you need to update doc, please check out [How to Write Document](./how_to_write_doc) for help. -### Detailed Description of the steps +### Detailed Description -#### Step 1: Fork Apache Kylin Repo +#### <span id="fork_repo">Step 1: Fork Apache Kylin Repo</span> +Visit https://github.com/apache/kylin in your browser, and click the **Fork** button.  -#### Step 2: Clone the fork repo +#### <span id="clone_repo">Step 2: Clone the fork repo</span> ```shell -git clone https://github.com/<YourUserName>/kylin.git +git clone https://github.com/<YourUserName>/kylin.git ``` -#### Step 3: Create a new development branch +#### <span id="create_branch">Step 3: Create a new development branch</span> -The **new_branch** is determined by which version of Kylin you are working on and [branches table](#branch_table). +The **base_branch** is determined by which version of Kylin you want to work on and [branches table](#branch_table). For better understanding of Git operations, please check [Contributing to a Project](https://www.git-scm.com/book/en/v2/Distributed-Git-Contributing-to-a-Project). ```shell -git checkout -b <new_branch> +## check out to base branch you want to work on +git checkout <base_branch> + +## create a development branch based on base branch +git checkout -b <development_branch> +``` + +For example, if I want to fix some issue for Kylin 5.0.0-alpha, you could execute following command: +```shell +git checkout kylin5 +git checkout -b fix_some_issue ``` #### <span id="open_issue">Step 5: Pick or create a task</span> @@ -93,6 +104,10 @@ If you want to create a new JIRA for bug or feature, remember to provide enough  +:::caution report a possible vulnerability +For a possible vulnerability, do not use JIRA, please follow the guide https://apache.org/security/committers.html#report . +::: + #### <span id="discuss_proposal">Step 6: Discuss your proposal</span> Do not forget to discuss in [mailing list](https://www.apache.org/foundation/mailinglists.html) before working on a big task. For how to discuss your idea/proposal in mailing list, please check [guide for ask good question](https://infra.apache.org/contrib-email-tips.html#usefulq) and [example for development's proposal](https://lists.apache.org/thread/gtcntp4s8k0fz1d4glospq15sycc599x) . @@ -107,21 +122,20 @@ For how to discuss your idea/proposal in mailing list, please check [guide for a * Push your code change to your personal repo -Now you can make changes to the code. Then you need to push it to **development_branch**: +Now you can make changes to the code. Then you need to push it to your remote : ```shell # After making changes to the code ... git commit -m "KYLIN-0000 COMMIT SUMMARY" -git push origin development_branch +git push origin <development_branch> ``` -* Click the ___Compare & pull request___ button. +* Click the ___Compare & pull request___ button in browser Once you push the changes to your repo, the Compare & pull request button will appear in GitHub.  -* Click ___Create pull request___ to open a new pull request. - +* Fill the message and click ___Create pull request___ to open a new pull request. :::caution Before you create pull request 1. Please add a detailed description in the `Proposed changes` of a pull request. @@ -131,7 +145,6 @@ Once you push the changes to your repo, the Compare & pull request button will a  - ----- ## Guide for Reviewer diff --git a/website/docs/development/how_to_debug_kylin_in_ide.md b/website/docs/development/how_to_debug_kylin_in_ide.md index 421a5e2228..92199b8478 100644 --- a/website/docs/development/how_to_debug_kylin_in_ide.md +++ b/website/docs/development/how_to_debug_kylin_in_ide.md @@ -16,33 +16,30 @@ last_update: author: Xiaoxiang Yu --- -# How to debug Kylin in IDEA using docker +## Debug Kylin in IDE with Hadoop ### Background - #### Why debug Kylin in IDEA using docker -This article aims to introduce a simple and useful way to develop and debug Kylin for developer, and provided similar deployment to user's real scenario. - -This guide **assumes** you have to the following: - -- Use a **laptop** with MacOS installed to do development work, and have another **remote linux server** for testing and deployment purpose. -- Docker Desktop on Mac(Windows is not verified at the moment) -- Docker Engine on Linux Server(optional) -- IntelliJ IDEA Community Edition -- kylin's source code is cloned into some directory in your laptop +This article aims to introduce a simple and useful way to develop and debug Kylin for developer, and provided similar deployment to user's real scenario. #### Deployment architecture Following is architecture of current deployment.  +This guide **assumes** you have prepared the following things: + +- [ ] A **laptop** with MacOS installed to do development work (Windows is not verified at the moment) +- [ ] A **remote linux server** for testing and deployment purpose(if you do not prepare remote linux server, you will deploy Hadoop on your laptop) +- [ ] kylin's source code is cloned into some directory in your laptop + ### Prepare IDEA and build source code -#### Software Requirement +#### Step 1: Check Software Requirement -Please visit [software_reqiurement](how_to_package#software_reqiurement). +Please visit [software_requirement](how_to_package#software_reqiurement), make sure your laptop has meet the requirement. -#### Build source code +#### Step 2: Build source code - Build back-end source code before your start debug. ```shell cd <path-to-kylin-source> @@ -50,55 +47,75 @@ mvn clean install -DskipTests ``` - Build front-end source code. -(Please use node.js v12.14.0, for how to use specific version of node.js, please check [how to package](./how_to_package.md) ) +(Please use node.js **v12.14.0**, for how to use specific version of node.js, please check [how to switch to specific node js](how_to_package#install_older_node) ) ```shell cd kystudio npm install ``` -#### Prepare IDEA configuration -- Download spark and create running IDEA configuration for debug purpose. -```shell -./dev-support/sandbox/sandbox.sh init -``` +#### Step 3: Install IntelliJ IDEA and build the source +1. Install IDEA Community edition (Ultimate edition is ok too). +2. Import the source code into IDEA. Click the **Open**, and choose the directory of **kylin source code**. +  + +3. Install scala plugin and restart + + +4. Configure SDK(JDK and Scala), make sure you use **JDK 1.8.X** and **Scala 2.12.X**. + + +5. Reload maven projects, and directory `scala` will be marked as source root(in blue color). + -- Following is the shell output. +6. Build the projects.(make sure you have executed `mvn clean package -DskipTests`, otherwise some source code is not generated by maven javacc plugin) +  - + +#### Step 4: Prepare IDEA configuration + +1. Download spark and create running IDEA configuration for debug purpose. + ```shell + ./dev-support/sandbox/sandbox.sh init + ``` + + Following is the shell output. +  ### Prepare the Hadoop Cluster #### Deploy Hadoop Cluster -- Install latest docker desktop in your laptop -- [**Optional**] Install docker engine on remote machine (https://docs.docker.com/engine/install/) -- [**Optional**] If you want to deploy hadoop cluster on remote machine, please set correct `DOCKER_HOST` +1. Install latest docker desktop in your laptop +2. [**Optional**] Install docker engine on remote machine (https://docs.docker.com/engine/install/) :::tip It is recommended to use **remote server** other than laptop to deploy Hadoop Cluster, because 7-8 containers may consume a lot of hardware resources and cause your laptop run slower than before. ::: -```shell -# see more detail at : https://docs.docker.com/compose/reference/envvars/#docker_host -export DOCKER_HOST=ssh://${USER}@${DOCKER_HOST} -``` +3. [**Optional**] If you want to deploy hadoop cluster on remote machine, please set correct `DOCKER_HOST`. -- Check available resource of your docker engine which you want to deploy Hadoop Cluster, make sure you leave 6 CPUs and 12 GB memory at least . + If you don't set `DOCKER_HOST`, you will deploy Hadoop Cluster in your laptop. (Make sure you can log in this server via SSH without input password each time by **ssh-copy-id**.) + ```shell + # see more detail at : https://docs.docker.com/compose/reference/envvars/#docker_host + export DOCKER_HOST=ssh://${USER}@${DOCKER_HOST} + ``` -Following is the setting page of Docker Desktop of MacBook. +4. Check available resource of your docker desktop in laptop (or docker engine in remote server, depends on which machine you want to deploy Hadoop Cluster), make sure you leave 6 CPUs and 12 GB memory at least . - + Following is the setting page of Docker Desktop of MacBook. -- Deploy hadoop cluster via docker compose on laptop(or on remote machine) +  -```shell -./dev-support/sandbox/sandbox.sh up -``` +5. Deploy hadoop cluster via docker compose on laptop(or on remote machine) - + ```shell + ./dev-support/sandbox/sandbox.sh up + ``` + +  #### Check status of hadoop cluster -- Wait 1-2 minutes, check health of hadoop, you can use following command to check status +- Wait 2-5 minutes, check health of hadoop, you can use following command to check status ```shell ./dev-support/sandbox/sandbox.sh ps @@ -128,11 +145,11 @@ Following output content shows all hadoop component are in health state. - Select "BootstrapServer[docker-sandbox]" on top of IDEA and click **Run** . -![click BootstrapServer[docker-sandbox]](images/how-to-debug-05.png) +![click BootstrapServer[docker-sandbox]](images/RUN_KYLIN_IN_IDEA.png) - Wait and check if Sparder is start succeed. - + - Check if SparkUI of Sparder is started. @@ -156,6 +173,8 @@ npm run devproxy  +- Create a new project, load table and create model + - Validate Cube Build and Query function  diff --git a/website/docs/development/how_to_package.md b/website/docs/development/how_to_package.md index 0272dd7621..b8d29eb65e 100644 --- a/website/docs/development/how_to_package.md +++ b/website/docs/development/how_to_package.md @@ -23,7 +23,7 @@ last_update: |---------------| ---------------------------------------------|----------------|--------------------| | Git | Fetch branch name and hash of latest commit | latest | https://git-scm.com/book/en/v2/Getting-Started-Installing-Git | | Apache Maven | Build Java and Scala source code | 3.8.2 or latest| https://maven.apache.org/download.cgi | -| Node.js | Build front end | 12.14.0 is recommended ( or 12.x ~ 14.x) | https://nodejs.org/en/download/ ([How to switch to older node.js](development/how_to_package.md#install_older_node))| +| Node.js | Build front end | 12.14.0 is recommended ( or 12.x ~ 14.x) | [How to switch to older node.js](development/how_to_package.md#install_older_node)| | JDK | Java Compiler and Development Tools | JDK 1.8.x | https://www.oracle.com/java/technologies/javase/javase8u211-later-archive-downloads.html | After installed above software, please do verify **software requirement** by following commands: @@ -90,18 +90,33 @@ For example, an unofficial package could be `apache-kylin-5.0.0-SNAPSHOT.2022081 ./build/release/release.sh -P hadoop3 ``` -### <span id="install_older_node">How to switch to older node.js</span> +### <span id="install_older_node">How to install older node.js</span> -If you install node.js which is higher than 14.X, I recommended you downgrade to lower version with some tools like https://github.com/nvm-sh/nvm. +1. Please visit https://nodejs.org/en/download/ to download and install the latest node.js . After installed, you may use follow command to verify if the latest node.js is in use: +```shell +$ node -v +v16.17.0 +``` + +2. Use some tools like https://github.com/nvm-sh/nvm to install specific version of node.js ```shell ## Switch to specific version using nvm curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash nvm install 12.14.0 -## Before packaging, please switch to specific version +## Before packaging, please switch to specific version nvm use 12.14.0 +``` +You may use follow command to verify if older node.js is in use: +```shell +$ node -v +v12.14.0 +``` + +3. Switch to latest node.js +```shell ## switch to original version nvm use system ``` diff --git a/website/docs/development/how_to_write_doc.md b/website/docs/development/how_to_write_doc.md index 21d223db4b..d2fe0e8840 100644 --- a/website/docs/development/how_to_write_doc.md +++ b/website/docs/development/how_to_write_doc.md @@ -21,7 +21,7 @@ From Kylin 5.0, Kylin community proposed to write documents using [Docusaurus](h ### Shortcut: Edit a single existent page :::info Shortcut for non-programme -1. This shortcut/way is extreme useful for **non-programmer**. If you found some minor typos or mistakes on a **single** page, you can edit document in browser in this way in 3-15 minutes without preparation. +1. This shortcut/way is extreme useful for **non-programmer**. If you found some minor typos or mistakes on a **single** page, you can edit document in browser in this way in 3-10 minutes. 2. But if you want to more complex works, like add/edit **several** pages, upload images, or change global config files, please jump to next paragraph: [**Before your work**](#Before_your_work). ::: @@ -135,16 +135,17 @@ DevelopmentSideBar: [ #### Step 4: Preview in your local machine -You can preview in your browser, please run following commands in the `website` directory, then access [doc5.0](http://127.0.0.1:3000) in your browser: +You can preview in your browser, please run following commands in the `website` directory, then access [doc of kylin in local](http://127.0.0.1:3000) in your browser: ``` npm run start ``` -:::caution Checklist +:::caution Check your doc - [ ] Whether **look and feel** meet your expectation? -- [ ] Whether the link/pictures works fine? -- [ ] Whether the most important part was highlighted? You may [check this to highlight a paragraph](#highlight_paragraph). +- [ ] Whether the **link/pictures** works fine? +- [ ] Whether the most important part was **highlighted**? You may [check this to highlight a paragraph](#highlight_paragraph). +- [ ] Whether the **title level** follow [this guide](#heading_level)? ::: #### Step 5: Create a pull request @@ -230,6 +231,32 @@ doc5.0 More details about structure which managed by Docusaurus, please refer to [Project structure rundown](https://docusaurus.io/docs/installation#project-structure-rundown). +#### <span id="heading_level">Title/Heading Level</span> + +Here is [official guide about heading](https://docusaurus.io/docs/markdown-features/toc#markdown-headings). Please use level 3 title("###") and level 4 title("####") in most of the article. + +Following is a general guide: +- Use level 2 heading(aka "##") as **top level** title. The number of top level title should not more than two. +- Use level 3 heading(aka "###") as **middle level** title. +- Use level 4 heading(aka "####") as **the lowest level** title. + + +We recommend you to check for [this article](how_to_contribute) for example. Following is toc of it. +``` +## Guide for Contributor +### Detailed Description +#### Step 1: Fork Apache Kylin Repo +#### Step 2: Clone the fork repo +#### Step 3: xxx +#### Step 4: xxx +... + +## Guide for Reviewer +### Code Review Guideline +### Patch +1 Policy +... +``` + #### Sidebar The Sidebar is managed by __sidebars.js__ , please refer to [Sidebar](https://docusaurus.io/docs/sidebar). @@ -245,7 +272,7 @@ Using relative path for site links, check this [Markdown links](https://docusaur #### How to add source code in doc -We are using [Code Blocks Doc](https://docusaurus.io/docs/markdown-features/code-blocks) to highlight code syntax, check this doc for more detail sample. +We are using [Code Block](https://docusaurus.io/docs/markdown-features/code-blocks) to highlight code syntax, check this doc for more detail sample. #### <span id="highlight_paragraph">How to highlight a sentence/paragraph</span> We recommend you to use [admonitions feature](https://docusaurus.io/docs/markdown-features/admonitions) to highlight a sentence/paragraph, following is a example: diff --git a/website/docs/development/images/IDEA_Install_Scala_plugin.png b/website/docs/development/images/IDEA_Install_Scala_plugin.png new file mode 100644 index 0000000000..4f76d76b48 Binary files /dev/null and b/website/docs/development/images/IDEA_Install_Scala_plugin.png differ diff --git a/website/docs/development/images/IDEA_Notify_Install_SDK.png b/website/docs/development/images/IDEA_Notify_Install_SDK.png new file mode 100644 index 0000000000..be07c15fdb Binary files /dev/null and b/website/docs/development/images/IDEA_Notify_Install_SDK.png differ diff --git a/website/docs/development/images/IDEA_RELOAD_ALL_MAVEN_PROJECT.png b/website/docs/development/images/IDEA_RELOAD_ALL_MAVEN_PROJECT.png new file mode 100644 index 0000000000..1ec4a8b0da Binary files /dev/null and b/website/docs/development/images/IDEA_RELOAD_ALL_MAVEN_PROJECT.png differ diff --git a/website/docs/development/images/IDEA_SANDBOX_INIT.png b/website/docs/development/images/IDEA_SANDBOX_INIT.png new file mode 100644 index 0000000000..5211499366 Binary files /dev/null and b/website/docs/development/images/IDEA_SANDBOX_INIT.png differ diff --git a/website/docs/development/images/OPEN_KYLIN_PROJECT.png b/website/docs/development/images/OPEN_KYLIN_PROJECT.png new file mode 100644 index 0000000000..fc77c2df70 Binary files /dev/null and b/website/docs/development/images/OPEN_KYLIN_PROJECT.png differ diff --git a/website/docs/development/images/PROJECT_BUILD_SUCCEED.png b/website/docs/development/images/PROJECT_BUILD_SUCCEED.png new file mode 100644 index 0000000000..6959d7f053 Binary files /dev/null and b/website/docs/development/images/PROJECT_BUILD_SUCCEED.png differ diff --git a/website/docs/development/images/RUN_KYLIN_IN_IDEA.png b/website/docs/development/images/RUN_KYLIN_IN_IDEA.png new file mode 100644 index 0000000000..b27be5fb26 Binary files /dev/null and b/website/docs/development/images/RUN_KYLIN_IN_IDEA.png differ diff --git a/website/docs/development/images/SPARDER_SUCCEED_IN_IDE.png b/website/docs/development/images/SPARDER_SUCCEED_IN_IDE.png new file mode 100644 index 0000000000..75cc3eaeb8 Binary files /dev/null and b/website/docs/development/images/SPARDER_SUCCEED_IN_IDE.png differ diff --git a/website/docs/development/images/how-to-debug-05.png b/website/docs/development/images/how-to-debug-05.png deleted file mode 100644 index 82fc6fbf22..0000000000 Binary files a/website/docs/development/images/how-to-debug-05.png and /dev/null differ diff --git a/website/docs/development/images/how-to-debug-06.png b/website/docs/development/images/how-to-debug-06.png deleted file mode 100644 index 3a0fa27433..0000000000 Binary files a/website/docs/development/images/how-to-debug-06.png and /dev/null differ diff --git a/website/sidebars.js b/website/sidebars.js index 20d7ebea2a..922dd4badc 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -850,6 +850,10 @@ const sidebars = { type: 'doc', id: 'development/how_to_release' }, + { + type: 'doc', + id: 'development/how_to_subscribe_mailing_list' + }, ], },