This is an automated email from the ASF dual-hosted git repository.
xxyu pushed a commit to branch kylin4_on_cloud
in repository https://gitbox.apache.org/repos/asf/kylin.git
commit 3c7294953db2a13b929f8ede3bb019e7c0c71059
Author: Mukvin <boyboys...@163.com>
AuthorDate: Tue Feb 8 18:20:08 2022 +0800
# update README.md
---
README.md | 36 +++---
backup/properties/default/kylin.properties | 2 +-
.../properties/templates/kylin.properties.template | 2 +-
images/cpulimit.png | Bin 0 -> 51155 bytes
readme/advanced_configs.md | 42 +++----
readme/commands.md | 32 +++---
readme/cost_calculation.md | 76 +++++++++++++
readme/prerequisites.md | 121 +++++++++++++++------
readme/quick_start.md | 28 +++--
readme/quick_start_for_multiple_clusters.md | 8 +-
readme/trouble_shooting.md | 98 +++++++++++++++++
11 files changed, 340 insertions(+), 105 deletions(-)
diff --git a/README.md b/README.md
index c400a83..6847bc2 100644
--- a/README.md
+++ b/README.md
@@ -1,8 +1,8 @@
-# Welcome To Kylin 4 On Cloud Deployment Solution !
+# Welcome To Kylin 4 On Cloud Deployment Solution!
-**Apache Kylin community** released Kylin 4.0 with a new architecture, which
dedicated to building a high-performance and low-cost OLAP engine. The
architecture of Kylin 4.0 supports the separation of storage and computing,
which enables Kylin users to run Kylin 4.0 by adopting a more flexible and
elastically scalable cloud deployment method.
+**Apache Kylin community** released Kylin 4.0 with a new architecture, which
is dedicated to building a high-performance and low-cost OLAP engine. The
architecture of Kylin 4.0 supports the separation of storage and computing,
which enables Kylin users to run Kylin 4.0 by adopting a more flexible and
elastically scalable cloud deployment method.
-For the best practices of Kylin4 on the cloud, **Apache Kylin community**
contribute a **tool** to deploy kylin4 clusters on **AWS** cloud easily and
conveniently.
+For the best practices of Kylin4 on the cloud, **Apache Kylin community
contributes a **tool** to deploy kylin4 clusters on **AWS** cloud easily and
conveniently.
# Introduction About This Tool
@@ -10,17 +10,17 @@ For the best practices of Kylin4 on the cloud, **Apache
Kylin community** con
1. Deploy a Kylin4 cluster on Ec2 with Spark Standalone mode in `10` minutes.
2. Support to scale nodes (Kylin & Spark Worker) quickly and conveniently.
-3. Improve performance for query of Kylin4 in using `Local Cache + Soft
Affinity` feature (`Experimental Feature`), please check the
[details](https://kylin.apache.org/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/).
-4. Support to monitor status of cluster with `prometheus server` and
`granfana`.
+3. Improve performance for the query of Kylin4 in using `Local Cache + Soft
Affinity` feature (`Experimental Feature`), please check the
[details](https://kylin.apache.org/blog/2021/10/21/Local-Cache-and-Soft-Affinity-Scheduling/).
+4. Support to monitor the status of the cluster with the `Prometheus server`
and `Granfana`.
## Architecture
-When cluster(s) created, services and nodes will like below:
+When cluster(s) created, services and nodes will be like below:
- **Every stack module means related services will be controlled by a stack.**
-- **Read-write separated cluster will be easy created as same as image of
architecture above.**
+- The **read-write separated cluster will be easily created as same as the
image of architecture above.**
- Services are created as the number order from 1 to 5.
- Every machine node is presented by a white box.
- `Kylin Node` and `Spark Worker` Node can be easy to scale.
@@ -31,22 +31,24 @@ When cluster(s) created, services and nodes will like below:
## Quick Start
- Details about **`quick start`** of tool, please referer to [quick
start](./readme/quick_start.md).
-- Details about **`quick start for mutilple clusters`** of tool, please
referer to [quick start for mutilple
clusters](./readme/quick_start_for_multiple_clusters.md).
+- Details about **`quick start for multiple clusters`** of tool, please
referer to [quick start for multiple
clusters](./readme/quick_start_for_multiple_clusters.md).
## Notes
-1. More details about `commands` of tool, see document
[commands](./readme/commands.md).
-2. More details about `prerequisites` of tool, see document
[prerequisites](./readme/prerequisites.md).
-3. More details about `advanced configs` of tool, see document [advanced
configs](./readme/advanced_configs.md).
-4. More details about `monitor services` supported by tool, see document
[monitor](./readme/monitor.md).
-5. Current tool already open the public port for some services. You can access
the service by `public ip` of related EC2 instance.
+1. For more details about `cost` of tool, see document [cost
calculation](./readme/cost_calculation.md).
+2. For more details about `commands` of tool, see document
[commands](./readme/commands.md).
+3. For more details about the `prerequisites` of tool, see document
[prerequisites](./readme/prerequisites.md).
+4. For more details about `advanced configs` of tool, see document [advanced
configs](./readme/advanced_configs.md).
+5. For more details about `monitor services` supported by tool, see document
[monitor](./readme/monitor.md).
+6. For more details about `troubleshooting`, see document
[troubleshooting](./readme/trouble_shooting.md).
+7. The current tool has already opened the public port for some services. You
can access the service by `public IP` of related EC2 instances.
1. `SSH`: 22.
2. `Granfana`: 3000.
- 3. `Prmetheus`: 9090, 9100.
+ 3. `Prometheus`: 9090, 9100.
4. `Kylin`: 7070.
5. `Spark`: 8080, 4040.
-6. More about cloudformation syntax, please check [aws
website](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html).
-7. Current Kylin version is 4.0.0.
-8. Current Spark version is 3.1.1.
+8. More about cloudformation syntax, please check [aws
website](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html).
+9. The current Kylin version is 4.0.0.
+10. The current Spark version is 3.1.1.
diff --git a/backup/properties/default/kylin.properties
b/backup/properties/default/kylin.properties
index f800b5d..0bd043c 100644
--- a/backup/properties/default/kylin.properties
+++ b/backup/properties/default/kylin.properties
@@ -26,7 +26,7 @@ kylin.engine.spark-conf.spark.eventLog.dir=s3a:/{{
S3_BUCKET_PATH }}/working_dir
kylin.engine.spark-conf.spark.history.fs.logDirectory=s3a:/{{ S3_BUCKET_PATH
}}/working_dir/spark-history
kylin.engine.spark-conf.spark.master=spark://{{ SPARK_MASTER }}:7077
-kylin.cube.cubeplanner.enabled=true
+kylin.cube.cubeplanner.enabled=false
kylin.engine.spark-conf.spark.executor.cores=3
kylin.engine.spark-conf.spark.executor.instances=20
kylin.engine.spark-conf.spark.executor.memory=12GB
diff --git a/backup/properties/templates/kylin.properties.template
b/backup/properties/templates/kylin.properties.template
index 94dd2dd..41f4a96 100644
--- a/backup/properties/templates/kylin.properties.template
+++ b/backup/properties/templates/kylin.properties.template
@@ -24,7 +24,7 @@ kylin.engine.spark-conf.spark.eventLog.dir=s3a:/{{
S3_BUCKET_PATH }}/working_dir
kylin.engine.spark-conf.spark.history.fs.logDirectory=s3a:/{{ S3_BUCKET_PATH
}}/working_dir/spark-history
kylin.engine.spark-conf.spark.master=spark://{{ SPARK_MASTER }}:7077
-kylin.cube.cubeplanner.enabled=true
+kylin.cube.cubeplanner.enabled=false
kylin.engine.spark-conf.spark.executor.cores=3
kylin.engine.spark-conf.spark.executor.instances=20
kylin.engine.spark-conf.spark.executor.memory=12GB
diff --git a/images/cpulimit.png b/images/cpulimit.png
new file mode 100644
index 0000000..972bb49
Binary files /dev/null and b/images/cpulimit.png differ
diff --git a/readme/advanced_configs.md b/readme/advanced_configs.md
index cd60193..cd36d3b 100644
--- a/readme/advanced_configs.md
+++ b/readme/advanced_configs.md
@@ -4,56 +4,56 @@
There are `9` modules params for tools. Introductions as below:
-- EC2_VPC_PARAMS: this params of module are for creating a vpc.
+- EC2_VPC_PARAMS: These params of the module are for creating a VPC.
-- EC2_RDS_PARAMS: this params of module are for creating a RDS.
+- EC2_RDS_PARAMS: These params of the module are for creating an RDS.
-- EC2_STATIC_SERVICES_PARAMS: this params of module are for creating a
Prometheus Server and other static services.
+- EC2_STATIC_SERVICES_PARAMS: These params of the module are for creating a
Prometheus Server and other static services.
-- EC2_ZOOKEEPERS_PARAMS: this params of module are for creating a Zookeeper
Cluster.
+- EC2_ZOOKEEPERS_PARAMS: These params of the module are for creating a
Zookeeper Cluster.
-- EC2_SPARK_MASTER_PARAMS: this params of module are for creating a Spark
Master node.
+- EC2_SPARK_MASTER_PARAMS: These params of the module are for creating a Spark
Master node.
-- EC2_KYLIN4_PARAMS: this params of module are for creating a Kylin4.
+- EC2_KYLIN4_PARAMS: These params of the module are for creating a Kylin4.
-- EC2_SPARK_WORKER_PARAMS: this params of module are for creating **Spark
Workers**, default is **3** spark workers for all clusters.
+- EC2_SPARK_WORKER_PARAMS: These params of the module are for creating **Spark
Workers**, the default is **3** spark workers for all clusters.
-- EC2_KYLIN4_SCALE_PARAMS: this params of module are for scaling **Kylin4
nodes**, the range of **Kylin4 nodes** is related to `KYLIN_SCALE_UP_NODES` and
`KYLIN_SCALE_DOWN_NODES`.
+- EC2_KYLIN4_SCALE_PARAMS: these params of the module are for scaling **Kylin4
nodes**, the range of **Kylin4 nodes** is related to `KYLIN_SCALE_UP_NODES` and
`KYLIN_SCALE_DOWN_NODES`.
> Note:
>
- > 1. `KYLIN_SCALE_UP_NODES` is for the range of kylin nodes to scale up.
- > 2. `KYLIN_SCALE_DOWN_NODES` is for the range of kylin nodes to scale down.
- > 3. The range of `KYLIN_SCALE_UP_NODES` must be contain the range of
`KYLIN_SCALE_DOWN_NODES`.
- > 4. **They are effective to all clusters which is not only `default
cluster` but also other cluster which index is in `${CLUSTER_INDEXES}`.**
+ > 1. `KYLIN_SCALE_UP_NODES` is for the range of Kylin nodes to scale up.
+ > 2. `KYLIN_SCALE_DOWN_NODES` is for the range of Kylin nodes to scale down.
+ > 3. The range of `KYLIN_SCALE_UP_NODES` must contain the range of
`KYLIN_SCALE_DOWN_NODES`.
+ > 4. **They are effective to all clusters which are not only `default
cluster` but also another cluster whose index is in `${CLUSTER_INDEXES}`.**
-- EC2_SPARK_SCALE_SLAVE_PARAMS: this params of module are for scaling **Spark
workers**, the range of **Spark Workers ** is related to
`SPARK_WORKER_SCALE_UP_NODES` and `SPARK_WORKER_SCALE_DOWN_NODES`.
+- EC2_SPARK_SCALE_SLAVE_PARAMS: these params of the module are for scaling
**Spark workers**, the range of **Spark Workers is related to
`SPARK_WORKER_SCALE_UP_NODES` and `SPARK_WORKER_SCALE_DOWN_NODES`.
> Note:
>
- > 1. `SPARK_WORKER_SCALE_UP_NODES` is for the range for spark workers to
scale up. **It's effective to all clusters which is not only `default cluster`
but also other cluster which index is in `${CLUSTER_INDEXES}`.**
- > 2. `SPARK_WORKER_SCALE_DOWN_NODES` is for the range for spark workers to
scale down. **It's effective to all clusters which is not only `default
cluster` but also other cluster which index is in `${CLUSTER_INDEXES}`.**
- > 3. The range of `SPARK_WORKER_SCALE_UP_NODES` must be contain the range of
`SPARK_WORKER_SCALE_DOWN_NODES`.
- > 4. **They are effective to all clusters which is not only `default
cluster` but also other cluster which index is in `${CLUSTER_INDEXES}`.**
+ > 1. `SPARK_WORKER_SCALE_UP_NODES` is for the range for spark workers to
scale up. **It's effective to all clusters which are not only `default cluster`
but also another cluster whose index is in `${CLUSTER_INDEXES}`.**
+ > 2. `SPARK_WORKER_SCALE_DOWN_NODES` is for the range for spark workers to
scale down. **It's effective to all clusters which are not only `default
cluster` but also another cluster whose index is in `${CLUSTER_INDEXES}`.**
+ > 3. The range of `SPARK_WORKER_SCALE_UP_NODES` must contain the range of
`SPARK_WORKER_SCALE_DOWN_NODES`.
+ > 4. **They are effective to all clusters which are not only `default
cluster` but also another cluster whose index is in `${CLUSTER_INDEXES}`.**
### Customize Configs
-User also can customize the params in `kylin_configs.yaml` to create an
expected instances. Such as **the type of instance**, **the volume size of
instance** and **the volumn type of instance** and so on.
+User also can customize the params in `kylin_configs.yaml` to create an
expected instance. Such as **the type of instance**, **the volume size of the
instance** and **the volume type of instance,** and so on.
1. If you want to customize configs for instances, you must modify the
`EC2Mode` from `test` to `product` in `kylin_configs.yml`.
2. `Ec2Mode` is only in the parms of `EC2_STATIC_SERVICES_PARAMS`,
`EC2_ZOOKEEPERS_PARAMS`, `EC2_SPARK_MASTER_PARAMS`, `EC2_KYLIN4_PARAMS`,
`EC2_SPARK_WORKER_PARAMS`, `EC2_KYLIN4_SCALE_PARAMS` and
`EC2_SPARK_SCALE_SLAVE_PARAMS`.
3. So instances can be customized to effect `Monitor
Node`(`EC2_STATIC_SERVICES_PARAMS`), `Zookeeper
Nodes`(`EC2_ZOOKEEPERS_PARAMS`), `Spark Master Node` (
`EC2_SPARK_MASTER_PARAMS`), `Kylin4 Node`( `EC2_KYLIN4_PARAMS`), `Spark workers
`(`EC2_SPARK_WORKER_PARAMS`), `Kylin4 scale nodes`(`EC2_KYLIN4_SCALE_PARAMS`)
and `Spark scale workers`(`EC2_SPARK_SCALE_SLAVE_PARAMS`).
4. Now`Ec2Mode` **only effect** the related params are
`Ec2InstanceTypeFor*`,`Ec2VolumeSizeFor*` and `Ec2VolumnTypeFor`* in the
params modules.
-5. If you don't change `ENABLE_LOCAL_CACHE_SOFT_AFFINITY` from `"false"` to
`"true"` then cluster will created normally without `Local Cache + Soft
Affinity` feature!
+5. If you don't change `ENABLE_LOCAL_CACHE_SOFT_AFFINITY` from `"false"` to
`"true"` then the cluster will be created normally without the `Local Cache +
Soft Affinity` feature!
#### Example
As an example in `EC2_STATIC_SERVICES_PARAMS`:
-- change `Ec2Mode` from `test`to `product`.
+- change `Ec2Mode` from `test` to `product`.
- change `Ec2InstanceTypeForStaticServices` from `m5.2xlarge` to `m5.4xlarge`.
- change `Ec2VolumeSizeForStaticServicesNode` from `'20'` to `'50'`.
- change `Ec2VolumnTypeForStaticServicesNode` from `gp2` to `standard`.
-- Then create the node of static service node will be a `m5.4xlarge` and it
attach a volume which size is `50` and type is `standard`.
+- Then create the node of static service node will be an `m5.4xlarge` and it
attaches a volume which size is `50` and type is `standard`.
diff --git a/readme/commands.md b/readme/commands.md
index 99e6306..99821ec 100644
--- a/readme/commands.md
+++ b/readme/commands.md
@@ -10,15 +10,15 @@ python deploy.py --type [deploy|destroy|list|scale]
--scale-type [up|down] --nod
- destroy: destroy created cluster(s).
-- list: list alive nodes which are with stack name, instance id, private ip
and public ip.
+- list: list alive nodes which are with stack name, instance id, private IP,
and public IP.
- scale: Must be used with `--scale-type` and `--node-type`.
> Note:
>
- > 1. Current support to scale up/down `kylin` or `spark_worker` for specific
cluster.
- > 2. Before scale up/down `kylin` or `spark_worker` nodes, Cluster services
must be ready.
- > 3. If you want to scale a `kylin` or `spark_worker` node to a specify
cluster, please add the `--cluster ${cluster num}` to specify the expected node
add to the cluster `${cluster num}`.
+ > 1. Current support to scale up/down `kylin` or `spark_worker` for a
specific cluster.
+ > 2. Before scaling up/down `kylin` or `spark_worker` nodes, Cluster
services must be ready.
+ > 3. If you want to scale a `kylin` or `spark_worker` node to a specified
cluster, please add the `--cluster ${cluster num}` to specify the expected node
add to the cluster `${cluster num}`.
### Command for deploy
@@ -28,7 +28,7 @@ python deploy.py --type [deploy|destroy|list|scale]
--scale-type [up|down] --nod
$ python deploy.py --type deploy [--cluster default]
```
-- Deploy a cluster with specific cluster index.
+- Deploy a cluster with a specific cluster index. <a name="deploycluster"></a>
```shell
$ python deploy.py --type deploy --cluster ${cluster num}
@@ -36,7 +36,7 @@ $ python deploy.py --type deploy --cluster ${cluster num}
> Note: the `${cluster num}` must be in the range of `CLUSTER_INDEXES`.
-- Deploy all cluster which contain default cluster and all cluster which index
in the range of `CLUSTER_INDEXES`.
+- Deploy all clusters which contain the default cluster and all clusters whose
index is in the range of `CLUSTER_INDEXES`.
```shell
$ python deploy.py --type deploy --cluster all
@@ -46,7 +46,7 @@ $ python deploy.py --type deploy --cluster all
> Note:
>
-> Destroy all cluster will not delete vpc, rds and monitor node.
So if user don't want to hold the env, please set the `ALWAYS_DESTROY_ALL` to
be `'true'`.
+> Destroy all clusters will not delete vpc, rds, and monitor
node. So if user doesn't want to hold the env, please set the
`ALWAYS_DESTROY_ALL` to be `'true'`.
- Destroy default cluster
@@ -54,7 +54,7 @@ $ python deploy.py --type deploy --cluster all
$ python deploy.py --type destroy [--cluster default]
```
-- Destroy a cluster with specific cluster index.
+- Destroy a cluster with a specific cluster index.
```shell
$ python deploy.py --type destroy --cluster ${cluster num}
@@ -62,7 +62,7 @@ $ python deploy.py --type destroy --cluster ${cluster num}
> Note: the `${cluster num}` must be in the range of `CLUSTER_INDEXES`.
-- Destroy all cluster which contain default cluster and all cluster which
index in the range of `CLUSTER_INDEXES`.
+- Destroy all clusters which contain the default cluster and all clusters
whose index is in the range of `CLUSTER_INDEXES`.
```shell
$ python deploy.py --type destroy --cluster all
@@ -70,7 +70,7 @@ $ python deploy.py --type destroy --cluster all
### Command for list
-- List nodes which are with **stack name**, **instance id**, **private ip**
and **public ip** in **available stacks** .
+- List nodes that are with **stack name**, **instance id**, **private IP,**
and **public IP** in **available stacks**.
```shell
$ python deploy.py --type list
@@ -81,14 +81,14 @@ $ python deploy.py --type list
> Note:
>
> 1. Scale command must be used with `--scale-type` and `--node-type`.
-> 2. If scale command not specify a cluster num, then the scaled node(kylin or
spark worker) will be add to `default`cluster.
+> 2. If the scale command does not specify a cluster num, then the scaled
node(Kylin or spark worker) will be added to the `default` cluster.
> 3. Scale command **not support** to **scale** node (kylin or spark worker)
> to **all clusters** at **one time**. It means that `python ./deploy.py
> --type scale --scale-type up[|down] --node-type kylin[|spark_worker]
> --cluster all` is invalid commad.
-> 4. Scale params which are `KYLIN_SCALE_UP_NODES`, `KYLIN_SCALE_DOWN_NODES`,
`SPARK_WORKER_SCALE_UP_NODES` and `SPARK_WORKER_SCALE_DOWN_NODES` effect on all
cluster. So if user want to scale node for a specify cluster, then modify the
scale params before **every run time.**
-> 5. **(Important!!!)** Current cluster is created with default `3` spark
workers and `1` kylin node. The `3` spark workers can not be scaled down. The
`1` kylin node also can not be scaled down.
-> 6. **(Important!!!)** Cluster can only scale up or down the range of nodes
which is in `KYLIN_SCALE_UP_NODES`, `KYLIN_SCALE_DOWN_NODES`,
`SPARK_WORKER_SCALE_UP_NODES` and `SPARK_WORKER_SCALE_DOWN_NODES` . Not the
default `3` spark workers and `1` kylin node in the cluster.
-> 7. **(Important!!!)** If user don't want to create a cluster with `3`
default spark workers, then user can remove the useless node module in the
`Ec2InstanceOfSlave0*` of
`cloudformation_templates/ec2-cluster-spark-slave.yaml`. User need to know
about the syntax of cloudformation as also.
+> 4. Scale params which are `KYLIN_SCALE_UP_NODES`, `KYLIN_SCALE_DOWN_NODES`,
`SPARK_WORKER_SCALE_UP_NODES` and `SPARK_WORKER_SCALE_DOWN_NODES` effect on all
cluster. So if user wants to scale a node for a specific cluster, then modify
the scale params before **every run time.**
+> 5. **(Important!!!)** The current cluster is created with default `3` spark
workers and `1` Kylin node. The `3` spark workers can not be scaled down. The
`1` Kylin node also can not be scaled down.
+> 6. **(Important!!!)** The current cluster can only scale up or down the
range of nodes which is in `KYLIN_SCALE_UP_NODES`, `KYLIN_SCALE_DOWN_NODES`,
`SPARK_WORKER_SCALE_UP_NODES,` and `SPARK_WORKER_SCALE_DOWN_NODES`. Not the
default `3` spark workers and `1` kylin node in the cluster.
+> 7. **(Important!!!)** If user doesn't want to create a cluster with `3`
default spark workers, then user can remove the useless node module in the
`Ec2InstanceOfSlave0*` of
`cloudformation_templates/ec2-cluster-spark-slave.yaml`. User needs to know
about the syntax of `cloudformation` as also.
-- Scale up/down kylin/spark workers in default cluster
+- Scale up/down Kylin/spark workers in default cluster
```shell
python deploy.py --type scale --scale-type up[|down] --node-type
kylin[|spark_worker] [--cluster default]
diff --git a/readme/cost_calculation.md b/readme/cost_calculation.md
new file mode 100644
index 0000000..91691b5
--- /dev/null
+++ b/readme/cost_calculation.md
@@ -0,0 +1,76 @@
+## Cost Calculation
+
+### Cost about a cluster
+
+Full cost about a cluster by tool contains as follows:
+
+1. The type of EC2 instances.
+2. The volume of EBS for EC2 instances.
+3. The volume of S3 is in use.
+4. The data transfer for EC2 instances.
+5. The RDS.
+
+#### The type of EC2 instances
+
+A total of EC2 instances contains 3 `m5.large` (for zookeeper), 1
`m5.large`(for monitor services), 3 `m5.xlarge`(for spark slaves), 1
`m5.xlarge`(for spark master) and 1 `m5.xlarge` (for kylin).
+
+For example, if the current region is `ap-northeast-2` which is `Asia Pacific
(Seoul)`, then user may cost **(3 + 1) * 0.118 + (3 + 1 + 1) * 0.236 = 1.652
USD/hourly**.
+
+> Note:
+>
+> 1. The type of EC2 instances for related services can be checked in the
`cloudformation_templates/*.yaml`.
+> 2. Other scaled up/down nodes can also calculate in manually if users scaled.
+> 3. For more details about the cost of EC2 instances, please check [On-Demand
Plans for Amazon EC2](https://aws.amazon.com/ec2/pricing/on-demand/).
+
+
+
+#### The volume of EBS for EC2 instances
+
+The default volume type for tool is `gp2`.
+
+A total volume size of a cluster contains 10 * 3 (for zookeeper), 20 * 1 (for
monitor services), 30 * 3 (for spark slaves), 30 * 1 (for spark master) and 30
* 1(for kylin).
+
+For example, if the current region is `ap-northeast-2` which is `Asia Pacific
(Seoul)`, then user may cost **[(10 * 3) + 20 * 1 + 30 * 3 + 30 * 1 + 30 * 1] *
0.114 = 22.8 USD/Per month which equals to 22.8 / (30d * 24h) = 0.0317
USD/hourly.**
+
+
+
+> Note:
+>
+> 1. The volume of EBS for EC2 instances for related services can be checked
in the `cloudformation_templates/*.yaml`.
+> 2. Other scaled up/down nodes can also calculate manually if users scaled.
+> 3. For more details about the cost volume of EBS, please check [Amazon EBS
pricing](https://aws.amazon.com/ebs/pricing/).
+
+
+
+#### The volume of S3 is in use
+
+the S3 volume pricing is a tiny fraction of the overall cost. **Data on S3 can
be stored for the long term.**
+
+For example, if the current region is `ap-northeast-2` which is `Asia Pacific
(Seoul)`, and a built data by Kylin such as `kylin_sales` and other needed
files in the cluster will take about `2` GB size on the S3. So user may cost
**2 * 0.025 GB = 0.05 USD.** **Note that there is no time limit.**
+
+> Note: For more details about the cost volume of S3, please check [Amazon S3
pricing](https://aws.amazon.com/s3/pricing/).
+
+#### The data transfer for EC2 instances
+
+**The tool will upload needed files for clusters to S3, and it's free.**
+
+**And Data transferred between Amazon EC2, Amazon RDS, Amazon Redshift, Amazon
ElastiCache instances, and Elastic Network Interfaces in the same Availability
Zone is free.**
+
+It will take a minimal fee for a user to login into EC2 instances by public
IP. And it's hard to calculate. For more details please check [Data Transfer
within the same AWS Region](https://aws.amazon.com/ec2/pricing/on-demand/#Data
Transfer within the same AWS Region).
+
+#### The RDS
+
+The type of current RDS is `db.t3.micro`.
+
+The volume size of current RDS is `10` GB.
+
+For example, if the current region is `ap-northeast-2` which is `Asia Pacific
(Seoul)`, and a user may cost **0.026 USD** for the type of RDS and 0.131 per
GB-month which equals to 0.131/(30d * 24h) = **0.00018 USD/hourly**.
+
+**So it will total cost 0.026 + 0.00018 = 0.02618 USD/hourly.**
+
+
+
+### Total Cost
+
+**As discuss above, the total cost is about 1.652 + 0.0317 + 0.05 + 0.02618 =
1.75988 USD/hourly.**
+
diff --git a/readme/prerequisites.md b/readme/prerequisites.md
index 541737d..97ccdf9 100644
--- a/readme/prerequisites.md
+++ b/readme/prerequisites.md
@@ -1,6 +1,6 @@
## Prerequisites
-### Download source code & checkout to branch of `kylin4_on_cloud`
+### Download source code & checkout to a branch of `kylin4_on_cloud`
commands:
@@ -14,19 +14,32 @@ git clone https://github.com/apache/kylin.git && cd kylin
&& git checkout kylin4
> Note:
>
-> `IAM` role must have the access which contains `AmazonEC2RoleforSSM` ,
`AmazonSSMFullAccess` and `AmazonSSMManagedInstanceCore`.
+> `IAM` role must have the access which contains `AmazonEC2RoleforSSM`,
`AmazonSSMFullAccess,` and `AmazonSSMManagedInstanceCore`.
>
-> This `IAM` Role will be used to initialize every ec2 instances which
are for creating an kylin4 cluster on aws. And it will configure in `Initilize
Env of Local Machine` part.
+> This `IAM` Role will be used to initialize every ec2 instances which is
for creating a kylin4 cluster on AWS. And it will configure in `Initialize Env
of Local Machine` part.
+
+
+
+> Q: Why should need an `IAM` role?
+>
+> A:
+>
+> 1. At first, the `IAM` role is an IAM identity that you can create in
your account that has specific permissions.
+> 1. An IAM role does not have standard long-term credentials such as a
password or access keys associated with it. Instead, when you assume a role,
it provides you with temporary security credentials for your role session.
+> 1. The access will not cost money. The cost will depend on what you
using on S3 and EC2 instances.
+> 1. For more details about `IAM`, please visit the [official
website](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) by aws.
+
+
#### II. Create a `User`
> Note:
>
-> The `User` who is created in aws will be used to execute some
operations on EC2 instance. So the `User` must has some authorities as below.
+> The `User` who is created in aws will be used to execute some
operations on an EC2 instance. So the `User` must have some authority as below.
| Services | Access level
| Resources | Request condition |
| ------------------- | ------------------------------------------------------
| ------------- | ----------------- |
-| **Cloudformation** | **Limited**: List, Read,Write
| All Resources | None |
+| **Cloudformation** | **Limited**: List, Read, Write
| All Resources | None |
| **EC2** | **Limited**: Read
| All Resources | None |
| **IAM** | **Limited**: List, Read, Write, Permissions management
| All Resources | None |
| **RDS** | **Limited**: List, Write, Tagging
| All Resources | None |
@@ -35,6 +48,19 @@ git clone https://github.com/apache/kylin.git && cd kylin &&
git checkout kylin4
| **STS** | **Limited**: Write
| All Resources | None |
| **Systems Manager** | **Limited**: Write
| All Resources | None |
+
+
+> Q: What's the relationship between the `User` and `IAM` role?
+>
+> A:
+>
+> 1. With `IAM` roles, you can establish trust relationships between your
***trusting*** account and other AWS ***trusted*** accounts.
+> 1. A `User` is an account.
+> 1. The establishment of the relationship will cost nothing.
+> 1. The cost majorly depends on what you are using on S3 and EC2
instances.
+
+
+
#### III. Login the created `User` to create key pair and Access Key<a
name="keypair"></a>
> Note:
@@ -45,11 +71,11 @@ git clone https://github.com/apache/kylin.git && cd kylin
&& git checkout kylin4
-##### 2. Genrate a new `Access Key` which in the `My Security Credentials`
+##### 2. Generate a new `Access Key` in the `My Security Credentials`
> Note:
>
-> Please download the generated the csv file of `Access Key` immediately.
Get the `Access Key` and `Secret Key` to initilize local mac to access aws.
+> Please download the generated CSV file of `Access Key` immediately. Get
the `Access Key` and `Secret Key` to initialize local mac to access aws.
@@ -57,17 +83,29 @@ git clone https://github.com/apache/kylin.git && cd kylin
&& git checkout kylin4
+##### (!!!Important)IV. Make sure current account has 32 vCpu at least
+
+The current tool will deploy a cluster by EC2 instances, and `2 cores * 3(for
zookeeper) + 2 cores(for monitor services) + 4 cores * 3(for spark slaves) + 4
cores(for spark master) + 4 cores (for kylin)=28 ` cpus to create instances in
default.
+
+> Note: **32 vCPUS is the limit of an default aws account.**
+
+**User can check this quota limit in the limit of EC2 dashborad.**
+
+
+
+For more details about user current limit, please check [View your current
limits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html).
+
### Initialize a S3 direcotry<a name="S3"></a>
#### I. Create a directory on S3
> Note:
>
-> This directory will store tars, scripts and jars below. And it will
also be the root path of working dir for kylin4.
+> This directory will store tars, scripts, and jars below. And it will
also be the root path of working dir for kylin4.
-Example: make a directory named `kylin4-aws-test` . You can also create a
directory named what you like.
+Example: make a directory named `kylin4-aws-test`. You can also create a
directory named what you like.
@@ -81,11 +119,10 @@ Example: make a directory named `kylin4-aws-test` . You
can also create a direct
-> Create the directory named `tar` **in the path which was created by
yourself**.
+> Create the directory named `tar` **in the path which was created by
yourself**. For example, the full path would be `s3://.../kylin4-aws-test/tar`.
>
-> As example, the full path would be `s3://.../kylin4-aws-test/tar`.
-1. Download Kylin4 package by [official
website](https://kylin.apache.org/download/).
+1. Download the Kylin4 package on the [official
website](https://kylin.apache.org/download/).
2. Download Hadoop, [version
3.2.0](https://archive.apache.org/dist/hadoop/common/hadoop-3.2.0/hadoop-3.2.0.tar.gz).
3. Download Spark with hadoop3.2, [version
3.1.1](https://archive.apache.org/dist/spark/spark-3.1.1/spark-3.1.1-bin-hadoop3.2.tgz).
4. Download Hive, [version
2.3.9](https://archive.apache.org/dist/hive/hive-2.3.9/apache-hive-2.3.9-bin.tar.gz).
@@ -93,12 +130,12 @@ Example: make a directory named `kylin4-aws-test` . You
can also create a direct
6. Download JDK, [version
1.8_301](https://www.oracle.com/java/technologies/javase/javase8u211-later-archive-downloads.html).
7. Download Node Exporter, [version
1.3.1.](https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz)
8. Download Prometheus Server, [version
2.31.1](https://github.com/prometheus/prometheus/releases/download/v2.31.1/prometheus-2.31.1.linux-amd64.tar.gz).
-9. Download Kylin4 package with local cache + soft affinity feature by [public
website.](https://s3.cn-north-1.amazonaws.com.cn/public.kyligence.io/kylin/tar/apache-kylin-4.0.0-bin-spark3-soft.tar.gz)
+9. Download the Kylin4 package with local cache + soft affinity feature by the
[public
website.](https://s3.cn-north-1.amazonaws.com.cn/public.kyligence.io/kylin/tar/apache-kylin-4.0.0-bin-spark3-soft.tar.gz)
> Note:
-> If you want to use Kylin4 with local cache + soft affinity feature,
please download the `experimental` package above.
+> If you want to use Kylin4 with a local cache + soft affinity feature,
please download the `experimental` package above.
@@ -112,9 +149,8 @@ Example: make a directory named `kylin4-aws-test` . You can
also create a direct
-> Create the directory named `jars` **in the path which was created by
yourself**.
+> Create the directory named `jars` **in the path which was created by
yourself**. For example, the full path would be
`s3://.../kylin4-aws-test/jars`.
>
-> As example, the full path would be `s3://.../kylin4-aws-test/jars`.
Kylin4 needed extra jars
@@ -135,9 +171,8 @@ Kylin4 needed extra jars
-> Create the directory named `scripts` **in the path which was created by
yourself**.
+> Create the directory named `scripts` **in the path which was created by
yourself**. For example, the full path would be
`s3://.../kylin4-aws-test/scripts`.
>
-> As example, the full path would be `s3://.../kylin4-aws-test/scripts`.
Scripts:
@@ -149,11 +184,11 @@ Scripts:
-### Initilize Env Of Local Machine
+### Initialize Env Of Local Machine
#### I. Initilize an aws account on local mac to access AWS<a
name="localaws"></a>
-> Use `Access Key` and `Secret Key` above to Initialize an aws account on
local mac.
+> Use `Access Key` and `Secret Key` above to Initialize an AWS account on a
local mac.
```shell
$ aws configure
@@ -166,8 +201,7 @@ Default output format : json
> Note:
>
> 1. If this command got the response `-bash: aws: command not found`, please
> check in [Getting started with the AWS
> CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html).
-> 2. Region name can be in [Available
Regions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions).
-> 3. Default output format is `json`.
+> 2. Region name can be in [Available
Regions](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions).
The default output format is `json`.
#### II. Init a Python virtual env
@@ -188,29 +222,44 @@ $ bin/init.sh
**Required parameters**:
- `AWS_REGION`: Current region for EC2 instances.
-- `IAMRole`: IAM role which has the access to aws authority. This parameter
will be set to created **name** of IAM role.
-- `S3_URI`: the prefix path of storing `jars/scripts/tar`. As example, this
parameter will be set to `s3://.../kylin4-aws-test`.
-- `KeyName`: Security key name is a set of security credentials that you use
to prove your identity when connecting to an instance. This parameter will be
set to created **name** of `key pair` .
-- `CIDR_IP`: An inbound rule permits instances to receive traffic from the
specified IPv4 or IPv6 CIDR address range, or from the instances associated
with the specified security group.
-- `DB_IDENTIFIER`: this param should be only one in the `RDS -> Databases` .
And it will be the name of created RDS database.
+- `IAMRole`: IAM role which has the access to aws authority. This parameter
will be set to the created **name** of the IAM role.
+- `S3_URI`: the prefix path of storing `jars/scripts/tar`. For example, this
parameter will be set to `s3://.../kylin4-aws-test`.
+- `KeyName`: Security key name is a set of security credentials that you use
to prove your identity when connecting to an instance. This parameter will be
set to the created **name** of key pair`.
+- `CIDR_IP`: An inbound rule permits instances to receive traffic from the
specified IPv4 or IPv6 CIDR address range, or the instances associated with the
specified security group.
+- `DB_IDENTIFIER`: this param should be only one in the `RDS -> Databases`.
And it will be the name of created RDS database.
- `DB_PORT`: this param will be the port of created RDS database, default is
`3306`.
-- `DB_USER`: this param will be a login ID for the master user of your DB
instance, default is `root`.
-- `DB_PASSWORD`: this param will be the password of `DB_USER` to access DB
instance. default is `123456test`, it's strongly suggest you to change it.
+- `DB_USER`: this param will be a login ID for the master user of your DB
instance, the default is `root`.
+- `DB_PASSWORD`: this param will be the password of `DB_USER` to access the DB
instance. default is `123456test`, it's strongly suggested you change it.
#### IV. Configure the `kylin.properties` in `backup/properties`
directories.<a name="cluster"></a>
-1. The `kylin.properties` is for starting kylin instance in the cluster.
-2. Default cluster will check the `kylin.properties` in the
`backup/properties/default`, and other specific cluster will check the related
num directory such as `1`, `2` and `3`.
-3. User need to create new dir for the cluster num in `backup/properties`, and
name it to the `${cluster num}`, such as `1`, `2` ,`3` and so on. The range of
cluster num must be in `CLUSTER_INDEXES` which is configured in the
`kylin_configs.yml`.
-4. Follow the `2.` step, copy the `kylin.properties.template` which is in
`backup/properties/templates` to the related `${cluster num} ` directories， and
rename the template to `kylin.properties`.
-5. The range of cluster nums must match the the config `CLUSTER_INDEXES`, such
as `CLUSTER_INDEXES: (1, 3)` then the directories must be `1`, `2`,`3` in the
`backup/properties`.
+> About the indexes of clusters:<a name="indexofcluster"></a>
+>
+> 1. **Tool will deploy a cluster which will mark to `default` if a user does
not specify a cluster to deploy.**
+> 2. The range of cluster indexes must be in `CLUSTER_INDEXES` which is
configured in the `kylin_configs.yml`.
+> 3. **`CLUSTER_INDEXES` is for users to create multiple clusters as
READ-WRITE separation mode**, such as a user may want to create 2 query nodes
of Kylin and 2 job nodes of Kylin.
+> 1. User can modify the `CLUSTER_INDEXES` to be `(1, 3)`, then as
following steps of this module to modify the `kylin.properties` file for
clusters.
+> 2. User can mark the Kylin node of the `default` cluster to be `query`
mode and the Kylin node of the cluster whose index is `1` to be `query` mode.
User can mark the Kylin node of the cluster which index is `2` to be `job` mode
and the Kylin node of the cluster which index is `3` to job mode.
+> 3. User can also modify the `CLUSTER_INDEXES` to be `(1, 4)`, and mark
the Kylin node of clusters whose index is `1` and `2` to be `query` mode and
the Kylin node of clusters whose index is `3` and `4` to be `job` mode. Just
don't use `default` to mark a cluster and execute exactly deploy cluster
commands. For details about commands, please check to [deploy multiple
clusters](./Commands.md#deploycluster).
+> 4. The list of mark name for clusters will be [`default`, `{cluster num}`
...] and `{cluster num}` is in the range of `CLUSTER_INDEXES`.
+> 5. For example, if `CLUSTER_INDEXES` is (1, 3) means that tool can deploy a
cluster and it can be marked as `default` or `1` or `2` or `3`. And tool can
execute to deploy total 1(`default`) + 3(clusters which mark name can be `1`,
`2`, `3`) = 4 clusters.
+> 6. Every cluster contains 3 `Zookeepers Node`, 1 `Kylin Node`, 1 `Spark
Master Node,` and 3 `Spark Slaves Node` after deployed and it can scale needed
nodes of Kylin and Spark workers.
+> 7. **The difference between clusters only can be the index or customized
configs of EC2 instances or customized properties of Kylin (and spark and
zookeeper).**
+
+
+
+1. The `kylin.properties` is for starting kylin instance in the cluster.
+2. The default cluster will check the `kylin.properties` in the
`backup/properties/default`, and other specific clusters will check the related
num directory such as `1`, `2,` and `3`.
+3. User needs to create a new dir for the cluster num in `backup/properties`,
and name it to the `${cluster num}`, such as `1`, `2` ,`3` and so on.
+4. Following the `2.` step, copy the `kylin.properties.template` which is in
`backup/properties/templates` to the related `${cluster num} ` directories， and
rename the template to `kylin.properties`.
+5. The range of cluster nums must match the config `CLUSTER_INDEXES`, such as
`CLUSTER_INDEXES: (1, 3)` then the directories must be `1`, `2`,`3` in the
`backup/properties`.
-6. Cusomize the config which you want to change except the config which
contains `{{ }}` mark.
+6. Customize the config which you want to change except the config which
contains `{{ }}` mark.
7. If the related `kylin.properties` was filled with cluster information, a
file named `.rendered` will create in the directory.
-8. **(Important !!!)** If you want to re-fill the `kylin.properties` for a
`kylin` node in specify cluster, you need to remove the `.rendered` file and
re-copy the `kylin.propertie.template` . Redo steps from `3` to `6`.
\ No newline at end of file
+8. **(Important !!!)** If you want to re-fill the `kylin.properties` for a
`kylin` node in a specific cluster, you need to remove the `.rendered` file
and re-copy the `kylin.propertie.template`. Redo steps from `3` to `6`.
\ No newline at end of file
diff --git a/readme/quick_start.md b/readme/quick_start.md
index 7aae18c..76df90e 100644
--- a/readme/quick_start.md
+++ b/readme/quick_start.md
@@ -2,9 +2,14 @@
+- **Services are created as the number order from 1 to 4.**
+- **A cluster will be easily created as same as the image of architecture
above.**
+
+
+
## Quick Start
-1. Initialize aws account credential on local machine, please check
[details](./prerequisites.md#localaws).
+1. Initialize aws account credential on the local machine, please check
[details](./prerequisites.md#localaws).
2. Download the source code:
@@ -16,11 +21,11 @@
1. Set the `AWS_REGION`.
- 2. Set the `IAMRole`,please check [details](./prerequisites.md#IAM).
+ 2. Set the `IAMRole`, please check the [details](./prerequisites.md#IAM).
3. Set the `S3_URI`, please check [details](./prerequisites.md#S3).
- 4. Set the `KeyName`,please check [details](./prerequisites.md#keypair).
+ 4. Set the `KeyName`, please check [details](./prerequisites.md#keypair).
5. Set the `CIDR_IP`, make sure that the `CIDR_IP` match the pattern
`xxx.xxx.xxx.xxx/16[|24|32]`.
@@ -28,7 +33,7 @@
>
> 1. this `CIDR_IP` is the specified IPv4 or IPv6 CIDR address range
which an inbound rule can permit instances to receive traffic from.
>
- > 2. In one word, it will let your mac which ip is in the `CIDR_IP` to
access instances.
+ > 2. In one word, it will let your mac which IP is in the `CIDR_IP` to
access instances.
4. Init local env.
@@ -38,21 +43,26 @@ $ bin/init.sh
> Note: Following the information into a python virtual env and get the help
> messages.
-5. Execute commands to deploy a `default` cluster, please check
[details](./prerequisites.md#cluster).
+5. Execute commands to deploy a `default` cluster.
```shell
$ python deploy.py --type deploy
```
-After `default` cluster is ready, you will see the message `Kylin Cluster
already start successfully.` in the console.
+After the `default` cluster is ready, you will see the message `Kylin Cluster
already start successfully.` in the console.
+
+> Note:
+>
+> 1. For details about the properties of kylin4 in a cluster, please check
[configure kylin.properties](./prerequisites.md#cluster).
+> 2. For details about the index of the cluster, please check [Indexes of
clusters](./prerequisites.md#indexofcluster).
-6. Execute commands to list nodes of cluster.
+6. Execute commands to list nodes of the cluster.
```shell
$ python deploy.py --type list
```
-Then you can check the `public ip` of Kylin Node.
+Then you can check the `public IP` of Kylin Node.
You can access `Kylin` web by `http://{kylin public ip}:7070/kylin`.
@@ -66,5 +76,5 @@ $ python deploy.py --type destroy
-> Note: If you want to check about quick start for multiple clusters, please
referer to [quick start for mutilple
clusters](./quick_start_for_multiple_clusters.md).
+> Note: If you want to check about a quick start for multiple clusters, please
referer to a [quick start for multiple
clusters](./quick_start_for_multiple_clusters.md).
diff --git a/readme/quick_start_for_multiple_clusters.md
b/readme/quick_start_for_multiple_clusters.md
index 86ef780..1b5d50b 100644
--- a/readme/quick_start_for_multiple_clusters.md
+++ b/readme/quick_start_for_multiple_clusters.md
@@ -1,14 +1,14 @@
## Quick Start For Multiple Clusters
-> Pre-steps is same as Quick Start steps which is from 1 to 5.
+> Pre-steps are the same as Quick Start steps which are from 1 to 5.
-1. Modify the config `CLUSTER_INDEXES` for multiple cluster.
+1. Modify the config `CLUSTER_INDEXES` for multiple clusters.
> Note:
>
> 1. `CLUSTER_INDEXES` means that cluster index is in the range of
`CLUSTER_INDEXES`.
- > 2. If user create multiple clusters, `default` cluster always be created.
If `CLUSTER_INDEXES` is (1, 3), there will be 4 cluster which contains the
cluster 1, 2, 3 and `default` will be created if user execute the commands.
- > 3. Configs for multiple clusters always are same as the `default` cluster
to read from `kylin_configs.yaml`
+ > 2. If a user creates multiple clusters, the `default` cluster always is
created. If `CLUSTER_INDEXES` is (1, 3), there will be 4 cluster that contains
cluster 1, 2, 3, and `default` will be created if a user executes the commands.
+ > 3. Configs for multiple clusters always are the same as the `default`
cluster to read from `kylin_configs.yaml`
2. Copy `kylin.properties.template` for expecting clusters to deploy, please
check the [details](./prerequisites.md#cluster).
diff --git a/readme/trouble_shooting.md b/readme/trouble_shooting.md
new file mode 100644
index 0000000..4f2f053
--- /dev/null
+++ b/readme/trouble_shooting.md
@@ -0,0 +1,98 @@
+## Trouble Shooting
+
+### Cloudformation stack is normal, but services can't access normally.
+
+Q: Sometimes users will found that the stack is normal in the cloudformation
dashboard, but services aren't started normally. Such as the port of related
service can not access by IP.
+
+A:
+
+1. Users can login into the instance of related services by the `credential
file` and the public `IP` which is in the stack `Outputs`. Access command will
be like `ssh -i xxx.pem ec2-user@{public ip}` on the terminal.
+2. Change to the `root` user by executing `sudo su` on the terminal.
+3. Check the `/var/log/cloud-init-output.log` to check any error happened in
the processing of deployment.
+
+
+
+### The stack of related services is normal, but services aren't started
normally.
+
+#### `Kylin` starts failing.
+
+Q: `Kylin` starts failing.
+
+A:
+
+1. Users login into the `Kylin` instances by the `credential file` and the
public `IP` which is in the `Kylin` stack `Outputs`. Access command will be
like `ssh -i xxx.pem ec2-user@{public ip}` on the terminal.
+2. Change to the `root` user by executing `sudo su` on the terminal.
+3. Refresh variables of env by executing `source ~/.bash_profile` on the
terminal.
+4. Get the Kylin home by executing `echo $KYLIN_HOME` on the terminal.
+5. Check the logs of Kylin in `$KYLIN_HOME/logs/*`.
+6. More details about `Kylin` starting, user can check the scripts in
`backup/scripts/prepare-ec2-env-for-kylin4.sh`.
+
+
+
+#### `Prometheus` starts failing.
+
+Q: `Prometheus` starts failing.
+
+A:
+
+1. Users login into the `Static Services` instances by the `credential file`
and the public `IP` which is in the `Static Services` stack `Outputs`. Access
command will be like `ssh -i xxx.pem ec2-user@{public ip}` on the terminal.
+2. Change to the `root` user by executing `sudo su` on the terminal.
+3. Refresh variables of env by executing `source ~/.bash_profile` on the
terminal.
+4. Get the `Prometheus` home by executing `echo $PROMETHEUS_HOME` on the
terminal.
+5. Check the logs of `Prometheus` in `$PROMETHEUS_HOME/output.log`.
+6. More details about `Prometheus` starting, user can check the scripts in
`backup/scripts/prepare-ec2-env-for-static-services.sh`.
+
+
+
+#### `Granfana` starts failing.
+
+Q: `Granfana` starts failing.
+
+A:
+
+1. Users login into the `Static Services` instances by the `credential file`
and the public `IP` which is in the `Static Services` stack `Outputs`. Access
command will be like `ssh -i xxx.pem ec2-user@{public ip}` on the terminal.
+2. Change to the `root` user by executing `sudo su` on the terminal.
+3. `Granfana` is started as a docker service.
+ 1. Use `docker ps -a` to get the docker id of `Granfana`.
+ 2. User `docker exec -it ${granfana docker id}` to into `Granfana` service.
+ 3. Check the logs of `Grafana` in `/var/log/grafana/grafana.log`.
+4. More details about `Granfana` starting, user can check the scripts in
`backup/scripts/prepare-ec2-env-for-static-services.sh`.
+
+
+
+#### `Spark` starts failing.
+
+Q: `Spark` starts failing.
+
+A:
+
+1. Users login into the `Spark Master/Spark Slave` instances by the
`credential file` and the public `IP` which is in the `Spark Master / Spark
Slave` stack `Outputs`. Access command will be like `ssh -i xxx.pem
ec2-user@{public ip}` on the terminal.
+2. Change to the `root` user by executing `sudo su` on the terminal.
+3. Refresh variables of env by executing `source ~/.bash_profile` on the
terminal.
+4. Get the `SPARK` home by executing `echo $SPARK_HOME` on the terminal.
+5. Check the logs of `SPARK` in `$SPARK_HOME/logs/*.log`.
+6. More details about `Spark` starting, user can check the scripts in
`backup/scripts/prepare-ec2-env-for-spark-*.sh`.
+
+
+
+#### `Kylin starts` failing because can not connect to the Zookeeper.
+
+Q: `Kylin starts` failing because can not connect to the Zookeeper.
+
+A:
+
+1. Users login into the `Zookeeper` instances by the `credential file` and the
public `IP` which is in the Zookeeper stack `Outputs`. Access command will be
like `ssh -i xxx.pem ec2-user@{public ip}` on the terminal.
+2. Change to the `root` user by executing `sudo su` on the terminal.
+3. Refresh variables of env by executing `source ~/.bash_profile` on the
terminal.
+4. Get the Zookeeper home by executing `echo $ZOOKEEPER_HOME` on the terminal.
+5. Check the Zookeeper config file in `$ZOOKEEPER_HOME/conf/zoo.cfg`.
+6. Check the logs of Zookeeper in `/tmp/zookeeper/zk{1..3}/log`.
+7. Check the data-dir of Zookeeper in ``/tmp/zookeeper/zk{1..3}/data`.
+8. More details about `Zookeeper` starting, user can check the scripts in
`backup/scripts/prepare-ec2-env-for-zk.sh`.
+
+
+
+# TO BE CONTINUED...
+
+
+
