This is an automated email from the ASF dual-hosted git repository.
xxyu pushed a commit to branch doc5.0
in repository https://gitbox.apache.org/repos/asf/kylin.git
The following commit(s) were added to refs/heads/doc5.0 by this push:
new 83a72e7428 KYLIN-5247 add doc for system operation
83a72e7428 is described below
commit 83a72e7428c9114bbed475a538d91df419dded0b
Author: Mukvin <boyboys...@163.com>
AuthorDate: Tue Sep 6 10:42:43 2022 +0800
KYLIN-5247 add doc for system operation
---
.../data-access-control/acl_table.md | 75 ---------------------
.../access-control/data-access-control/intro.md | 4 +-
.../data-access-control/project_acl.md | 8 +--
.../operations/access-control/group_management.md | 2 +-
.../operations/access-control/images/acl_5.png | Bin 0 -> 105298 bytes
.../operations/access-control/images/acl_5_en.png | Bin 337881 -> 0 bytes
website/docs/operations/logs/audit_log.md | 18 ++---
website/docs/operations/logs/intro.md | 2 +-
website/docs/operations/logs/system_log.md | 9 +--
.../operations/monitoring/images/dashboard.jpg | Bin 235619 -> 51351 bytes
.../operations/monitoring/influxdb/influxdb.md | 17 +++--
.../monitoring/influxdb/influxdb_maintenance.md | 9 ++-
.../docs/operations/monitoring/metrics_intro.md | 30 ++++++---
website/docs/operations/monitoring/service.md | 2 -
website/docs/operations/overview.md | 4 +-
.../project-operation/images/add_project.png | Bin 66925 -> 20453 bytes
.../project-operation/images/project_list.png | Bin 262909 -> 61506 bytes
.../project-operation/images/project_settings.png | Bin 244469 -> 103717 bytes
.../images/project_settings_basic_general.png | Bin 170160 -> 28427 bytes
.../images/project_settings_cc_expose.png | Bin 48912 -> 11921 bytes
.../images/project_settings_custom_config.png | Bin 58667 -> 14203 bytes
.../images/project_settings_default_database.png | Bin 0 -> 42001 bytes
.../project_settings_default_database_en.png | Bin 297604 -> 0 bytes
.../images/project_settings_job_notification.png | Bin 144128 -> 65850 bytes
.../project_settings_job_notification_page.png | Bin 354436 -> 47490 bytes
.../images/project_settings_model_rewrite.png | Bin 250137 -> 66670 bytes
.../images/project_settings_pushdown.png | Bin 302372 -> 38508 bytes
.../images/project_settings_segment.png | Bin 0 -> 51780 bytes
.../images/project_settings_segment_en.png | Bin 191173 -> 0 bytes
.../images/project_settings_storage.png | Bin 233852 -> 0 bytes
.../images/project_settings_yarn_queue.png | Bin 90074 -> 23899 bytes
.../project-operation/images/service_state.png | Bin 58508 -> 0 bytes
.../project-operation/images/service_status.png | Bin 0 -> 29809 bytes
.../project-operation/images/storage_quota.png | Bin 34171 -> 0 bytes
.../project-operation/images/toolbar.png | Bin 688505 -> 11784 bytes
.../project-operation/project_management.md | 2 +-
.../project-operation/project_settings.md | 18 ++---
.../docs/operations/project-operation/toolbar.md | 27 ++------
.../system-operation/cli_tool/diagnosis.md | 7 +-
.../cli_tool/environment_dependency_check.md | 1 -
.../system-operation/cli_tool/images/accept_en.png | Bin 76977 -> 0 bytes
.../system-operation/cli_tool/images/button_en.png | Bin 195535 -> 0 bytes
.../cli_tool/images/rollback_en.png | Bin 80422 -> 0 bytes
.../metadata_tool/metadata_backup_restore.md | 2 +-
.../system-operation/cli_tool/rollback.md | 31 ++-------
.../system-operation/diagnosis/diagnosis.md | 13 ++--
.../diagnosis/images/diagnosis_web.png | Bin 0 -> 49064 bytes
.../diagnosis/images/diagnosis_web_en.png | Bin 179703 -> 0 bytes
.../diagnosis/images/job_diagnosis_web_01.png | Bin 0 -> 137453 bytes
.../diagnosis/images/job_diagnosis_web_02.png | Bin 0 -> 26558 bytes
.../diagnosis/images/job_diagnosis_web_en.png | Bin 141217 -> 0 bytes
.../diagnosis/images/query_diagnosis_web.png | Bin 127898 -> 0 bytes
.../diagnosis/images/query_diagnosis_web_01.png | Bin 0 -> 129183 bytes
.../diagnosis/images/query_diagnosis_web_02.png | Bin 0 -> 26307 bytes
...den_modify_metadata_during_maintenance_mode.png | Bin 189044 -> 72307 bytes
..._metadata_promoting_during_maintenance_mode.png | Bin 188242 -> 57544 bytes
.../system-operation/images/maintenance_mode.png | Bin 246534 -> 91393 bytes
.../system_recover_from_maintenance_mode.png | Bin 178347 -> 111166 bytes
.../operations/system-operation/junk_file_clean.md | 5 +-
website/docs/quickstart/quick_start.md | 3 +
website/sidebars.js | 4 --
61 files changed, 83 insertions(+), 210 deletions(-)
diff --git
a/website/docs/operations/access-control/data-access-control/acl_table.md
b/website/docs/operations/access-control/data-access-control/acl_table.md
deleted file mode 100644
index 340bc0b009..0000000000
--- a/website/docs/operations/access-control/data-access-control/acl_table.md
+++ /dev/null
@@ -1,75 +0,0 @@
----
-title: Table Access Control
-language: en
-sidebar_label: Table Access Control
-pagination_label: Table Access Control
-toc_min_heading_level: 2
-toc_max_heading_level: 6
-pagination_prev: null
-pagination_next: null
-keywords:
- - table access control
-draft: true
-last_update:
- date: Aug 16, 2022
----
-
-### <span id="table">Table ACL</span>
-
-**Introduction**
-
-Table ACLs determines whether a user/user group can access a certain table
loaded into Kylin. When a user/group is restricted to a table, the user/user
group cannot query the table.
-
-- System administrator can grant table-level data access to a user/user group.
All users/user groups that have access rights to the current project can read
or query all tables in the current project by default.
-- When the access rights of the user/user group are deleted, or the user/user
group is removed from the system, the corresponding table-level permissions are
also deleted.
-- When the table is deleted from the project, the Table ACL for all users/user
groups are also deleted.
-- When the table is reloaded in the project, the Table ACL for all users/user
groups are retained.
-
-**Example**
-
-Now we will use specific examples to show the product behavior after setting
table ACLs.
-
-If a user do not have the access to table `SSB.SUPPLIER`, he cannot see the
corresponding table under the project or in the connected BI tool.
-
-
-
-
-
-If you query the table, the result will show an error.
-
-
-
-### <span id="set">Grant Table ACL</span>
-
-1. In the **Admin** -> Project page, the system administrator can grant
user/user group project-level access rights. For details, please refer to
[Project ACL](project_acl.md).
-
-2. Select the project you want to authorize，then click **Authorization** under
the right **Action** bar and go to the authorization page.
-
-3. Click the left arrow button and expand the user/user group that requires
authorization, and the interface displays the tables / columns / rows that the
user/user group has access to.
-
- 
-
-4. Click the **Edit** button and check the tables and columns that need to be
granted access. There is no row-level access restriction by default, which
means that users can access all the rows in the column.
-
- 
-
-5. Click the **+ Add** button on the right of the **Row Access List**, select
the column in the pop-up window and enter the rows that can be accessed.
Support for **IN** or **LIKE** rules to set row level permissions, increase or
decrease row-level access via the **+/-** button on the right, and click
**Submit**.
-
- 
-
-6. For granted row-level access, click the **...** button on the right to
modify the settings, or click **Delete** on the right.
-
- 
-
-7. Confirm that the currently settings for Table ACLs are correct and click
**Submit**.
-
-
-
-### <span id="notice">Notices</span>
-
-- The authorization operation is a whitelist display for the user/user group,
that is, only the tables / rows / columns permissions added for the user/user
group are accessible.
-- By default, a user/user group will be automatically granted all access
permissions on all tables in this project after added into this project. You
can modify the configuration item
`kylin.acl.project-internal-default-permission-granted=true` in the
configuration file. After setting it to `false`, when the user/user group is
added to the project, there is no table granted by default. System
administrators can manually select tables and set access permissions.
-- The access to the user/user group takes the largest set of results that can
be accessed (union).
- - If user `user1` has access to table `table1`, user `user1` is in user
group `group1`, and user group `group1` has access to table `table2`. At this
time user `user1` can also access the tables `table1` and `table2`.
- - If user group `group1` has access to tables `table1` and `table2`, user
group `group2` has access to tables `table2` and `table3`, user `user1` belongs
to user group `group1` and `group2`. At this time `user1` has access to the
tables `table1`, `table2` and `table3`.
-- Only system administrator users can perform **add/delete/change **
operations at the Table / Column / Row ACL.
diff --git
a/website/docs/operations/access-control/data-access-control/intro.md
b/website/docs/operations/access-control/data-access-control/intro.md
index b84e0958ab..2738ca98d1 100644
--- a/website/docs/operations/access-control/data-access-control/intro.md
+++ b/website/docs/operations/access-control/data-access-control/intro.md
@@ -29,11 +29,9 @@ Kylin provides a rich set of access control features for big
enterprise. Start f
### Examples of Permission Check
-To perform an action, user must have both operation permission and data access
permission. Below are a few examples.
+To perform an action, user must have operation permission. Below is few
examples.
-- To manage source tables, user needs the ADMIN permission, and only the
tables user can access can be seen and acted. (Column and row ACLs does not
impact the source table management.)
- To edit a model, user must have the MANAGEMENT permission and have access to
all the tables and columns in the model.
-- Running queries is mostly about data access control, since all users in a
project have at least QUERY permission. First user must have access to all the
tables and columns in the query, or the system will prompt permission error and
refuse to execute. Second the system will only return rows that are accessible
to a user. If different row ACLs are set for users, they may see different
results from a same query.
### Other Notes
diff --git
a/website/docs/operations/access-control/data-access-control/project_acl.md
b/website/docs/operations/access-control/data-access-control/project_acl.md
index 2b38cac6e7..e1cb9baf26 100755
--- a/website/docs/operations/access-control/data-access-control/project_acl.md
+++ b/website/docs/operations/access-control/data-access-control/project_acl.md
@@ -33,7 +33,7 @@ After the system administrator assigns project access
permission to a group, use
3. Click **+ User / Group** to grant access for a user / user group.
4. Select the grant type : by **User** or by **User Group**. Then select the
user / user group and access permission to be granted, and click **Submit**.
-
+
### Modify Project ACL
@@ -46,9 +46,3 @@ After the system administrator assigns project access
permission to a group, use
1. Select the project and click the **Authorization** icon under the
**Actions** column on the right to enter the authorization page.
2. Select the user / user group in the list and click the **Delete** icon
under the **Actions** column on the right.
-
-> **Caution**: When user/user group's project access permission has been
revoked, all access permission on this project including table-level, row-level
and column-level will be revoked subsequently.
-
-### Data authority control
-
-After setting the system level configuration parameter
**kylin.security.allow-project-admin-grant-acl=false**, the project
administrator in each project has no right to assign table / row / column
access rights to users or user groups, but can still view the table / row /
column access rights of users or user groups.
diff --git a/website/docs/operations/access-control/group_management.md
b/website/docs/operations/access-control/group_management.md
index 1045c03d45..93fcb68474 100644
--- a/website/docs/operations/access-control/group_management.md
+++ b/website/docs/operations/access-control/group_management.md
@@ -18,7 +18,7 @@ This chapter provides an overview of what a user group is and
how a user group c
### <span id="group">About User Group</span>
-A user group is a collection of users, and users in a user group share the
same ACL. By default, Kylin initializes four user groups, namely ALL_USERS,
ROLE_ADMIN, ROLE_ANALYST, and ROLE_MODELER, and ALL_USERS group is a default
user group, all users are included in the ALL_USERS user group. ALL_USERS and
ROLE_ADMIN user group cannot be modified or deleted. System administrators can
add or remove users in user groups except ALL_USERS, or add a user to multiple
groups except ALL_USERS. Use [...]
+A user group is a collection of users, and users in a user group share the
same ACL. By default, Kylin initializes four user groups, namely **ALL_USERS**,
**ROLE_ADMIN**, **ROLE_ANALYST**, and **ROLE_MODELER**. And **ALL_USERS** group
is a default user group, all users are included in the **ALL_USERS** user
group. **ALL_USERS** and **ROLE_ADMIN** user group cannot be modified or
deleted. System administrators can add or remove users in user groups except
ALL_USERS, or add a user to multi [...]
### <span id="authority">About User Group Permissions</span>
diff --git a/website/docs/operations/access-control/images/acl_5.png
b/website/docs/operations/access-control/images/acl_5.png
new file mode 100644
index 0000000000..b89fdef259
Binary files /dev/null and
b/website/docs/operations/access-control/images/acl_5.png differ
diff --git a/website/docs/operations/access-control/images/acl_5_en.png
b/website/docs/operations/access-control/images/acl_5_en.png
deleted file mode 100644
index ae06dff693..0000000000
Binary files a/website/docs/operations/access-control/images/acl_5_en.png and
/dev/null differ
diff --git a/website/docs/operations/logs/audit_log.md
b/website/docs/operations/logs/audit_log.md
index 3cd52a124f..5c80fda56b 100644
--- a/website/docs/operations/logs/audit_log.md
+++ b/website/docs/operations/logs/audit_log.md
@@ -32,9 +32,9 @@ When Kylin is started for the first time, according to the
metadata table name y
e.g：
-`kylin.metadata.url=ke_metadata@jdbc,driverClassName=org.postgresql.Driver,url=jdbc:postgresql://sandbox:5432/kylin,username=postgres,password=`
+`kylin.metadata.url=kylin_metadata@jdbc,driverClassName=org.postgresql.Driver,url=jdbc:postgresql://sandbox:5432/kylin,username=postgres,password=`
-The metadata is named `kylin`, the metadata table is named` ke_metadata`, and
the Audit Log table is named `ke_metadata_audit_log`.
+The metadata is named `kylin`, the metadata table is named` kylin_metadata`,
and the Audit Log table is named `kylin_metadata_audit_log`.
@@ -107,31 +107,31 @@ e.g：
1. View the history of an item of metadata
```
- select * from ke_metadata_audit_log where meta_key =
'/_global/project/default.json';
+ select * from kylin_metadata_audit_log where meta_key =
'/_global/project/default.json';
```
2. View the history of a model
```
- select * from ke_metadata_audit_log where meta_key =
'/project/model_desc/49529000-c161-4013-bb80-9a78f4f0248d.json'
+ select * from kylin_metadata_audit_log where meta_key =
'/project/model_desc/49529000-c161-4013-bb80-9a78f4f0248d.json'
```
3. View the change of metadata in a transaction
```
- select * from ke_metadata_audit_log where unit_id =
'6090bfb5-2401-4176-8475-fe6fd82bc439';
+ select * from kylin_metadata_audit_log where unit_id =
'6090bfb5-2401-4176-8475-fe6fd82bc439';
```
4. View audit logs for metadata over a period of time
```
- select * from ke_metadata_audit_log where meta_ts > 1325376000000 and
meta_ts < 1328054400000 ;
+ select * from kylin_metadata_audit_log where meta_ts > 1325376000000 and
meta_ts < 1328054400000 ;
```
5. Associate the metadata table to view the history changes of a user
```
- select a.meta_mvcc, a.meta_content, b.meta_table_mvcc, b.meta_table_content
from ke_metadata_audit_log a left join ke_metadata b on a.meta_key =
b.meta_table_key where a.meta_key = '/_global/user/ADMIN'
+ select a.meta_mvcc, a.meta_content, b.meta_table_mvcc, b.meta_table_content
from kylin_metadata_audit_log a left join kylin_metadata b on a.meta_key =
b.meta_table_key where a.meta_key = '/_global/user/ADMIN'
```
@@ -150,7 +150,7 @@ e.g：
-
+
- Monitor if the model has been modified
@@ -199,7 +199,7 @@ There are two ways to execute commands on the KE node:
If you have the Audit Log file exported locally and want to view and analyze
it through the database, you can use the following methods to import.
-On a machine that already has a Kylin environment, use the AuditLogTool tool:
`${KYLIN_HOME}/kylin.sh io.kyligence.kap.tool.AuditLogTool -restore -table
${target_table_name} -dir ${auditlog_dir}`
+On a machine that already has a Kylin environment, use the AuditLogTool tool:
`${KYLIN_HOME}/kylin.sh org.apache.kylin.tool.AuditLogTool -restore -table
${target_table_name} -dir ${auditlog_dir}`
- The `$ {target_table_name}` parameter specifies the name of the Audit Log
table to be generated. Be careful not to duplicate the name of the Audit Log
table already in the Kylin environment.
- The `$ {auditlog_dir}` parameter specifies the directory where the Audit Log
file is located;
diff --git a/website/docs/operations/logs/intro.md
b/website/docs/operations/logs/intro.md
index 9c2f2ebec6..00cd4e57f8 100644
--- a/website/docs/operations/logs/intro.md
+++ b/website/docs/operations/logs/intro.md
@@ -16,6 +16,6 @@ last_update:
This chapter mainly introduces different log types:
-* [System Log](logging.md)
+* [System Log](system_log.md)
* [Audit Log](audit_log.md)
diff --git a/website/docs/operations/logs/system_log.md
b/website/docs/operations/logs/system_log.md
index d80321a73d..8e17b360e6 100755
--- a/website/docs/operations/logs/system_log.md
+++ b/website/docs/operations/logs/system_log.md
@@ -15,8 +15,6 @@ last_update:
---
-## System Log
-
After being successfully started, Kylin will create a directory named `logs/`
by default, all logs generated during Kylin runtime will be stored in this
directory.
### Log files
@@ -64,9 +62,6 @@ This file records logs related to task scheduling, whose
default logging level i
#### `kylin.query.log`
This file records query related logs, whose default logging level is DEBUG.
-#### `kylin.smart.log`
-This file records recommendation-related logs, whose default logging level is
DEBUG.
-
#### `dump.hprof`
When Out of Memory (OOM) occurs in Kylin, it will dump the entire heap, which
is convenient for checking the cause.
> Note: When you have a large memory setting and Out of Memory OOM occurs, the
> file dump.hprof will occupy a large storage space, which may cause your disk
> space to be insufficient and the node to be abnormal. You can manually clean
> up the historical file.
@@ -83,7 +78,7 @@ User: ADMIN
Success: true
Duration: 1.243
Project: ssb100_10
-Realization Names: [AUTO_MODEL_P_LINEORDER_1]
+Realization Names: [P_LINEORDER_1]
Index Layout Ids: [30001]
Snapshot Names: []
Is Partial Match Model: [false]
@@ -106,6 +101,7 @@ Is forced to Push-Down: false
User Agent: null
Scan Segment Count: 1
Scan File Count: 1
+Is Refused: false
==========================[QUERY]===============================
```
The main fields in the above clip are described as follows:
@@ -139,6 +135,7 @@ The main fields in the above clip are described as follows:
* `User Agent`： The environment information used to submit the query
* `Scan Segment Count`： Number of scanned segments
* `Scan File Count`： Number of scanned files
+* `Is Refused`: Whether to refuse to query
### Logging Configuration
diff --git a/website/docs/operations/monitoring/images/dashboard.jpg
b/website/docs/operations/monitoring/images/dashboard.jpg
index 9d893aaa28..d54f464d2e 100644
Binary files a/website/docs/operations/monitoring/images/dashboard.jpg and
b/website/docs/operations/monitoring/images/dashboard.jpg differ
diff --git a/website/docs/operations/monitoring/influxdb/influxdb.md
b/website/docs/operations/monitoring/influxdb/influxdb.md
index dfaa38a3c0..f9e40194f6 100644
--- a/website/docs/operations/monitoring/influxdb/influxdb.md
+++ b/website/docs/operations/monitoring/influxdb/influxdb.md
@@ -21,9 +21,9 @@ Starting with Kylin 5.0, the system uses RDBMS to store query
history, which onl
If you need this information, you need to configure the time series database
InfluxDB in advance to store data such as the monitoring information of the
system.
-We recommend you to use InfluxDB v1.6.4, which is provided in the Kylin
installation package.
+We recommend you to use InfluxDB v1.6.4, which can download by the script
`$KYLIN_HOME/sbin/download-influxdb.sh`.
-The InfluxDB installation package, `influxdb-1.6.4.x86_64.rpm` is under the
`influxdb` directory in the installation directory of Kylin.
+The InfluxDB installation package, `influxdb-1.6.4.x86_64.rpm` will be under
the `influxdb` directory in the installation directory of Kylin.
If you need to use the existed InfluxDB database in your environment, please
use the versions below:
@@ -45,9 +45,13 @@ The following steps are using InfluxDB 1.6.4 as an example.
service influxdb status
```
- If not, you can go to the directory where the InfluxDB installation package
is located and install InfluxDB.
+ If not, you can download the influxdb and go to the directory where the
InfluxDB installation package is located and install InfluxDB.
```shell
+ # download influxDB
+ $KYLIN_HOME/sbin/download-influxdb.sh
+
+ # go to influxDB directory and install it
cd $KYLIN_HOME/influxdb
rpm -ivh influxdb-1.6.4.x86_64.rpm
```
@@ -146,14 +150,17 @@ The following steps are using InfluxDB 1.6.4 as an
example.
7. start Kylin.
-### <span id="not_root">Installation and Configuration for Non `root` User
</span>
+### <span id="not_root">Installation and Configuration for `Non root` User
</span>
The following steps are using InfluxDB 1.6.4 as an example.
-1. Suppose you install as user `abc` . Then create a directory
`home/abc/influx` to copy the InfluxDB installation package,
`influxdb-1.6.4.x86_64.rpm` ,from `$KYLIN_HOME/influxdb` to this directory.
+1. Suppose you install as user `abc` . Then create a directory
`home/abc/influx` to copy the InfluxDB installation package,
`influxdb-1.6.4.x86_64.rpm` ,from `$KYLIN_HOME/influxdb` to this directory
after executing download influxDB script.
```sh
+ # download influxDB
+ $KYLIN_HOME/sbin/download-influxdb.sh
+
mkdir /home/abc/influx
cp $KYLIN_HOME/influxdb/influxdb-1.6.4.x86_64.rpm /home/abc/influx
cd /home/abc/influx
diff --git
a/website/docs/operations/monitoring/influxdb/influxdb_maintenance.md
b/website/docs/operations/monitoring/influxdb/influxdb_maintenance.md
index 9cc609194c..a546e114d1 100644
--- a/website/docs/operations/monitoring/influxdb/influxdb_maintenance.md
+++ b/website/docs/operations/monitoring/influxdb/influxdb_maintenance.md
@@ -47,7 +47,7 @@ InfluxDB provides the availability to do backup and restore.
- **Backup**
```sh
- influxd backup -portable -database KE_METRIC -host 127.0.0.1:8089
/path/to/backup
+ influxd backup -portable -database KYLIN_METRIC -host 127.0.0.1:8089
/path/to/backup
```
- **Restore**
@@ -55,10 +55,10 @@ InfluxDB provides the availability to do backup and restore.
Please make sure that the database exists, otherwise the restore will
be failed.
```sh
- influxd restore -portable -database KE_METRIC -host 127.0.0.1:8089
/path/to/backup
+ influxd restore -portable -database KYLIN_METRIC -host 127.0.0.1:8089
/path/to/backup
```
-> **note:** Please replace KE_METRIC with the actual database name, replace
127.0.0.1:8089 with the actual IP and port, replace `/path/to/backup` with the
path you would like to set.
+> **note:** Please replace KYLIN_METRIC with the actual database name, replace
127.0.0.1:8089 with the actual IP and port, replace `/path/to/backup` with the
path you would like to set.
### Monitoring and Diagnosis
@@ -67,7 +67,7 @@ InfluxDB provides the availability to do backup and restore.
- Check runtime
Run following command to check GC, memory usage, etc.
- `influx -database KE_METRIC -execute "show stats for 'runtime'"`
+ `influx -database KYLIN_METRIC -execute "show stats for 'runtime'"`
Please focus on these important arguments:
- *HeapAlloc* -> Heap allocation size
@@ -121,4 +121,3 @@ InfluxDB provides the availability to do backup and restore.
WHERE time > now() - 10d
GROUP BY time(1h)
```
-
diff --git a/website/docs/operations/monitoring/metrics_intro.md
b/website/docs/operations/monitoring/metrics_intro.md
index fc7bc1a0a1..0a6428be0f 100644
--- a/website/docs/operations/monitoring/metrics_intro.md
+++ b/website/docs/operations/monitoring/metrics_intro.md
@@ -17,7 +17,15 @@ last_update:
By default, the system collects metric data every minute, including storage,
query, job, metadata, and cleanup mechanism. The monitoring data is stored in
the specified [InfluxDB](https://www.influxdata.com/time-series-platform/) and
displayed through [Grafana](https://grafana.com/grafana). It can help
administrators to understand the health of the system in order to take
necessary actions.
-> **Note**: Since Grafana depends on InfluxDB, please make sure that InfluxDB
is correctly configured and started according to [Use InfluxDB as Time-Series
Database](influxdb/influxdb.md) before you use Grafana.
+> **Note**: Since Grafana depends on **InfluxDB**, please make sure that
InfluxDB is correctly configured and started according to [Use InfluxDB as
Time-Series Database](influxdb/influxdb.md) and download **Grafana** before you
use Grafana.
+
+```shell
+# Download Grafana
+$KYLIN_HOME/sbin/download-grafana.sh
+```
+
+> **Note**: The Grafana installation path, will be under the `grafana`
directory in the installation directory of Kylin.
+
### <span id="grafana_startup">Grafana</span>
@@ -30,11 +38,11 @@ By default, the system collects metric data every minute,
including storage, que
After the startup is successful, you may access Grafana through web browser
with default port: 3000, username: admin, password: admin
-[comment]: <#TODO> ()
+
### <span id="dashboard">Dashboard</span>
-Default Dashboard: ```Kylin```
+Default Dashboard: ```Kylin 5.0```
The dashboard consists of 10 modules: Cluster, Summaries, Models, Queries,
Favorites, Jobs, Cleanings, Metadata Operations, Transactions, among which
Summaries module is automatically displayed in detail. Read more details about
the modules, please refer to [Metrics Explanation](#explanation). If you want
to make some changes for the dashboard, please refer to Grafana official
website manual [Provisioning
Grafana](https://grafana.com/docs/administration/provisioning/).
@@ -65,7 +73,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
> **Tip **: “Project related” in the following table indicates whether the
> metric is related to the project, “Y” indicates that the metric is related
> to the project, and “N” indicates that the metric is not related to the
> project. "Host related" in the following table indicates whether the metric
> is related to Kylin nodes, "Y" indicates that the metric is related to the
> Kylin nodes, "N" indicates that the metric is not related to the host.
> "all", "job", "query" is Kylin nodes' server mode.
-<span id="cluster">**Cluster**：Cluster overview</span>
+#### <span id="cluster">Cluster: Cluster overview</span>
| Name | Meaning | Project related |
| :------------- | :---------- | :----------- |
@@ -92,7 +100,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
| query_load | spark sql load | N | Y(all, query) | - |
| cpu_cores | The number of cup cores for query configured in
kylin.properties | N | Y(all, query) | Refer "Spark-related Configuration" |
-<span id="models">**Models**：Model related metrics</span>
+#### <span id="models">Models：Model related metrics</span>
| Name | Meaning | Project related | Host related |
| :------------- | :---------- | :----------- | :----------- |
@@ -108,7 +116,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
| expansion_rate_of_models | Expansion rate of models | Y | N |
| model_build_duration (avg) | Avg build time of models | Y | N |
-<span id="queries">**Queries**：Query related metrics</span>
+#### <span id="queries">Queries：Query related metrics</span>
| Name | Meaning | Project related | Host related | Remark |
| :------------- | :---------- | :----------- | :----------- | :----------- |
@@ -143,7 +151,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
| query_scan_bytes_of_host | Query scan bytes per host | N |
Y(all, query) |-|
| mean_scan_bytes_of_queries | The mean scan bytes of queries | Y |
Y(all, query) | - |
-<span id="favorites">**Favorites**：Favorite Query related metrics</span>
+#### <span id="favorites">Favorites：Favorite Query related metrics</span>
| Name | Meaning | Project related | Host related | Remark |
| :------------- | :---------- | :----------- | :----------- | :----------- |
@@ -163,7 +171,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
| fq_pending_num_gauge | Favorite Query pending | Y | N | Favorite Query
lacks of necessary conditions, such as missing column names, requiring user
intervention |
| fq_blacklist_num_gauge | Favorite Query in blacklist | Y | N | Refer to the
definition of "Blacklist" |
-<span id="jobs">**Jobs**：Job related metrics</span>
+#### <span id="jobs">Jobs：Job related metrics</span>
| Name | Meaning | Project related | Host related |
| :------------- | :---------- | :----------- | :----------- |
@@ -182,7 +190,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
| job_duration | The build duration of job | Y |
Y(all, job) |
| job_wait_duration | The wait duration of job | Y |
Y(all, job) |
-<span id="cleanings">**Cleanings**：Cleanup mechanisms related metrics</span>
+#### <span id="cleanings">Cleanings：Cleanup mechanisms related metrics</span>
| Name | Meaning | Project related | Host related |
| :------------- | :---------- | :----------- | :----------- |
@@ -190,7 +198,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
| storage_clean_total_duration | Storage cleanup total duration | N | Y(all,
job, query) |
| failed_storage_clean_total_times | Storage cleanup failed total times | N |
Y(all, job, query) |
-<span id="metadata">**Metadata Operations**：Metadata operations related
metrics</span>
+#### <span id="metadata">Metadata Operations：Metadata operations related
metrics</span>
| Name | Meaning | Project related | Host related | Remark |
| :------------- | :---------- | :----------- | :----------- | :----------- |
@@ -201,7 +209,7 @@ Located in the upper left corner of the dashboard, the data
granularity: auto, 1
| metadata_ops_total_times | Metadata daily operations total times | N |
Y(all, job, query) | Fixed time per day (configurable): automatically backup
metadata; rotate audit_log; cleanup metadata and storage space; adjust FQ;
cleanup query histories. |
| metadata_success_ops_total_times | Metadata daily operations failed total
times | N | Y(all, job, query) |-|
-<span id="transactions">**Transactions**：Transaction mechanisms related
metrics</span>
+#### <span id="transactions">Transactions：Transaction mechanisms related
metrics</span>
| Name | Meaning | Project related | Host related | Remark |
| :------------- | :---------- | :----------- | :----------- |:----------- |
diff --git a/website/docs/operations/monitoring/service.md
b/website/docs/operations/monitoring/service.md
index 586ac0409b..d6947e17cc 100644
--- a/website/docs/operations/monitoring/service.md
+++ b/website/docs/operations/monitoring/service.md
@@ -14,8 +14,6 @@ last_update:
date: 08/12/2022
---
-## Service Monitoring
-
Kylin provides the service monitoring for main components to help
administrators obtain the service status and maintain instances.
Currently, we provide the following methods to monitor the core components in
Kylin:
diff --git a/website/docs/operations/overview.md
b/website/docs/operations/overview.md
index d3c9b22883..21f90e587a 100755
--- a/website/docs/operations/overview.md
+++ b/website/docs/operations/overview.md
@@ -16,7 +16,7 @@ last_update:
As Kylin system administrator, typical daily operations include:
-- To ensure Kylin service running smoothly, system administrator should
monitor system logs on a regular basis.
+- To ensure **Kylin** service running smoothly, system administrator should
monitor system logs on a regular basis.
- To ensure building jobs run successfully, system administrator needs to
monitor jobs execution status via email notification or system web UI.
-- To ensure there are enough cluster resources for Kylin, system administrator
should check the YARN queue and storage utilization frequently.
+- To ensure there are enough cluster resources for **Kylin**, system
administrator should check the YARN queue and storage utilization frequently.
- To prevent any data loss or system failure, system administrator should make
plans for system backup and disaster recovery.
diff --git a/website/docs/operations/project-operation/images/add_project.png
b/website/docs/operations/project-operation/images/add_project.png
index 474e307aac..a05182481d 100644
Binary files a/website/docs/operations/project-operation/images/add_project.png
and b/website/docs/operations/project-operation/images/add_project.png differ
diff --git a/website/docs/operations/project-operation/images/project_list.png
b/website/docs/operations/project-operation/images/project_list.png
index f9763fe6b1..26e4dfc795 100644
Binary files
a/website/docs/operations/project-operation/images/project_list.png and
b/website/docs/operations/project-operation/images/project_list.png differ
diff --git
a/website/docs/operations/project-operation/images/project_settings.png
b/website/docs/operations/project-operation/images/project_settings.png
index 20a687a052..c6684927d5 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings.png and
b/website/docs/operations/project-operation/images/project_settings.png differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_basic_general.png
b/website/docs/operations/project-operation/images/project_settings_basic_general.png
index 6f5ddd598b..0d5a459e75 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_basic_general.png
and
b/website/docs/operations/project-operation/images/project_settings_basic_general.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_cc_expose.png
b/website/docs/operations/project-operation/images/project_settings_cc_expose.png
index 1f5a5a9ea1..98f7a42244 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_cc_expose.png
and
b/website/docs/operations/project-operation/images/project_settings_cc_expose.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_custom_config.png
b/website/docs/operations/project-operation/images/project_settings_custom_config.png
index 84324896a0..388d725c99 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_custom_config.png
and
b/website/docs/operations/project-operation/images/project_settings_custom_config.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_default_database.png
b/website/docs/operations/project-operation/images/project_settings_default_database.png
new file mode 100644
index 0000000000..c8086d5aa5
Binary files /dev/null and
b/website/docs/operations/project-operation/images/project_settings_default_database.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_default_database_en.png
b/website/docs/operations/project-operation/images/project_settings_default_database_en.png
deleted file mode 100644
index c7a28165ac..0000000000
Binary files
a/website/docs/operations/project-operation/images/project_settings_default_database_en.png
and /dev/null differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_job_notification.png
b/website/docs/operations/project-operation/images/project_settings_job_notification.png
index 1e082cc79b..a6fa9ce7bd 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_job_notification.png
and
b/website/docs/operations/project-operation/images/project_settings_job_notification.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_job_notification_page.png
b/website/docs/operations/project-operation/images/project_settings_job_notification_page.png
index 05de4610b3..5fdbbf0f74 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_job_notification_page.png
and
b/website/docs/operations/project-operation/images/project_settings_job_notification_page.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_model_rewrite.png
b/website/docs/operations/project-operation/images/project_settings_model_rewrite.png
index 43b61350ac..43296e820b 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_model_rewrite.png
and
b/website/docs/operations/project-operation/images/project_settings_model_rewrite.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_pushdown.png
b/website/docs/operations/project-operation/images/project_settings_pushdown.png
index c4b9a45cdd..308002a27b 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_pushdown.png
and
b/website/docs/operations/project-operation/images/project_settings_pushdown.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_segment.png
b/website/docs/operations/project-operation/images/project_settings_segment.png
new file mode 100644
index 0000000000..927b408fcd
Binary files /dev/null and
b/website/docs/operations/project-operation/images/project_settings_segment.png
differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_segment_en.png
b/website/docs/operations/project-operation/images/project_settings_segment_en.png
deleted file mode 100644
index 825e3a384f..0000000000
Binary files
a/website/docs/operations/project-operation/images/project_settings_segment_en.png
and /dev/null differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_storage.png
b/website/docs/operations/project-operation/images/project_settings_storage.png
deleted file mode 100644
index 5aa17d2588..0000000000
Binary files
a/website/docs/operations/project-operation/images/project_settings_storage.png
and /dev/null differ
diff --git
a/website/docs/operations/project-operation/images/project_settings_yarn_queue.png
b/website/docs/operations/project-operation/images/project_settings_yarn_queue.png
index 3d3590f228..c5f3e970bd 100644
Binary files
a/website/docs/operations/project-operation/images/project_settings_yarn_queue.png
and
b/website/docs/operations/project-operation/images/project_settings_yarn_queue.png
differ
diff --git a/website/docs/operations/project-operation/images/service_state.png
b/website/docs/operations/project-operation/images/service_state.png
deleted file mode 100644
index 69dd4e44b1..0000000000
Binary files
a/website/docs/operations/project-operation/images/service_state.png and
/dev/null differ
diff --git
a/website/docs/operations/project-operation/images/service_status.png
b/website/docs/operations/project-operation/images/service_status.png
new file mode 100644
index 0000000000..fecb2f4994
Binary files /dev/null and
b/website/docs/operations/project-operation/images/service_status.png differ
diff --git a/website/docs/operations/project-operation/images/storage_quota.png
b/website/docs/operations/project-operation/images/storage_quota.png
deleted file mode 100644
index 40953f6ebc..0000000000
Binary files
a/website/docs/operations/project-operation/images/storage_quota.png and
/dev/null differ
diff --git a/website/docs/operations/project-operation/images/toolbar.png
b/website/docs/operations/project-operation/images/toolbar.png
index 2e88c1d75a..632c419475 100644
Binary files a/website/docs/operations/project-operation/images/toolbar.png and
b/website/docs/operations/project-operation/images/toolbar.png differ
diff --git a/website/docs/operations/project-operation/project_management.md
b/website/docs/operations/project-operation/project_management.md
index ba84002070..4a9cde21d5 100644
--- a/website/docs/operations/project-operation/project_management.md
+++ b/website/docs/operations/project-operation/project_management.md
@@ -59,4 +59,4 @@ On the Project Management page, select a project to change
owner, click the **..
### <span id="grant">Grant Porject-level Access Rights to User</span>
-On the Project Management page, select a project to authorizer, click the
**Authorization** button under the **Actions** bar on the right and set
permissions for the project. You can find out more at the [Project
ACL](../access-control/project_acl.md) section.
+On the Project Management page, select a project to authorizer, click the
**Authorization** button under the **Actions** bar on the right and set
permissions for the project. You can find out more at the [Project
ACL](../access-control/data-access-control/project_acl.md) section.
diff --git a/website/docs/operations/project-operation/project_settings.md
b/website/docs/operations/project-operation/project_settings.md
index 4f247ffbd6..3843169003 100644
--- a/website/docs/operations/project-operation/project_settings.md
+++ b/website/docs/operations/project-operation/project_settings.md
@@ -29,25 +29,17 @@ All settings are composed of three main parts:
#### 1. General Information
-In this part, you can check current project's name, type and description,
also, you can modify project description.
+In this part, you can check current project's name and description, also, you
can modify project description.
-#### 2. Storage Setting
-
-In this part, you can check the storage quota, which is the upper limit of the
storage. By default, storage size is 10TB.
-
-The picture below shows where to set storage quota:
-
-
-
-#### 3. Pushdown Setting
+#### 2. Pushdown Setting
If you don't have any models or there is no avaliable models to answer your
queries, you can use **pushdown** to get results for a more flexible user
experience, and your queries will be pushdown to Spark SQL by default. This
setting is turned on by default.
-#### 4. Segment Settings
+#### 3. Segment Settings
This part is about settings of segment, including Segment Merge, Retention
Threshold and Creating Reserved Segments.
@@ -73,7 +65,7 @@ This part is about settings of segment, including Segment
Merge, Retention Thres
The picture below is the segment setting page.
-
+
### <span id="advanced">Advanced Settings</span>
@@ -85,7 +77,7 @@ After setting the default database, the database name can be
omitted in SQL stat
The picture below is the default database setting page:
-
+
#### 2. Job Notification
diff --git a/website/docs/operations/project-operation/toolbar.md
b/website/docs/operations/project-operation/toolbar.md
index b096da99bb..d4d37e04b4 100644
--- a/website/docs/operations/project-operation/toolbar.md
+++ b/website/docs/operations/project-operation/toolbar.md
@@ -23,32 +23,15 @@ The toolbar of Kylin system is as shown in the following
figure.
Project list is located on the left side of the toolbar, and the current
project is selected. If it is the first time to start Kylin, current project
would be empty. The icon on the left side of this list is used for expand and
collapse the navigation bar. The button of `+` on the right side is used for
adding new project. The newly added project will be set as the current project.
-### <span id="storage_quota">Storage Quota</span>
-
-Click **Storage Quota** for an overview of quota usage.
-
-
-
-- **Used Storage**: The percentage of storage quota consumed for this project.
The values enclosed in parentheses represent the storage quota consumed and
total storage quota allocated. By default, the total storage quota is 10 TB. To
adjust the default value, see [Project Settings](project_settings.md).
-
- The stored data mainly consists of model and index data, build job outputs,
and dictionary files, etc. For more information, see [Capacity
Billing](../capacity_billing.en.md).
-
-- **Low Usage Storage**: Indexes that are less frequently queried within the
set time window. By default, indexes that are queried less than 5 times in one
month would be considered as low usage storage. To modify this rule, see
[Project Settings](project_settings.md).
-
- > [!NOTE]
- >
- > You can manually clean low usage storage (with
[Tutorial](../../quickstart/expert_mode_tutorial.md) enabled) or set scheduled
cleaning task. For more information, see [Junk File
Clean](../system-operation/junk_file_clean.md).
-
-### <span id="storage_quota">Service State</span>
-
-The colored dot indicates the status of the service status for the current
project. Clicking on it will prompt a dialog shown as follow. The first line in
the figure shows the current time of the system. The second line shows the
percentage of data used.The third line shows the currently used node
information, and the icon on the right `>` can expand to view the detailed node
information.
+### <span id="system_management">System Management</span>
-
+After the system admin logs into Kylin, click **Admin** on the top bar to
enter the administration mode. For details, please refer to the [Project
Management](project_management.md) chapter.
+### <span id="storage_quota">Service Status</span>
-### <span id="system_management">System Management</span>
+The colored dot indicates the status of the service status for the current
project. Clicking on it will prompt a dialog shown as follow. The first line in
the figure shows the current time of the system. The second line shows the
currently used node information, and the icon on the right `>` can expand to
view the detailed node information.
-After the system admin logs into Kylin, click **Admin** on the top bar to
enter the administration mode. For details, please refer to the [Project
Management](project_management.md) chapter.
+
### <span id="user_info">User Information</span>
diff --git a/website/docs/operations/system-operation/cli_tool/diagnosis.md
b/website/docs/operations/system-operation/cli_tool/diagnosis.md
index e26e470720..18d180cf89 100644
--- a/website/docs/operations/system-operation/cli_tool/diagnosis.md
+++ b/website/docs/operations/system-operation/cli_tool/diagnosis.md
@@ -18,7 +18,6 @@ Kylin users may face with many problems during usage, such as
failure in creatin
Diagnosis function including System Diagnosis、Job Diagnosis and Query
Diagnosis.
### Generate Diagnostic Package using bash script
-- In FusionInsight, you need to execute command `source
/opt/hadoopclient/bigdata_env` first. The `hadoopclient` is a variable.
- Generate system diagnostic package
Execute `$KYLIN_HOME/bin/diag.sh [-startTime <START_TIMESTAMP> -endTime
<END_TIMESTAMP>] [-destDir <DESTINATION_DIR>]` to generate the system
diagnostic package. The default time range is last 1 day, START_TIMESTAMP and
END_TIMESTAMP is in unix timestamp. You may get the current unix timestamp by
run ```echo $((`date +%s`*1000))``` in Linux.
@@ -55,8 +54,7 @@ Diagnostic packages generated by scripts are stored under
`$KYLIN_HOME/diag_dump
- `/client`: operating system resources occupied information, Hadoop version
and Kerberos information.
- `/bin`: program execute and manager binary files.
- `/monitor_metrics`: System monitoring statistics.
-- `/rec_candidate`: optimization suggestions for the model.
-- `/write_ hadoop_ conf`：`$KYLIN_HOME/write_hadoop_conf`, Hadoop configuration
of the build cluster. This directory will not be available when Read/Write
separation deployment is not configured.
+- `/write_hadoop_conf`：`$KYLIN_HOME/write_hadoop_conf`, Hadoop configuration
of the build cluster. This directory will not be available when Read/Write
separation deployment is not configured.
- file `catalog_info`: directory structure of install package.
- file `commit_SHA1`: git-commit version.
- file `hadoop_env`: hadoop environment information.
@@ -79,8 +77,7 @@ Diagnostic packages generated by scripts are stored under
`$KYLIN_HOME/diag_dump
- `/client`: operating system resources occupied information, Hadoop version
and Kerberos information.
- `/bin`: program execute and manager binary files.
- `/monitor_metrics`: System monitoring statistics.
-- `/rec_candidate`: optimization suggestions for the model.
-- `/write_ hadoop_ conf`：`$KYLIN_HOME/write_hadoop_conf`, Hadoop configuration
of the build cluster. This directory will not be available when Read/Write
separation deployment is not configured.
+- `/write_hadoop_conf`：`$KYLIN_HOME/write_hadoop_conf`, Hadoop configuration
of the build cluster. This directory will not be available when Read/Write
separation deployment is not configured.
- file `catalog_info`: directory structure of install package.
- file `commit_SHA1`: git-commit version.
- file `hadoop_env`: hadoop environment information.
diff --git
a/website/docs/operations/system-operation/cli_tool/environment_dependency_check.md
b/website/docs/operations/system-operation/cli_tool/environment_dependency_check.md
index c6bf7a05a9..5d204c979e 100644
---
a/website/docs/operations/system-operation/cli_tool/environment_dependency_check.md
+++
b/website/docs/operations/system-operation/cli_tool/environment_dependency_check.md
@@ -43,6 +43,5 @@ The following table describes what will be checked in the
tool.
| Metastore | Check if the metastore is accessible and current user can
perform necessary operations on metadata. |
| InfluxDB | 1. Check if InfluxDB is accessible <br /> 2.
Check if current user has read and write privileges |
| ZooKeeper | Check if the service discovery is available. |
-| Clickhouse| Checking Clickhouse cluster health. |
| KylinConfig | Checking kylin config, must starts with kylin / spring /
server. |
| Query history | Check whether the current user has permissions of reading
and writing on the `query_history` and `query_history_realization` tables in
the RDBMS database|
diff --git
a/website/docs/operations/system-operation/cli_tool/images/accept_en.png
b/website/docs/operations/system-operation/cli_tool/images/accept_en.png
deleted file mode 100644
index f79cbb094e..0000000000
Binary files
a/website/docs/operations/system-operation/cli_tool/images/accept_en.png and
/dev/null differ
diff --git
a/website/docs/operations/system-operation/cli_tool/images/button_en.png
b/website/docs/operations/system-operation/cli_tool/images/button_en.png
deleted file mode 100644
index cfdcb1966d..0000000000
Binary files
a/website/docs/operations/system-operation/cli_tool/images/button_en.png and
/dev/null differ
diff --git
a/website/docs/operations/system-operation/cli_tool/images/rollback_en.png
b/website/docs/operations/system-operation/cli_tool/images/rollback_en.png
deleted file mode 100644
index c3f5742818..0000000000
Binary files
a/website/docs/operations/system-operation/cli_tool/images/rollback_en.png and
/dev/null differ
diff --git
a/website/docs/operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore.md
b/website/docs/operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore.md
index 2669da34e5..0d7b6f9bb6 100644
---
a/website/docs/operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore.md
+++
b/website/docs/operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore.md
@@ -85,4 +85,4 @@ Metadata recovery is required in Kylin with the **command
line**.
- `METADATA_BACKUP_PATH` - required, represents the metadata path that are
going to be recovered, the default value is `${KYLIN_HOME}/meta_backups/
- `--after-truncate` - optional, if this parameter is added, the project
metadata will be completely restored, otherwise only the deleted and modified
metadata will be restored, and the new metadata will still be retained.
-**Caution** When run this command, Kylin will enter maintenance mode. If the
command is interrupted by force, you may need to exit maintain mode manually.
Refer [maintenance_mode](../../maintenance_mode.en.md).
+**Caution** When run this command, Kylin will enter maintenance mode. If the
command is interrupted by force, you may need to exit maintain mode manually.
Refer [maintenance_mode](../../maintenance_mode.md).
diff --git a/website/docs/operations/system-operation/cli_tool/rollback.md
b/website/docs/operations/system-operation/cli_tool/rollback.md
index 9d55a8b9b9..a5e1898852 100644
--- a/website/docs/operations/system-operation/cli_tool/rollback.md
+++ b/website/docs/operations/system-operation/cli_tool/rollback.md
@@ -42,27 +42,6 @@ When a user error causes metadata or data loss, or when
Kylin is unavailable due
- `-t, --time <arg>`: Historical time point to roll back [required]. `<arg>`
is the specific time to roll back to, the format is ` yyyy-MM-dd HH: mm: ss`.
The available value is the time point from the earliest backup version time to
date.
- `--skip-check-data`: Skip checking whether the resource file is available
[optional].
-### Use Case {#user_case}
-
-Below is a common scenario of using the rollback tool in the operational
process, describing how the tool can be used to roll back and downgrade issues
in the event of product agnostic issues.
-
-- The `rollback_example` project has recommendation with recommendation mode
turned on. If the user accepts the recommendations under the model at 16:06:05
on September 1, 2020, then the original model of this project and the metadata
related to the index will be updated.
- 
-- At this point, it can be seen that the index recommended by the system has
been generated, but at this point there is an unknown problem, which results in
the index not being built properly and affects the health of the entire
cluster, and the product is also in an inoperable state.
-- To guarantee production availability, the rollback tool can be used to roll
back the metadata and data for this item to 15:20 on September 1.
-
-```sh
-$KYLIN_HOME/bin/rollback.sh --project rollback_example --time '2020-09-01
15:20:19'
-```
-
-- It can be seen that the item has returned to the state before the accident,
the index recommended by the system has disappeared, and the cluster has
returned to a stable state..
-
-
-
-- After the rollback is complete, a downgrade is also required for issues that
occur. In this example, you can click **Setting**, temporarily turn off the
smart recommendation feature under the **Basic Settings** tab first and
communicate with the Kylin technical support team in a timely manner.
- 
-- After the product problem is fixed, you can turn on the recommendation mode
again.
-
### Difference from Metadata Backup and Rollback Tool{#difference}
Kylin now offers metadata backup and restore tools that go some way to
protecting metadata from loss. However, the tool has some limitations.
@@ -75,7 +54,7 @@ Kylin now offers metadata backup and restore tools that go
some way to protectin
The premise of using the rollback tool is that the resource data (cube data,
dictionary data, snapshot data, etc.) must be guaranteed not to be deleted
within the rollback time range. The retention period of the resource data
involves two configurations
-* `kylin.storage.time-machine-enabled` After this configuration is enabled,
the resources in the retention period will not be deleted in the KE service.
After being enabled, the snapshot data retention time will be the same as the
time configured in `kylin.storage.resource-survival-time-threshold`, the
default value is False.
+* `kylin.storage.time-machine-enabled` After this configuration is enabled,
the resources in the retention period will not be deleted in the Kylin service.
After being enabled, the snapshot data retention time will be the same as the
time configured in `kylin.storage.resource-survival-time-threshold`, the
default value is False.
* `kylin.storage.resource-survival-time-threshold` Resource data retention
time, the default value is `7d`, unit description:` d` (day), `h` (hour),` m`
(minute).
### Caution and Common Errors {#caution_and_common_errors}
@@ -84,9 +63,9 @@ The following are some errors and points of attention that
may be encountered du
**Points to take attention**
-- Using the rollback tool will roll back the state of the task execution to
the state of the historical moment, and will restart the execution after the KE
service is started.
+- Using the rollback tool will roll back the state of the task execution to
the state of the historical moment, and will restart the execution after the
Kylin service is started.
- After the rollback tool configuration is turned on, more garbage files may
be saved and more storage space may be token up. Using the garbage cleaning
tool during the retention period cannot clean up the expired resource data
during the retention period.
-- During the execution of the tool, if it is run multiple times, each run will
keep a backup of the current metadata in the `{working-dir}/_ current_backup`
directory, the file names are distinguished by time.
+- During the execution of the tool, if it is run multiple times, each run will
keep a backup of the current metadata in the `{working-dir}/_current_backup`
directory, the file names are distinguished by time.
- The time specified by the user cannot be greater than the current time.
- All service nodes must be shut down before using the tool, otherwise it will
cause data inconsistency.
- If the user manually deletes the dictionary data of the project and then
regenerates the dictionary data again, using the rollback tool will cause the
dictionary data and the index data to be inconsistent.
@@ -96,7 +75,7 @@ The following are some errors and points of attention that
may be encountered du
**Possible error results**
-- Using the rollback tool reverts the state of the task execution back to the
historical moment, and the execution is triggered again when the KE is started.
+- Using the rollback tool reverts the state of the task execution back to the
historical moment, and the execution is triggered again when the Kylin is
started.
- Turning on the time machine causes more junk files to be saved, taking up
more storage space, and using the junk cleanup tool during the retention period
does not clean up resource data that has expired during the retention period.
- During tool execution, if there are multiple runs, each run keeps a backup
of the current metadata in the `{working-dir}/_current_backup` directory,
distinguishing the file name by time.
- The time specified by the user cannot be greater than the current time.
@@ -108,7 +87,7 @@ The following are some errors and points of attention that
may be encountered du
- `dectect port available failed` -> Failure to detect user ports requires
shutting down the service nodes of the cluster.
- `check storage data available failed` -> Failed to detect resource file,
user can use `--skip-check-data` parameter to force rollback。
- `restore current metadata failed, please restore the metadata database
manually` -> The metadata rollback fails, and overwriting with the current
backup also fails. Manual intervention is required to solve the problem. This
situation must be handled carefully to avoid loss of metadata.
-- The rollback scope of the rollback tool does not include historical
recommendations and projects manually deleted by the user
+- The rollback scope of the rollback tool does not include historical
recommendations and projects manually deleted by the user.
### Appendix {#appendix}
diff --git a/website/docs/operations/system-operation/diagnosis/diagnosis.md
b/website/docs/operations/system-operation/diagnosis/diagnosis.md
index 63954cccfd..9df8c7cd42 100644
--- a/website/docs/operations/system-operation/diagnosis/diagnosis.md
+++ b/website/docs/operations/system-operation/diagnosis/diagnosis.md
@@ -17,9 +17,6 @@ last_update:
date: 08/12/2022
---
-
-## System, Job and Query Diagnosis
-
Kylin users may face with many problems during usage, such as job failure,
SQL query failure, SQL query overtime, etc. In order to help solve these
problems efficiently, Kylin provides a *Diagnosis* function on Web UI to pack
related information into a zip package to help operation staff and Kylin
Community to analyze the root cause.
Diagnosis function includes System, Job and Query Diagnosis. In addition to
the method of generating diagnostic package on web UI described in this
chapter, you can also generate the diagnostic package through the bash script.
For details, please refer to [Diagnosis Tool](../cli_tool/diagnosis.md).
@@ -29,7 +26,7 @@ Diagnosis function includes System, Job and Query Diagnosis.
In addition to the
System diagnostic package includes all diagnostic information of a Kylin
instance, and users can generate system diagnostic package on Web UI following
the following steps:
-
+
1. Enter the **Admin** page and click the **Diagnosis** button in the lower
left corner.
@@ -49,7 +46,9 @@ System diagnostic package includes all diagnostic information
of a Kylin instanc
Job diagnostic package includes all diagnostic information of a specific job,
and users can generate job diagnostic package on Web UI by following the
following steps:
-
+
+
+
1. After logging in to Kylin, click **Monitor** in the left navigation bar. In
the action bar of a job on the **Jobs List** page, click the **Download Job
Diagnostic Package** button.
2. Select **Server**.
@@ -59,7 +58,9 @@ Job diagnostic package includes all diagnostic information of
a specific job, an
Query diagnosis package includes all diagnostic information of a specific
query, and users can generate query diagnosis package on Web UI by following
the following steps:
-
+
+
+
1. After logging in to Kylin, click **Query** in the left navigation bar. In
the action bar of a query on the **History** page, click the **Download Query
Diagnostic Package** button in **Action**.
2. Click **Generate and Download** button: After the diagnosis package is
generated, the downloads will be triggered automatically. If the diagnosis
package generation fails, you can view the details of the failure on the
interface.
diff --git
a/website/docs/operations/system-operation/diagnosis/images/diagnosis_web.png
b/website/docs/operations/system-operation/diagnosis/images/diagnosis_web.png
new file mode 100644
index 0000000000..3e5318e5bd
Binary files /dev/null and
b/website/docs/operations/system-operation/diagnosis/images/diagnosis_web.png
differ
diff --git
a/website/docs/operations/system-operation/diagnosis/images/diagnosis_web_en.png
b/website/docs/operations/system-operation/diagnosis/images/diagnosis_web_en.png
deleted file mode 100644
index 329ee9c778..0000000000
Binary files
a/website/docs/operations/system-operation/diagnosis/images/diagnosis_web_en.png
and /dev/null differ
diff --git
a/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_01.png
b/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_01.png
new file mode 100644
index 0000000000..3b1e878a01
Binary files /dev/null and
b/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_01.png
differ
diff --git
a/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_02.png
b/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_02.png
new file mode 100644
index 0000000000..e6812f431c
Binary files /dev/null and
b/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_02.png
differ
diff --git
a/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_en.png
b/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_en.png
deleted file mode 100644
index 5a6f0457e2..0000000000
Binary files
a/website/docs/operations/system-operation/diagnosis/images/job_diagnosis_web_en.png
and /dev/null differ
diff --git
a/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web.png
b/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web.png
deleted file mode 100644
index 9c84ffd914..0000000000
Binary files
a/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web.png
and /dev/null differ
diff --git
a/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web_01.png
b/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web_01.png
new file mode 100644
index 0000000000..59e9dd3cff
Binary files /dev/null and
b/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web_01.png
differ
diff --git
a/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web_02.png
b/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web_02.png
new file mode 100644
index 0000000000..117d131e43
Binary files /dev/null and
b/website/docs/operations/system-operation/diagnosis/images/query_diagnosis_web_02.png
differ
diff --git
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_during_maintenance_mode.png
b/website/docs/operations/system-operation/images/forbidden_modify_metadata_during_maintenance_mode.png
index b5fdfd120d..373277d910 100644
Binary files
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_during_maintenance_mode.png
and
b/website/docs/operations/system-operation/images/forbidden_modify_metadata_during_maintenance_mode.png
differ
diff --git
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_promoting_during_maintenance_mode.png
b/website/docs/operations/system-operation/images/forbidden_modify_metadata_promoting_during_maintenance_mode.png
index 6f5c7afdd2..5adfae28a0 100644
Binary files
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_promoting_during_maintenance_mode.png
and
b/website/docs/operations/system-operation/images/forbidden_modify_metadata_promoting_during_maintenance_mode.png
differ
diff --git
a/website/docs/operations/system-operation/images/maintenance_mode.png
b/website/docs/operations/system-operation/images/maintenance_mode.png
index 2becffa013..eb94f1d29f 100644
Binary files
a/website/docs/operations/system-operation/images/maintenance_mode.png and
b/website/docs/operations/system-operation/images/maintenance_mode.png differ
diff --git
a/website/docs/operations/system-operation/images/system_recover_from_maintenance_mode.png
b/website/docs/operations/system-operation/images/system_recover_from_maintenance_mode.png
index 23b1050322..0007e33fef 100644
Binary files
a/website/docs/operations/system-operation/images/system_recover_from_maintenance_mode.png
and
b/website/docs/operations/system-operation/images/system_recover_from_maintenance_mode.png
differ
diff --git a/website/docs/operations/system-operation/junk_file_clean.md
b/website/docs/operations/system-operation/junk_file_clean.md
index f25653e8b5..cc517119ba 100644
--- a/website/docs/operations/system-operation/junk_file_clean.md
+++ b/website/docs/operations/system-operation/junk_file_clean.md
@@ -14,9 +14,6 @@ last_update:
date: 08/12/2022
---
-
-## Junk File Cleanup
-
After Kylin runs for a period of time, the system may generate a certain
number of junk files, which may occupy a large amount of storage space. At this
time, junk cleaning is required.
Junk file cleaning can improve the stability and performance of the Kylin
system. Effective junk file cleaning can not only save storage space, but also
ensure the ecological health of the cluster where Kylin is located.
@@ -82,7 +79,7 @@ This command supports standard short and long parameters. The
parameter descript
**Note**: Whether this command executes metadata junk file cleaning through -m
or data junk file cleaning through -c, Kylin will enter maintenance mode. If
junk file cleaning is forcibly interrupted, you need to manually exit
maintenance mode. Refer to [ Maintenance Mode](maintenance_mode.md).
-In addition, from Kylin 4.5.6, the new command line tool `FastRoutineTool`
+In addition, from Kylin 5, the new command line tool `FastRoutineTool`
````sh
$KYLIN_HOME/bin/kylin.sh org.apache.kylin.tool.routine.FastRoutineTool
````
diff --git a/website/docs/quickstart/quick_start.md
b/website/docs/quickstart/quick_start.md
index 61bdc96bf8..36723d664c 100644
--- a/website/docs/quickstart/quick_start.md
+++ b/website/docs/quickstart/quick_start.md
@@ -58,6 +58,9 @@ Before proceeding, please make sure the
[Prerequisite](../deployment/on-premises
Kylin uses InfluxDB to save various system monitoring information. If you
do not need to view related information, you can skip this step. It is strongly
recommended to complete this step in a production environment and use related
monitoring functions.
```sh
+ # download influxdb
+ $KYLIN_HOME/sbin/download-influxdb.sh
+
cd $KYLIN_HOME/influxdb
# install influxdb
diff --git a/website/sidebars.js b/website/sidebars.js
index c0e380101d..20d7ebea2a 100644
--- a/website/sidebars.js
+++ b/website/sidebars.js
@@ -294,10 +294,6 @@ const sidebars = {
type: 'doc',
id:
'operations/access-control/data-access-control/project_acl'
},
- {
- type: 'doc',
- id:
'operations/access-control/data-access-control/acl_table'
- },
],
},
],
