This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new d1cda1a16d5 [fix] tiered storage for 2.1 and 3.0 (#1596) d1cda1a16d5 is described below commit d1cda1a16d542565324d972ab6bc4e39dce8f3ea Author: Yongqiang YANG <yangyongqi...@selectdb.com> AuthorDate: Wed Dec 25 19:15:58 2024 +0800 [fix] tiered storage for 2.1 and 3.0 (#1596) ## Versions - [x] dev - [x] 3.0 - [x] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --------- Co-authored-by: Yongqiang YANG <yangyogqi...@selectdb.com> --- .../data-partitioning/dynamic-partitioning.md | 2 +- ...-disk-medium-migration.md => tiered-ssd-hdd.md} | 2 +- .../data-partitioning/dynamic-partitioning.md | 2 +- ...-disk-medium-migration.md => tiered-ssd-hdd.md} | 0 .../data-partitioning/dynamic-partitioning.md | 2 +- .../tiered-storage/diff-disk-medium-migration.md | 61 --------------------- .../table-design/tiered-storage/tiered-ssd-hdd.md} | 0 .../data-partitioning/dynamic-partitioning.md | 2 +- .../tiered-storage/diff-disk-medium-migration.md | 61 --------------------- .../table-design/tiered-storage/tiered-ssd-hdd.md} | 0 sidebars.json | 2 +- .../data-partitioning/dynamic-partitioning.md | 2 +- .../tiered-storage/diff-disk-medium-migration.md | 62 ---------------------- .../table-design/tiered-storage/tiered-ssd-hdd.md | 2 +- .../data-partitioning/dynamic-partitioning.md | 2 +- .../tiered-storage/diff-disk-medium-migration.md | 62 ---------------------- .../table-design/tiered-storage/tiered-ssd-hdd.md | 2 +- versioned_sidebars/version-2.1-sidebars.json | 2 +- versioned_sidebars/version-3.0-sidebars.json | 2 +- 19 files changed, 12 insertions(+), 258 deletions(-) diff --git a/docs/table-design/data-partitioning/dynamic-partitioning.md b/docs/table-design/data-partitioning/dynamic-partitioning.md index 3093f209070..f96f74bcf0b 100644 --- a/docs/table-design/data-partitioning/dynamic-partitioning.md +++ b/docs/table-design/data-partitioning/dynamic-partitioning.md @@ -128,7 +128,7 @@ Note that if the user sets history_partition_num (> 0), the starting partition f When `time_unit` is` MONTH`, this parameter is used to specify the start date of each month. The value ranges from 1 to 28. 1 means the 1st of every month, and 28 means the 28th of every month. The default is 1, which means that every month starts at 1st. The 29, 30 and 31 are not supported at the moment to avoid ambiguity caused by lunar years or months. -- Doris supports multi-level storage with SSD and HDD tiers. For more details, please refer to [tiered-storage](../tiered-storage/diff-disk-medium-migration.md) +- Doris supports multi-level storage with SSD and HDD tiers. For more details, please refer to [tiered-storage](../tiered-storage/tiered-ssd-hdd.md) - `dynamic_partition.create_history_partition` diff --git a/docs/table-design/tiered-storage/diff-disk-medium-migration.md b/docs/table-design/tiered-storage/tiered-ssd-hdd.md similarity index 99% copy from docs/table-design/tiered-storage/diff-disk-medium-migration.md copy to docs/table-design/tiered-storage/tiered-ssd-hdd.md index 64f2ed1b896..9be01f6659a 100644 --- a/docs/table-design/tiered-storage/diff-disk-medium-migration.md +++ b/docs/table-design/tiered-storage/tiered-ssd-hdd.md @@ -1,7 +1,7 @@ --- { "title": "Tiered Storage of SSD and HDD", - "language": "en-US" + "language": "en" } --- diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/dynamic-partitioning.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/dynamic-partitioning.md index cfa08984817..97d117978e0 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/dynamic-partitioning.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/data-partitioning/dynamic-partitioning.md @@ -125,7 +125,7 @@ under the License. 当 `time_unit` 为 `MONTH` 时,该参数用于指定每月的起始日期。取值为 1 到 28。其中 1 表示每月 1 号,28 表示每月 28 号。默认为 1,即表示每月以 1 号为起始点。暂不支持以 29、30、31 号为起始日,以避免因闰年或闰月带来的歧义。 -- doris支持SSD和HDD层级存储,可参考[分层存储](./tiered-storage/diff-disk-medium-migration.md) +- doris支持SSD和HDD层级存储,可参考[分层存储](./tiered-storage/tiered-ssd-hdd.md) - `dynamic_partition.create_history_partition` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/diff-disk-medium-migration.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/tiered-ssd-hdd.md similarity index 100% copy from i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/diff-disk-medium-migration.md copy to i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/tiered-ssd-hdd.md diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md index 37f86d2c235..0fd472ed234 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md @@ -125,7 +125,7 @@ under the License. 当 `time_unit` 为 `MONTH` 时,该参数用于指定每月的起始日期。取值为 1 到 28。其中 1 表示每月 1 号,28 表示每月 28 号。默认为 1,即表示每月以 1 号为起始点。暂不支持以 29、30、31 号为起始日,以避免因闰年或闰月带来的歧义。 -- doris支持SSD和HDD层级存储,可参考[分层存储](./tiered-storage/diff-disk-medium-migration.md) +- doris支持SSD和HDD层级存储,可参考[分层存储](./tiered-storage/tiered-ssd-hdd.md) - `dynamic_partition.create_history_partition` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/tiered-storage/diff-disk-medium-migration.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/tiered-storage/diff-disk-medium-migration.md deleted file mode 100644 index 0b9472e2ebf..00000000000 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/tiered-storage/diff-disk-medium-migration.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -{ - "title": "SSD 和 HDD 层级存储", - "language": "zh-CN" -} ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one -or more contributor license agreements. See the NOTICE file -distributed with this work for additional information -regarding copyright ownership. The ASF licenses this file -to you under the Apache License, Version 2.0 (the -"License"); you may not use this file except in compliance -with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, -software distributed under the License is distributed on an -"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -KIND, either express or implied. See the License for the -specific language governing permissions and limitations -under the License. ---> - -可以配置动态分区参数,在不同的磁盘类型上创建动态分区,Doris 会根据配置参数将冷数据从SSD迁移到HDD。这样的做法在降低成本的同时,也提升了Doris的读写性能。 - -动态分区参数可以参考[数据划分-动态分区](../../table-design/data-partitioning/dynamic-partitioning) - -`dynamic_partition.hot_partition_num` - -:::caution - 注意,dynamic_partition.storage_medium 必须设置为HDD,否则 hot_partition_num 将不会生效 -::: - - 指定最新的多少个分区为热分区。对于热分区,系统会自动设置其 `storage_medium` 参数为 SSD,并且设置 `storage_cooldown_time`。 - - 注意:若存储路径下没有 SSD 磁盘路径,配置该参数会导致动态分区创建失败。 - - `hot_partition_num` 表示当前时间所在分区及之前的 hot_partition_num - 1 个分区,以及所有未来的分区,将被存储在 SSD 介质上。 - - 我们举例说明。假设今天是 2021-05-20,按天分区,动态分区的属性设置为:hot_partition_num=2, end=3, start=-3。则系统会自动创建以下分区,并且设置 `storage_medium` 和 `storage_cooldown_time` 参数: - - ```Plain - p20210517:["2021-05-17", "2021-05-18") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210518:["2021-05-18", "2021-05-19") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210519:["2021-05-19", "2021-05-20") storage_medium=SSD storage_cooldown_time=2021-05-21 00:00:00 - p20210520:["2021-05-20", "2021-05-21") storage_medium=SSD storage_cooldown_time=2021-05-22 00:00:00 - p20210521:["2021-05-21", "2021-05-22") storage_medium=SSD storage_cooldown_time=2021-05-23 00:00:00 - p20210522:["2021-05-22", "2021-05-23") storage_medium=SSD storage_cooldown_time=2021-05-24 00:00:00 - p20210523:["2021-05-23", "2021-05-24") storage_medium=SSD storage_cooldown_time=2021-05-25 00:00:00 - ``` - -- `dynamic_partition.storage_medium` - -指定创建的动态分区的默认存储介质。默认是 HDD,可选择 SSD。 - -:::caution - 注意,当设置为 SSD 时,`hot_partition_num` 属性将不再生效,所有分区将默认为 SSD 存储介质并且冷却时间为 9999-12-31 23:59:59。 -::: \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/diff-disk-medium-migration.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/tiered-storage/tiered-ssd-hdd.md similarity index 100% copy from i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/diff-disk-medium-migration.md copy to i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/table-design/tiered-storage/tiered-ssd-hdd.md diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md index fea724d8f00..f35ac5ccd43 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md @@ -126,7 +126,7 @@ under the License. 当 `time_unit` 为 `MONTH` 时,该参数用于指定每月的起始日期。取值为 1 到 28。其中 1 表示每月 1 号,28 表示每月 28 号。默认为 1,即表示每月以 1 号为起始点。暂不支持以 29、30、31 号为起始日,以避免因闰年或闰月带来的歧义。 -- doris支持SSD和HDD层级存储,可参考[分层存储](./tiered-storage/diff-disk-medium-migration.md) +- doris支持SSD和HDD层级存储,可参考[分层存储](./tiered-storage/tiered-ssd-hdd.md) - `dynamic_partition.create_history_partition` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/tiered-storage/diff-disk-medium-migration.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/tiered-storage/diff-disk-medium-migration.md deleted file mode 100644 index 0b9472e2ebf..00000000000 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/tiered-storage/diff-disk-medium-migration.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -{ - "title": "SSD 和 HDD 层级存储", - "language": "zh-CN" -} ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one -or more contributor license agreements. See the NOTICE file -distributed with this work for additional information -regarding copyright ownership. The ASF licenses this file -to you under the Apache License, Version 2.0 (the -"License"); you may not use this file except in compliance -with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, -software distributed under the License is distributed on an -"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -KIND, either express or implied. See the License for the -specific language governing permissions and limitations -under the License. ---> - -可以配置动态分区参数,在不同的磁盘类型上创建动态分区,Doris 会根据配置参数将冷数据从SSD迁移到HDD。这样的做法在降低成本的同时,也提升了Doris的读写性能。 - -动态分区参数可以参考[数据划分-动态分区](../../table-design/data-partitioning/dynamic-partitioning) - -`dynamic_partition.hot_partition_num` - -:::caution - 注意,dynamic_partition.storage_medium 必须设置为HDD,否则 hot_partition_num 将不会生效 -::: - - 指定最新的多少个分区为热分区。对于热分区,系统会自动设置其 `storage_medium` 参数为 SSD,并且设置 `storage_cooldown_time`。 - - 注意:若存储路径下没有 SSD 磁盘路径,配置该参数会导致动态分区创建失败。 - - `hot_partition_num` 表示当前时间所在分区及之前的 hot_partition_num - 1 个分区,以及所有未来的分区,将被存储在 SSD 介质上。 - - 我们举例说明。假设今天是 2021-05-20,按天分区,动态分区的属性设置为:hot_partition_num=2, end=3, start=-3。则系统会自动创建以下分区,并且设置 `storage_medium` 和 `storage_cooldown_time` 参数: - - ```Plain - p20210517:["2021-05-17", "2021-05-18") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210518:["2021-05-18", "2021-05-19") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210519:["2021-05-19", "2021-05-20") storage_medium=SSD storage_cooldown_time=2021-05-21 00:00:00 - p20210520:["2021-05-20", "2021-05-21") storage_medium=SSD storage_cooldown_time=2021-05-22 00:00:00 - p20210521:["2021-05-21", "2021-05-22") storage_medium=SSD storage_cooldown_time=2021-05-23 00:00:00 - p20210522:["2021-05-22", "2021-05-23") storage_medium=SSD storage_cooldown_time=2021-05-24 00:00:00 - p20210523:["2021-05-23", "2021-05-24") storage_medium=SSD storage_cooldown_time=2021-05-25 00:00:00 - ``` - -- `dynamic_partition.storage_medium` - -指定创建的动态分区的默认存储介质。默认是 HDD,可选择 SSD。 - -:::caution - 注意,当设置为 SSD 时,`hot_partition_num` 属性将不再生效,所有分区将默认为 SSD 存储介质并且冷却时间为 9999-12-31 23:59:59。 -::: \ No newline at end of file diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/diff-disk-medium-migration.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/tiered-storage/tiered-ssd-hdd.md similarity index 100% rename from i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/tiered-storage/diff-disk-medium-migration.md rename to i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/table-design/tiered-storage/tiered-ssd-hdd.md diff --git a/sidebars.json b/sidebars.json index 55cfd7f0701..c9a76083277 100644 --- a/sidebars.json +++ b/sidebars.json @@ -145,7 +145,7 @@ "label": "Tiered Storage", "items": [ "table-design/tiered-storage/overview", - "table-design/tiered-storage/diff-disk-medium-migration", + "table-design/tiered-storage/tiered-ssd-hdd", "table-design/tiered-storage/remote-storage" ] }, diff --git a/versioned_docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md b/versioned_docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md index b9f9382631f..562ae7a2d51 100644 --- a/versioned_docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md +++ b/versioned_docs/version-2.1/table-design/data-partitioning/dynamic-partitioning.md @@ -129,7 +129,7 @@ Note that if the user sets history_partition_num (> 0), the starting partition f When `time_unit` is` MONTH`, this parameter is used to specify the start date of each month. The value ranges from 1 to 28. 1 means the 1st of every month, and 28 means the 28th of every month. The default is 1, which means that every month starts at 1st. The 29, 30 and 31 are not supported at the moment to avoid ambiguity caused by lunar years or months. -- Doris supports multi-level storage with SSD and HDD tiers. For more details, please refer to [tiered-storage](../tiered-storage/diff-disk-medium-migration.md) +- Doris supports multi-level storage with SSD and HDD tiers. For more details, please refer to [tiered-storage](../tiered-storage/tiered-ssd-hdd.md) - `dynamic_partition.create_history_partition` diff --git a/versioned_docs/version-2.1/table-design/tiered-storage/diff-disk-medium-migration.md b/versioned_docs/version-2.1/table-design/tiered-storage/diff-disk-medium-migration.md deleted file mode 100644 index 26fea4f2301..00000000000 --- a/versioned_docs/version-2.1/table-design/tiered-storage/diff-disk-medium-migration.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -{ -"title": "SSD and HDD tiered storage", -"language": "en" -} ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one -or more contributor license agreements. See the NOTICE file -distributed with this work for additional information -regarding copyright ownership. The ASF licenses this file -to you under the Apache License, Version 2.0 (the -"License"); you may not use this file except in compliance -with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, -software distributed under the License is distributed on an -"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -KIND, either express or implied. See the License for the -specific language governing permissions and limitations -under the License. ---> - -You can set parameters for dynamic partitions across different disk types, facilitating data migration from SSDs to HDDs based on the parameters. This strategy improves read and write performance in Doris while lowering costs. - -By configuring `dynamic_partition.hot_partition_num` and `dynamic_partition.storage_medium`, you can use SSD and HDD tiered storage. For specific usage, please refer to [Data Partitioning - Dynamic Partitioning](../../table-design/data-partitioning/dynamic-partitioning). - -*`dynamic_partition.hot_partition_num`* - - :::tip - - If the storage path does not include an SSD disk path, configuring this parameter will result in the failure of dynamic partition creation. - - ::: - - `hot_partition_num` indicates that the current partition and the previous hot_partition_num - 1 partitions, along with all future partitions, will be stored on SSD media. - - Let us give an example. Suppose today is 2021-05-20, partition by day, and the properties of dynamic partition are set to: hot_partition_num=2, end=3, start=-3. Then the system will automatically create the following partitions, and set the `storage_medium` and `storage_cooldown_time` properties: - - ```sql - p20210517: ["2021-05-17", "2021-05-18") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210518: ["2021-05-18", "2021-05-19") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210519: ["2021-05-19", "2021-05-20") storage_medium=SSD storage_cooldown_time=2021-05-21 00:00:00 - p20210520: ["2021-05-20", "2021-05-21") storage_medium=SSD storage_cooldown_time=2021-05-22 00:00:00 - p20210521: ["2021-05-21", "2021-05-22") storage_medium=SSD storage_cooldown_time=2021-05-23 00:00:00 - p20210522: ["2021-05-22", "2021-05-23") storage_medium=SSD storage_cooldown_time=2021-05-24 00:00:00 - p20210523: ["2021-05-23", "2021-05-24") storage_medium=SSD storage_cooldown_time=2021-05-25 00:00:00 - ``` - -*`dynamic_partition.storage_medium`* - - - :::info Note - This parameteres is supported since Doris version 1.2.3 - ::: - - Specifies the final storage medium for the newly created dynamic partition. HDD is the default, but SSD can be selected. - - Note that when set to SSD, the `hot_partition_num` property will no longer take effect, all partitions will default to SSD storage media and the cooldown time will be 9999-12-31 23:59:59. \ No newline at end of file diff --git a/docs/table-design/tiered-storage/diff-disk-medium-migration.md b/versioned_docs/version-2.1/table-design/tiered-storage/tiered-ssd-hdd.md similarity index 99% copy from docs/table-design/tiered-storage/diff-disk-medium-migration.md copy to versioned_docs/version-2.1/table-design/tiered-storage/tiered-ssd-hdd.md index 64f2ed1b896..9be01f6659a 100644 --- a/docs/table-design/tiered-storage/diff-disk-medium-migration.md +++ b/versioned_docs/version-2.1/table-design/tiered-storage/tiered-ssd-hdd.md @@ -1,7 +1,7 @@ --- { "title": "Tiered Storage of SSD and HDD", - "language": "en-US" + "language": "en" } --- diff --git a/versioned_docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md b/versioned_docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md index 1afc9246547..aa9d01b9b8e 100644 --- a/versioned_docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md +++ b/versioned_docs/version-3.0/table-design/data-partitioning/dynamic-partitioning.md @@ -128,7 +128,7 @@ Note that if the user sets history_partition_num (> 0), the starting partition f When `time_unit` is` MONTH`, this parameter is used to specify the start date of each month. The value ranges from 1 to 28. 1 means the 1st of every month, and 28 means the 28th of every month. The default is 1, which means that every month starts at 1st. The 29, 30 and 31 are not supported at the moment to avoid ambiguity caused by lunar years or months. -- Doris supports multi-level storage with SSD and HDD tiers. For more details, please refer to [tiered-storage](../tiered-storage/diff-disk-medium-migration.md) +- Doris supports multi-level storage with SSD and HDD tiers. For more details, please refer to [tiered-storage](../tiered-storage/tiered-ssd-hdd.md) - `dynamic_partition.create_history_partition` diff --git a/versioned_docs/version-3.0/table-design/tiered-storage/diff-disk-medium-migration.md b/versioned_docs/version-3.0/table-design/tiered-storage/diff-disk-medium-migration.md deleted file mode 100644 index 26fea4f2301..00000000000 --- a/versioned_docs/version-3.0/table-design/tiered-storage/diff-disk-medium-migration.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -{ -"title": "SSD and HDD tiered storage", -"language": "en" -} ---- - -<!-- -Licensed to the Apache Software Foundation (ASF) under one -or more contributor license agreements. See the NOTICE file -distributed with this work for additional information -regarding copyright ownership. The ASF licenses this file -to you under the Apache License, Version 2.0 (the -"License"); you may not use this file except in compliance -with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, -software distributed under the License is distributed on an -"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -KIND, either express or implied. See the License for the -specific language governing permissions and limitations -under the License. ---> - -You can set parameters for dynamic partitions across different disk types, facilitating data migration from SSDs to HDDs based on the parameters. This strategy improves read and write performance in Doris while lowering costs. - -By configuring `dynamic_partition.hot_partition_num` and `dynamic_partition.storage_medium`, you can use SSD and HDD tiered storage. For specific usage, please refer to [Data Partitioning - Dynamic Partitioning](../../table-design/data-partitioning/dynamic-partitioning). - -*`dynamic_partition.hot_partition_num`* - - :::tip - - If the storage path does not include an SSD disk path, configuring this parameter will result in the failure of dynamic partition creation. - - ::: - - `hot_partition_num` indicates that the current partition and the previous hot_partition_num - 1 partitions, along with all future partitions, will be stored on SSD media. - - Let us give an example. Suppose today is 2021-05-20, partition by day, and the properties of dynamic partition are set to: hot_partition_num=2, end=3, start=-3. Then the system will automatically create the following partitions, and set the `storage_medium` and `storage_cooldown_time` properties: - - ```sql - p20210517: ["2021-05-17", "2021-05-18") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210518: ["2021-05-18", "2021-05-19") storage_medium=HDD storage_cooldown_time=9999-12-31 23:59:59 - p20210519: ["2021-05-19", "2021-05-20") storage_medium=SSD storage_cooldown_time=2021-05-21 00:00:00 - p20210520: ["2021-05-20", "2021-05-21") storage_medium=SSD storage_cooldown_time=2021-05-22 00:00:00 - p20210521: ["2021-05-21", "2021-05-22") storage_medium=SSD storage_cooldown_time=2021-05-23 00:00:00 - p20210522: ["2021-05-22", "2021-05-23") storage_medium=SSD storage_cooldown_time=2021-05-24 00:00:00 - p20210523: ["2021-05-23", "2021-05-24") storage_medium=SSD storage_cooldown_time=2021-05-25 00:00:00 - ``` - -*`dynamic_partition.storage_medium`* - - - :::info Note - This parameteres is supported since Doris version 1.2.3 - ::: - - Specifies the final storage medium for the newly created dynamic partition. HDD is the default, but SSD can be selected. - - Note that when set to SSD, the `hot_partition_num` property will no longer take effect, all partitions will default to SSD storage media and the cooldown time will be 9999-12-31 23:59:59. \ No newline at end of file diff --git a/docs/table-design/tiered-storage/diff-disk-medium-migration.md b/versioned_docs/version-3.0/table-design/tiered-storage/tiered-ssd-hdd.md similarity index 99% rename from docs/table-design/tiered-storage/diff-disk-medium-migration.md rename to versioned_docs/version-3.0/table-design/tiered-storage/tiered-ssd-hdd.md index 64f2ed1b896..9be01f6659a 100644 --- a/docs/table-design/tiered-storage/diff-disk-medium-migration.md +++ b/versioned_docs/version-3.0/table-design/tiered-storage/tiered-ssd-hdd.md @@ -1,7 +1,7 @@ --- { "title": "Tiered Storage of SSD and HDD", - "language": "en-US" + "language": "en" } --- diff --git a/versioned_sidebars/version-2.1-sidebars.json b/versioned_sidebars/version-2.1-sidebars.json index 76a70837a3b..bd98ecce03d 100644 --- a/versioned_sidebars/version-2.1-sidebars.json +++ b/versioned_sidebars/version-2.1-sidebars.json @@ -99,7 +99,7 @@ "type": "category", "label": "Tiered Storage", "items": [ - "table-design/tiered-storage/diff-disk-medium-migration", + "table-design/tiered-storage/tiered-ssd-hdd", "table-design/tiered-storage/remote-storage" ] }, diff --git a/versioned_sidebars/version-3.0-sidebars.json b/versioned_sidebars/version-3.0-sidebars.json index b009755e39f..e34fa90e407 100644 --- a/versioned_sidebars/version-3.0-sidebars.json +++ b/versioned_sidebars/version-3.0-sidebars.json @@ -117,7 +117,7 @@ "type": "category", "label": "Tiered Storage", "items": [ - "table-design/tiered-storage/diff-disk-medium-migration", + "table-design/tiered-storage/tiered-ssd-hdd", "table-design/tiered-storage/remote-storage" ] }, --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org