This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new 7e63c6d1d42 (multicatalog) add docs for hive on tencent dlc (#887) 7e63c6d1d42 is described below commit 7e63c6d1d4286675d62c61db764d91feeabc8353 Author: Yulei-Yang <yulei.yang0...@gmail.com> AuthorDate: Tue Jul 23 09:50:33 2024 +0800 (multicatalog) add docs for hive on tencent dlc (#887) doris pr: https://github.com/apache/doris/pull/36823 --------- Co-authored-by: yuleiyang <yuleiy...@tencent.com> --- docs/lakehouse/datalake-analytics/hive.md | 24 +++++++++++++++++++++ .../current/lakehouse/datalake-analytics/hive.md | 24 +++++++++++++++++++++ .../version-2.0/lakehouse/datalake/hive.md | 24 +++++++++++++++++++++ .../lakehouse/datalake-analytics/hive.md | 24 +++++++++++++++++++++ .../version-2.0/lakehouse/datalake/hive.md | 25 ++++++++++++++++++++++ .../lakehouse/datalake-analytics/hive.md | 24 +++++++++++++++++++++ 6 files changed, 145 insertions(+) diff --git a/docs/lakehouse/datalake-analytics/hive.md b/docs/lakehouse/datalake-analytics/hive.md index 741c9bd373d..26dafdee022 100644 --- a/docs/lakehouse/datalake-analytics/hive.md +++ b/docs/lakehouse/datalake-analytics/hive.md @@ -108,6 +108,30 @@ CREATE CATALOG hive PROPERTIES ( ); ``` +### Doris query DLC + +:::note +Supported since Doris version 2.0.13 / 2.1.5 +::: + +[DLC](https://cloud.tencent.com/product/dlc) of Tencent Cloud use HMS to manage its metadata, so Hive catalog can be used。 +DLC stores data on LakeFS or COSN, below catalog is compatible with both file system. +```sql +CREATE CATALOG dlc PROPERTIES ( + 'type' = 'hms', + 'hive.metastore.uris' = 'thrift://<dlc_metastore_ip>:<dlc_metastore_port>', + 's3.access_key' = 'xxxxx', + 's3.secret_key' = 'xxxxx', + 's3.region' = 'ap-xxx', + 's3.endpoint' = 'cos.ap-xxx.myqcloud.com', + 'fs.cosn.bucket.region' = 'ap-xxx', + 'fs.ofs.user.appid' = '<your_tencent_cloud_appid>', + 'fs.ofs.tmp.cache.dir' = '<tmp_cache_dir>' +); +``` + +After create catalog, you need to [grant](https://cloud.tencent.com/document/product/436/38648) essential privilege to the appid your dlc catalog is using. + ### Hive On S3 ```sql diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/hive.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/hive.md index 9b7b9ebedde..c5e590d7611 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/hive.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/hive.md @@ -108,6 +108,30 @@ CREATE CATALOG hive PROPERTIES ( ); ``` +### Doris 访问腾讯云 DLC + +:::note +Doris 2.0.13 / 2.1.5 后支持该功能 +::: + +腾讯云 [DLC](https://cloud.tencent.com/product/dlc) 采用HMS管理元数据,因此可用 Hive catalog 进行联邦分析。 +DLC 可基于 lakefs 或 cosn 进行数据存储。以下 catalog 创建方法对两种 FS 都适用。 +```sql +CREATE CATALOG dlc PROPERTIES ( + 'type' = 'hms', + 'hive.metastore.uris' = 'thrift://<dlc_metastore_ip>:<dlc_metastore_port>', + 's3.access_key' = 'xxxxx', + 's3.secret_key' = 'xxxxx', + 's3.region' = 'ap-xxx', + 's3.endpoint' = 'cos.ap-xxx.myqcloud.com', + 'fs.cosn.bucket.region' = 'ap-xxx', + 'fs.ofs.user.appid' = '<your_tencent_cloud_appid>', + 'fs.ofs.tmp.cache.dir' = '<tmp_cache_dir>' +); +``` + +创建完 catalog 后需要对 catalog 绑定的 appid 进行[授权](https://cloud.tencent.com/document/product/436/38648) 。 + ### Hive On S3 ```sql diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/lakehouse/datalake/hive.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/lakehouse/datalake/hive.md index f3389452268..9bad8516b87 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/lakehouse/datalake/hive.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/lakehouse/datalake/hive.md @@ -111,6 +111,30 @@ CREATE CATALOG hive PROPERTIES ( ); ``` +### Doris 访问腾讯云 DLC + +:::note +Doris 2.0 版本自 2.0.13 后支持该功能 +::: + +腾讯云 [DLC](https://cloud.tencent.com/product/dlc) 采用HMS管理元数据,因此可用 Hive catalog 进行联邦分析。 +DLC 可基于 LakeFS 或 COSN 进行数据存储。以下 catalog 创建方法对两种 FS 都适用。 +```sql +CREATE CATALOG dlc PROPERTIES ( + 'type' = 'hms', + 'hive.metastore.uris' = 'thrift://<dlc_metastore_ip>:<dlc_metastore_port>', + 's3.access_key' = 'xxxxx', + 's3.secret_key' = 'xxxxx', + 's3.region' = 'ap-xxx', + 's3.endpoint' = 'cos.ap-xxx.myqcloud.com', + 'fs.cosn.bucket.region' = 'ap-xxx', + 'fs.ofs.user.appid' = '<your_tencent_cloud_appid>', + 'fs.ofs.tmp.cache.dir' = '<tmp_cache_dir>' +); +``` + +创建完 catalog 后需要对 catalog 绑定的 appid 进行[授权](https://cloud.tencent.com/document/product/436/38648) 。 + ### Hive On S3 ```sql diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/hive.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/hive.md index 9b7b9ebedde..69fd80c7380 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/hive.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/hive.md @@ -108,6 +108,30 @@ CREATE CATALOG hive PROPERTIES ( ); ``` +### Doris 访问腾讯云 DLC + +:::note +Doris 2.1 版本自 2.1.5 后支持该功能 +::: + +腾讯云 [DLC](https://cloud.tencent.com/product/dlc) 采用HMS管理元数据,因此可用 Hive catalog 进行联邦分析。 +DLC 可基于 LakeFS 或 COSN 进行数据存储。以下 catalog 创建方法对两种 FS 都适用。 +```sql +CREATE CATALOG dlc PROPERTIES ( + 'type' = 'hms', + 'hive.metastore.uris' = 'thrift://<dlc_metastore_ip>:<dlc_metastore_port>', + 's3.access_key' = 'xxxxx', + 's3.secret_key' = 'xxxxx', + 's3.region' = 'ap-xxx', + 's3.endpoint' = 'cos.ap-xxx.myqcloud.com', + 'fs.cosn.bucket.region' = 'ap-xxx', + 'fs.ofs.user.appid' = '<your_tencent_cloud_appid>', + 'fs.ofs.tmp.cache.dir' = '<tmp_cache_dir>' +); +``` + +创建完 catalog 后需要对 catalog 绑定的 appid 进行[授权](https://cloud.tencent.com/document/product/436/38648) 。 + ### Hive On S3 ```sql diff --git a/versioned_docs/version-2.0/lakehouse/datalake/hive.md b/versioned_docs/version-2.0/lakehouse/datalake/hive.md index 41354622e88..23fc5bf9ba2 100644 --- a/versioned_docs/version-2.0/lakehouse/datalake/hive.md +++ b/versioned_docs/version-2.0/lakehouse/datalake/hive.md @@ -109,6 +109,31 @@ CREATE CATALOG hive PROPERTIES ( ); ``` +### Doris query DLC + +:::note +Supported since Doris version 2.0.13 +::: + +[DLC](https://cloud.tencent.com/product/dlc) of Tencent Cloud use HMS to manage its metadata, so Hive catalog can be used。 +DLC stores data on LakeFS or COSN, below catalog is compatible with both file system. +```sql +CREATE CATALOG dlc PROPERTIES ( + 'type' = 'hms', + 'hive.metastore.uris' = 'thrift://<dlc_metastore_ip>:<dlc_metastore_port>', + 's3.access_key' = 'xxxxx', + 's3.secret_key' = 'xxxxx', + 's3.region' = 'ap-xxx', + 's3.endpoint' = 'cos.ap-xxx.myqcloud.com', + 'fs.cosn.bucket.region' = 'ap-xxx', + 'fs.ofs.user.appid' = '<your_tencent_cloud_appid>', + 'fs.ofs.tmp.cache.dir' = '<tmp_cache_dir>' +); +``` + +After create catalog, you need to [grant](https://cloud.tencent.com/document/product/436/38648) essential privilege to the appid your dlc catalog is using. + + ### Hive on S3 ```SQL diff --git a/versioned_docs/version-2.1/lakehouse/datalake-analytics/hive.md b/versioned_docs/version-2.1/lakehouse/datalake-analytics/hive.md index 741c9bd373d..c39742a5d45 100644 --- a/versioned_docs/version-2.1/lakehouse/datalake-analytics/hive.md +++ b/versioned_docs/version-2.1/lakehouse/datalake-analytics/hive.md @@ -108,6 +108,30 @@ CREATE CATALOG hive PROPERTIES ( ); ``` +### Doris query DLC + +:::note +Supported since Doris version 2.1.5 +::: + +[DLC](https://cloud.tencent.com/product/dlc) of Tencent Cloud use HMS to manage its metadata, so Hive catalog can be used。 +DLC stores data on LakeFS or COSN, below catalog is compatible with both file system. +```sql +CREATE CATALOG dlc PROPERTIES ( + 'type' = 'hms', + 'hive.metastore.uris' = 'thrift://<dlc_metastore_ip>:<dlc_metastore_port>', + 's3.access_key' = 'xxxxx', + 's3.secret_key' = 'xxxxx', + 's3.region' = 'ap-xxx', + 's3.endpoint' = 'cos.ap-xxx.myqcloud.com', + 'fs.cosn.bucket.region' = 'ap-xxx', + 'fs.ofs.user.appid' = '<your_tencent_cloud_appid>', + 'fs.ofs.tmp.cache.dir' = '<tmp_cache_dir>' +); +``` + +After create catalog, you need to [grant](https://cloud.tencent.com/document/product/436/38648) essential privilege to the appid your dlc catalog is using. + ### Hive On S3 ```sql --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org