This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new 5cf3d2cf00e [fix](catalog) fix iceberg catalog with glue doc (#1265) 5cf3d2cf00e is described below commit 5cf3d2cf00e7c30dcc6cc180db816c89e753c694 Author: Mingyu Chen (Rayner) <morning...@163.com> AuthorDate: Fri Nov 1 23:54:36 2024 +0800 [fix](catalog) fix iceberg catalog with glue doc (#1265) # Versions - [x] dev - [x] 3.0 - [x] 2.1 - [ ] 2.0 # Languages - [x] Chinese - [x] English --- docs/lakehouse/cloud-auth.md | 14 ++++++-------- docs/lakehouse/datalake-analytics/iceberg.md | 13 ++++++++----- .../current/lakehouse/cloud-auth.md | 14 ++++++-------- .../current/lakehouse/datalake-analytics/iceberg.md | 13 ++++++++----- .../version-2.1/lakehouse/cloud-auth.md | 14 ++++++-------- .../version-2.1/lakehouse/datalake-analytics/iceberg.md | 13 ++++++++----- .../version-3.0/lakehouse/cloud-auth.md | 14 ++++++-------- .../version-3.0/lakehouse/datalake-analytics/iceberg.md | 13 ++++++++----- versioned_docs/version-2.1/lakehouse/cloud-auth.md | 14 ++++++-------- .../version-2.1/lakehouse/datalake-analytics/iceberg.md | 13 ++++++++----- versioned_docs/version-3.0/lakehouse/cloud-auth.md | 14 ++++++-------- .../version-3.0/lakehouse/datalake-analytics/iceberg.md | 13 ++++++++----- 12 files changed, 84 insertions(+), 78 deletions(-) diff --git a/docs/lakehouse/cloud-auth.md b/docs/lakehouse/cloud-auth.md index b110a5885a2..619a5ad5af1 100644 --- a/docs/lakehouse/cloud-auth.md +++ b/docs/lakehouse/cloud-auth.md @@ -34,19 +34,17 @@ Now Doris support two types of authentication to access AWS service. ### Catalog Credentials -The Catalog supports filling in basic Credentials properties, such as: -1. For S3: `s3.endpoint`,`s3.access_key`,`s3.secret_key`。 -2. For Glue: `glue.endpoint`,`glue.access_key`,`glue.secret_key`。 - When access Glue though Iceberg Catalog, we can access tables on Glue by filling in the following properties: ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` diff --git a/docs/lakehouse/datalake-analytics/iceberg.md b/docs/lakehouse/datalake-analytics/iceberg.md index 4d2f2f311a1..73b056a11c9 100644 --- a/docs/lakehouse/datalake-analytics/iceberg.md +++ b/docs/lakehouse/datalake-analytics/iceberg.md @@ -114,20 +114,23 @@ CREATE CATALOG iceberg PROPERTIES ( #### AWS Glue > When connecting Glue, if it's not on the EC2 environment, need copy the > `~/.aws` from the EC2 environment to the current environment. And can also > download and configure the [AWS Cli > tools](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), > which also creates the `.aws` directory under the current user directory. +> Please upgrade to Doris 2.1.7, 3.0.3 or later version to connect to Glue. ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` 1. For Iceberg properties, see [Iceberg Glue Catalog](https://iceberg.apache.org/docs/latest/aws/#glue-catalog). -2. If you do not fill the credentials(`glue.access_key` and `glue.secret_key`) in glue catalog, the default DefaultAWSCredentialsProviderChain will be used, and it will read credentials and the system environment variables or instance profile properties on AWS EC2. +2. If you do not specify `client.credentials-provider`, the default DefaultAWSCredentialsProviderChain will be used, and it will read credentials and the system environment variables or instance profile properties on AWS EC2. #### Alibaba Cloud DLF diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/cloud-auth.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/cloud-auth.md index 8d31711eabd..2d01370bb9c 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/cloud-auth.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/cloud-auth.md @@ -34,19 +34,17 @@ under the License. ### 使用Catalog属性认证 -Catalog支持填写基本的Credentials属性,比如: -1. 访问S3时,可以使用s3.endpoint,s3.access_key,s3.secret_key。 -2. 访问Glue时,可以使用glue.endpoint,glue.access_key,glue.secret_key。 - 以Iceberg Catalog访问Glue为例,我们可以填写以下属性访问在Glue上托管的表: ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/iceberg.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/iceberg.md index b61c0d03700..731ed8fcc4a 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/iceberg.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/datalake-analytics/iceberg.md @@ -115,20 +115,23 @@ CREATE CATALOG iceberg PROPERTIES ( #### AWS Glue > 连接 Glue 时,如果是在非 EC2 环境,需要将 EC2 环境里的 `~/.aws` 目录拷贝到当前环境里。也可以下载[AWS > Cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)工具进行配置,这种方式也会在当前用户目录下创建`.aws`目录。 +> 请升级到 Doris 2.1.7 或 3.0.3 之后的版本使用该功能。 ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` 1. Iceberg 属性详情参见 [Iceberg Glue Catalog](https://iceberg.apache.org/docs/latest/aws/#glue-catalog) -2. 如果在 AWS 服务(如 EC2)中,不填写 Credentials 相关信息(`glue.access_key`和`glue.secret_key`),Doris 就会使用默认的 DefaultAWSCredentialsProviderChain,它会读取系统环境变量或者 InstanceProfile 中配置的属性。 +2. 如果不指定 `client.credentials-provider`,Doris 就会使用默认的 DefaultAWSCredentialsProviderChain,它会读取系统环境变量或者 InstanceProfile 中配置的属性。 #### 阿里云 DLF diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/cloud-auth.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/cloud-auth.md index 8d31711eabd..2d01370bb9c 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/cloud-auth.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/cloud-auth.md @@ -34,19 +34,17 @@ under the License. ### 使用Catalog属性认证 -Catalog支持填写基本的Credentials属性,比如: -1. 访问S3时,可以使用s3.endpoint,s3.access_key,s3.secret_key。 -2. 访问Glue时,可以使用glue.endpoint,glue.access_key,glue.secret_key。 - 以Iceberg Catalog访问Glue为例,我们可以填写以下属性访问在Glue上托管的表: ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/iceberg.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/iceberg.md index b61c0d03700..731ed8fcc4a 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/iceberg.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/datalake-analytics/iceberg.md @@ -115,20 +115,23 @@ CREATE CATALOG iceberg PROPERTIES ( #### AWS Glue > 连接 Glue 时,如果是在非 EC2 环境,需要将 EC2 环境里的 `~/.aws` 目录拷贝到当前环境里。也可以下载[AWS > Cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)工具进行配置,这种方式也会在当前用户目录下创建`.aws`目录。 +> 请升级到 Doris 2.1.7 或 3.0.3 之后的版本使用该功能。 ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` 1. Iceberg 属性详情参见 [Iceberg Glue Catalog](https://iceberg.apache.org/docs/latest/aws/#glue-catalog) -2. 如果在 AWS 服务(如 EC2)中,不填写 Credentials 相关信息(`glue.access_key`和`glue.secret_key`),Doris 就会使用默认的 DefaultAWSCredentialsProviderChain,它会读取系统环境变量或者 InstanceProfile 中配置的属性。 +2. 如果不指定 `client.credentials-provider`,Doris 就会使用默认的 DefaultAWSCredentialsProviderChain,它会读取系统环境变量或者 InstanceProfile 中配置的属性。 #### 阿里云 DLF diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/cloud-auth.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/cloud-auth.md index 8d31711eabd..2d01370bb9c 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/cloud-auth.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/cloud-auth.md @@ -34,19 +34,17 @@ under the License. ### 使用Catalog属性认证 -Catalog支持填写基本的Credentials属性,比如: -1. 访问S3时,可以使用s3.endpoint,s3.access_key,s3.secret_key。 -2. 访问Glue时,可以使用glue.endpoint,glue.access_key,glue.secret_key。 - 以Iceberg Catalog访问Glue为例,我们可以填写以下属性访问在Glue上托管的表: ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/datalake-analytics/iceberg.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/datalake-analytics/iceberg.md index b61c0d03700..731ed8fcc4a 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/datalake-analytics/iceberg.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/lakehouse/datalake-analytics/iceberg.md @@ -115,20 +115,23 @@ CREATE CATALOG iceberg PROPERTIES ( #### AWS Glue > 连接 Glue 时,如果是在非 EC2 环境,需要将 EC2 环境里的 `~/.aws` 目录拷贝到当前环境里。也可以下载[AWS > Cli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)工具进行配置,这种方式也会在当前用户目录下创建`.aws`目录。 +> 请升级到 Doris 2.1.7 或 3.0.3 之后的版本使用该功能。 ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` 1. Iceberg 属性详情参见 [Iceberg Glue Catalog](https://iceberg.apache.org/docs/latest/aws/#glue-catalog) -2. 如果在 AWS 服务(如 EC2)中,不填写 Credentials 相关信息(`glue.access_key`和`glue.secret_key`),Doris 就会使用默认的 DefaultAWSCredentialsProviderChain,它会读取系统环境变量或者 InstanceProfile 中配置的属性。 +2. 如果不指定 `client.credentials-provider`,Doris 就会使用默认的 DefaultAWSCredentialsProviderChain,它会读取系统环境变量或者 InstanceProfile 中配置的属性。 #### 阿里云 DLF diff --git a/versioned_docs/version-2.1/lakehouse/cloud-auth.md b/versioned_docs/version-2.1/lakehouse/cloud-auth.md index b110a5885a2..619a5ad5af1 100644 --- a/versioned_docs/version-2.1/lakehouse/cloud-auth.md +++ b/versioned_docs/version-2.1/lakehouse/cloud-auth.md @@ -34,19 +34,17 @@ Now Doris support two types of authentication to access AWS service. ### Catalog Credentials -The Catalog supports filling in basic Credentials properties, such as: -1. For S3: `s3.endpoint`,`s3.access_key`,`s3.secret_key`。 -2. For Glue: `glue.endpoint`,`glue.access_key`,`glue.secret_key`。 - When access Glue though Iceberg Catalog, we can access tables on Glue by filling in the following properties: ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` diff --git a/versioned_docs/version-2.1/lakehouse/datalake-analytics/iceberg.md b/versioned_docs/version-2.1/lakehouse/datalake-analytics/iceberg.md index 4d2f2f311a1..73b056a11c9 100644 --- a/versioned_docs/version-2.1/lakehouse/datalake-analytics/iceberg.md +++ b/versioned_docs/version-2.1/lakehouse/datalake-analytics/iceberg.md @@ -114,20 +114,23 @@ CREATE CATALOG iceberg PROPERTIES ( #### AWS Glue > When connecting Glue, if it's not on the EC2 environment, need copy the > `~/.aws` from the EC2 environment to the current environment. And can also > download and configure the [AWS Cli > tools](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), > which also creates the `.aws` directory under the current user directory. +> Please upgrade to Doris 2.1.7, 3.0.3 or later version to connect to Glue. ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` 1. For Iceberg properties, see [Iceberg Glue Catalog](https://iceberg.apache.org/docs/latest/aws/#glue-catalog). -2. If you do not fill the credentials(`glue.access_key` and `glue.secret_key`) in glue catalog, the default DefaultAWSCredentialsProviderChain will be used, and it will read credentials and the system environment variables or instance profile properties on AWS EC2. +2. If you do not specify `client.credentials-provider`, the default DefaultAWSCredentialsProviderChain will be used, and it will read credentials and the system environment variables or instance profile properties on AWS EC2. #### Alibaba Cloud DLF diff --git a/versioned_docs/version-3.0/lakehouse/cloud-auth.md b/versioned_docs/version-3.0/lakehouse/cloud-auth.md index b110a5885a2..619a5ad5af1 100644 --- a/versioned_docs/version-3.0/lakehouse/cloud-auth.md +++ b/versioned_docs/version-3.0/lakehouse/cloud-auth.md @@ -34,19 +34,17 @@ Now Doris support two types of authentication to access AWS service. ### Catalog Credentials -The Catalog supports filling in basic Credentials properties, such as: -1. For S3: `s3.endpoint`,`s3.access_key`,`s3.secret_key`。 -2. For Glue: `glue.endpoint`,`glue.access_key`,`glue.secret_key`。 - When access Glue though Iceberg Catalog, we can access tables on Glue by filling in the following properties: ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` diff --git a/versioned_docs/version-3.0/lakehouse/datalake-analytics/iceberg.md b/versioned_docs/version-3.0/lakehouse/datalake-analytics/iceberg.md index 4d2f2f311a1..73b056a11c9 100644 --- a/versioned_docs/version-3.0/lakehouse/datalake-analytics/iceberg.md +++ b/versioned_docs/version-3.0/lakehouse/datalake-analytics/iceberg.md @@ -114,20 +114,23 @@ CREATE CATALOG iceberg PROPERTIES ( #### AWS Glue > When connecting Glue, if it's not on the EC2 environment, need copy the > `~/.aws` from the EC2 environment to the current environment. And can also > download and configure the [AWS Cli > tools](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html), > which also creates the `.aws` directory under the current user directory. +> Please upgrade to Doris 2.1.7, 3.0.3 or later version to connect to Glue. ```sql -CREATE CATALOG glue PROPERTIES ( +-- Using access key and secret key +CREATE CATALOG glue2 PROPERTIES ( "type"="iceberg", "iceberg.catalog.type" = "glue", - "glue.endpoint" = "https://glue.us-east-1.amazonaws.com", - "glue.access_key" = "ak", - "glue.secret_key" = "sk" + "glue.endpoint" = "https://glue.us-east-1.amazonaws.com/", + "client.credentials-provider" = "com.amazonaws.glue.catalog.credentials.ConfigAWSProvider", + "client.credentials-provider.glue.access_key" = "ak", + "client.credentials-provider.glue.secret_key" = "sk" ); ``` 1. For Iceberg properties, see [Iceberg Glue Catalog](https://iceberg.apache.org/docs/latest/aws/#glue-catalog). -2. If you do not fill the credentials(`glue.access_key` and `glue.secret_key`) in glue catalog, the default DefaultAWSCredentialsProviderChain will be used, and it will read credentials and the system environment variables or instance profile properties on AWS EC2. +2. If you do not specify `client.credentials-provider`, the default DefaultAWSCredentialsProviderChain will be used, and it will read credentials and the system environment variables or instance profile properties on AWS EC2. #### Alibaba Cloud DLF --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org