This is an automated email from the ASF dual-hosted git repository.
kxiao pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 608fd1d38c5 [fix](multi-catalog)add the FAQ for Aliyun DLF and add the
fs.xx.impl check #25594 (#26422)
608fd1d38c5 is described below
commit 608fd1d38c55bea13f4e34b8e27639831e38e7d6
Author: slothever <18522955+w...@users.noreply.github.com>
AuthorDate: Sat Nov 4 15:42:45 2023 +0800
[fix](multi-catalog)add the FAQ for Aliyun DLF and add the fs.xx.impl check
#25594 (#26422)
---
docs/en/docs/lakehouse/faq.md | 278 +++++++++++++++++++++
docs/en/docs/lakehouse/multi-catalog/dlf.md | 32 +--
docs/zh-CN/docs/lakehouse/faq.md | 273 ++++++++++++++++++++
docs/zh-CN/docs/lakehouse/multi-catalog/dlf.md | 34 +--
.../doris/datasource/hive/HiveMetaStoreCache.java | 6 +-
.../datasource/property/PropertyConverter.java | 7 +-
6 files changed, 594 insertions(+), 36 deletions(-)
diff --git a/docs/en/docs/lakehouse/faq.md b/docs/en/docs/lakehouse/faq.md
new file mode 100644
index 00000000000..a6c97cfbb6d
--- /dev/null
+++ b/docs/en/docs/lakehouse/faq.md
@@ -0,0 +1,278 @@
+---
+{
+ "title": "FAQ",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+
+# FAQ
+
+## Kerberos
+
+
+1. What to do with the `GSS initiate failed` error when connecting to Hive
Metastore with Kerberos authentication?
+
+ Usually it is caused by incorrect Kerberos authentication information, you
can troubleshoot by the following steps:
+
+ 1. In versions before 1.2.1, the libhdfs3 library that Doris depends on
does not enable gsasl. Please update to a version later than 1.2.2.
+ 2. Confirm that the correct keytab and principal are set for each
component, and confirm that the keytab file exists on all FE and BE nodes.
+
+ 1. `hadoop.kerberos.keytab`/`hadoop.kerberos.principal`: for Hadoop HDFS
+ 2. `hive.metastore.kerberos.principal`: for hive metastore.
+
+ 3. Try to replace the IP in the principal with a domain name (do not use
the default `_HOST` placeholder)
+ 4. Confirm that the `/etc/krb5.conf` file exists on all FE and BE nodes.
+
+2. An error is reported when connecting to the Hive database through the Hive
Catalog: `RemoteException: SIMPLE authentication is not enabled. Available:
[TOKEN, KERBEROS]`
+
+ If both `show databases` and `show tables` are OK, and the above error
occurs when querying, we need to perform the following two operations:
+ - Core-site.xml and hdfs-site.xml need to be placed in the fe/conf and
be/conf directories
+ - The BE node executes the kinit of Kerberos, restarts the BE, and then
executes the query.
+
+3. If an error is reported while querying the catalog with Kerberos:
`GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos Ticket)`.
+ - Restarting FE and BE can solve the problem in most cases.
+ - Before the restart all the nodes, can put
`-Djavax.security.auth.useSubjectCredsOnly=false` to the `JAVA_OPTS` in
`"${DORIS_HOME}/be/conf/be.conf"`, which can obtain credentials through the
underlying mechanism, rather than through the application.
+ - Get more solutions to common JAAS errors from the [JAAS
Troubleshooting](https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html).
+
+4. The solutions when configuring Kerberos in the catalog and encounter an
error: `Unable to obtain password from user`.
+ - The principal used must exist in the klist, use `klist -kt your.keytab`
to check.
+ - Ensure the catalog configuration correct, such as missing the
`yarn.resourcemanager.principal`.
+ - If the preceding checks are correct, the JDK version installed by yum or
other package-management utility in the current system maybe have an
unsupported encryption algorithm. It is recommended to install JDK by yourself
and set `JAVA_HOME` environment variable.
+
+5. An error is reported when using KMS to access HDFS:
`java.security.InvalidKeyException: Illegal key size`
+
+ Upgrade the JDK version to a version >= Java 8 u162. Or download and
install the JCE Unlimited Strength Jurisdiction Policy Files corresponding to
the JDK.
+
+6. If an error is reported while configuring Kerberos in the catalog: `SIMPLE
authentication is not enabled. Available:[TOKEN, KERBEROS]`.
+
+ Need to put `core-site.xml` to the `"${DORIS_HOME}/be/conf"` directory.
+
+ If an error is reported while accessing HDFS: `No common protection layer
between client and server`, check the `hadoop.rpc.protection` on the client and
server to make them consistent.
+
+ ```
+ <?xml version="1.0" encoding="UTF-8"?>
+ <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+
+ <configuration>
+
+ <property>
+ <name>hadoop.security.authentication</name>
+ <value>kerberos</value>
+ </property>
+
+ </configuration>
+ ```
+
+7. If an error is reported while configuring Kerberos for Broker Load: `Cannot
locate default realm.`.
+
+ Add `-Djava.security.krb5.conf=/your-path` to the `JAVA_OPTS` of the broker
startup script `start_broker.sh`.
+
+8. When using Kerberos configuration in the Catalog, the `hadoop.username`
property cannot be appeared in Catalog properties.
+
+## JDBC Catalog
+
+1. An error is reported when connecting to SQLServer through JDBC Catalog:
`unable to find valid certification path to requested target`
+
+ Please add `trustServerCertificate=true` option in `jdbc_url`.
+
+2. When connecting to the MySQL database through the JDBC Catalog, the Chinese
characters are garbled, or the Chinese character condition query is incorrect
+
+ Please add `useUnicode=true&characterEncoding=utf-8` in `jdbc_url`
+
+ > Note: After version 1.2.3, these parameters will be automatically added
when using JDBC Catalog to connect to the MySQL database.
+
+3. An error is reported when connecting to the MySQL database through the JDBC
Catalog: `Establishing SSL connection without server's identity verification is
not recommended`
+
+ Please add `useSSL=true` in `jdbc_url`
+
+4. When using JDBC Catalog to synchronize MySQL data to Doris, the date data
synchronization error occurs. It is necessary to check whether the MySQL
version corresponds to the MySQL driver package. For example, the driver
com.mysql.cj.jdbc.Driver is required for MySQL8 and above.
+
+
+## Hive Catalog
+
+1. What to do with errors such as `failed to get schema` and `Storage schema
reading not supported` when accessing Icerberg tables via Hive Metastore?
+
+ To fix this, please place the Jar file package of `iceberg` runtime in the
`lib/` directory of Hive.
+
+ And configure as follows in `hive-site.xml` :
+
+ ```
+
metastore.storage.schema.reader.impl=org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader
+ ```
+
+ After configuring, please restart Hive Metastore.
+
+2. An error is reported when connecting Hive Catalog: `Caused by:
java.lang.NullPointerException`
+
+ If there is stack trace in fe.log:
+
+ ```
+ Caused by: java.lang.NullPointerException
+ at
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:78)
~[hive-exec-3.1.3-core.jar:3.1.3]
+ at
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:55)
~[hive-exec-3.1.3-core.jar:3.1.3]
+ at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1548)
~[doris-fe.jar:3.1.3]
+ at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1542)
~[doris-fe.jar:3.1.3]
+ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_181]
+ ```
+
+ Try adding `"metastore.filter.hook" =
"org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl"` in `create
catalog` statement.
+
+3. If the `show tables` is normal after creating the Hive Catalog, but the
query report `java.net.UnknownHostException: xxxxx`
+
+ Add a property in CATALOG:
+ ```
+ 'fs.defaultFS' = 'hdfs://<your_nameservice_or_actually_HDFS_IP_and_port>'
+ ```
+
+4. The table in orc format of Hive 1.x may encounter system column names such
as `_col0`, `_col1`, `_col2`... in the underlying orc file schema, which need
to be specified in the catalog configuration. Add `hive.version` to 1.x.x so
that it will use the column names in the hive table for mapping.
+
+ ```sql
+ CREATE CATALOG hive PROPERTIES (
+ 'hive.version' = '1.x.x'
+ );
+ ```
+
+5. If an error related to the Hive Metastore is reported while querying the
catalog: `Invalid method name`.
+
+ Configure the `hive.version`.
+
+ ```sql
+ CREATE CATALOG hive PROPERTIES (
+ 'hive.version' = '2.x.x'
+ );
+ ```
+
+6. When querying a table in ORC format, FE reports an error `Could not obtain
block` or `Caused by: java.lang.NoSuchFieldError: types`
+
+ For ORC files, by default, FE will access HDFS to obtain file information
and split files. In some cases, FE may not be able to access HDFS. It can be
solved by adding the following parameters:
+
+ `"hive.exec.orc.split.strategy" = "BI"`
+
+ Other options: HYBRID (default), ETL.
+
+7. The values of the partition fields in the hudi table can be found on hive,
but they cannot be found on doris.
+
+ Doris and hive currently query hudi differently. Doris needs to add
partition fields to the avsc file of the hudi table structure. If not added, it
will cause Doris to query partition_ Val is empty (even if home. datasource.
live_sync. partition_fields=partition_val is set)
+
+ ```
+ {
+ "type": "record",
+ "name": "record",
+ "fields": [{
+ "name": "partition_val",
+ "type": [
+ "null",
+ "string"
+ ],
+ "doc": "Preset partition field, empty string when not partitioned",
+ "default": null
+ },
+ {
+ "name": "name",
+ "type": "string",
+ "doc": "名称"
+ },
+ {
+ "name": "create_time",
+ "type": "string",
+ "doc": "创建时间"
+ }
+ ]
+ }
+ ```
+
+8. Query the appearance of hive and encounter this
error:`java.lang.ClassNotFoundException: Class
com.hadoop.compression.lzo.LzoCodec not found`
+
+ Search in the hadoop environment hadoop-lzo-*.jar, and put it under
"${DORIS_HOME}/fe/lib/",then restart fe.
+
+ Starting from version 2.0.2, this file can be placed in BE's `custom_lib/`
directory (if it does not exist, just create it manually) to prevent the file
from being lost due to the replacement of the lib directory when upgrading the
cluster.
+
+9. Create a hive table specifying `serde` as
`org.apache.hadoop.hive.contrib.serde2.MultiDelimitserDe`, and an error is
reported when accessing the table: `storage schema reading not supported`
+
+ Add the following configuration to the hive-site .xml file and restart the
HMS service:
+
+ ```
+ <property>
+ <name>metastore.storage.schema.reader.impl</name>
+ <value>org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader</value>
+ </property>
+ ```
+
+10. Error：java.security.InvalidAlgorithmParameterException: the trustAnchors
parameter must be non-empty
+
+ Entire error info found in FE.log is shown as below:
+ ```
+ org.apache.doris.common.UserException: errCode = 2, detailMessage = S3
list path failed. path=s3://bucket/part-*,msg=errors while get file status
listStatus on s3://bucket: com.amazonaws.SdkClientException: Unable to execute
HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty: Unable to execute HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be [...]
+ org.apache.doris.common.UserException: errCode = 2, detailMessage = S3
list path exception. path=s3://bucket/part-*, err: errCode = 2, detailMessage =
S3 list path failed. path=s3://bucket/part-*,msg=errors while get file status
listStatus on s3://bucket: com.amazonaws.SdkClientException: Unable to execute
HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty: Unable to execute HTTP request: Unexpected error: ja [...]
+ org.apache.hadoop.fs.s3a.AWSClientIOException: listStatus on s3://bucket:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Unexpected
error: java.security.InvalidAlgorithmParameterException: the trustAnchors
parameter must be non-empty: Unable to execute HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty
+ Caused by: com.amazonaws.SdkClientException: Unable to execute HTTP
request: Unexpected error: java.security.InvalidAlgorithmParameterException:
the trustAnchors parameter must be non-empty
+ Caused by: javax.net.ssl.SSLException: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty
+ Caused by: java.lang.RuntimeException: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty
+ Caused by: java.security.InvalidAlgorithmParameterException: the
trustAnchors parameter must be non-empty
+ ```
+
+ Try to update FE node CA certificates, use command `update-ca-trust
(CentOS/RockyLinux)`, then restart FE process.
+
+## HDFS
+
+1. What to do with the`java.lang.VerifyError: xxx` error when accessing HDFS
3.x?
+
+ Doris 1.2.1 and the older versions rely on Hadoop 2.8. Please update Hadoop
to 2.10.2 or update Doris to 1.2.2 or newer.
+
+2. Use Hedged Read to optimize the problem of slow HDFS reading.
+
+ In some cases, the high load of HDFS may lead to a long time to read the
data on HDFS, thereby slowing down the overall query efficiency. HDFS Client
provides Hedged Read.
+ This function can start another read thread to read the same data when a
read request exceeds a certain threshold and is not returned, and whichever is
returned first will use the result.
+
+ This feature can be enabled in two ways:
+
+ - Specify in the parameters to create the Catalog:
+
+ ```
+ create catalog regression properties (
+ 'type'='hms',
+ 'hive.metastore.uris' = 'thrift://172.21.16.47:7004',
+ 'dfs.client.hedged.read.threadpool.size' = '128',
+ 'dfs.client.hedged.read.threshold.millis' = "500"
+ );
+ ```
+
+ `dfs.client.hedged.read.threadpool.size` indicates the number of
threads used for Hedged Read, which are shared by one HDFS Client. Usually, for
an HDFS cluster, BE nodes will share an HDFS Client.
+
+ `dfs.client.hedged.read.threshold.millis` is the read threshold in
milliseconds. When a read request exceeds this threshold and is not returned,
Hedged Read will be triggered.
+
+ After enabling it, you can see related parameters in Query Profile:
+
+ `TotalHedgedRead`: The number of Hedged Reads initiated.
+
+ `HedgedReadWins`: The number of successful Hedged Reads (numbers
initiated and returned faster than the original request)
+
+ Note that the value here is the cumulative value of a single HDFS Client,
not the value of a single query. The same HDFS Client will be reused by
multiple queries.
+
+## DLF Catalog
+
+1. When using DLF Catalog, BE reads `Invalid address` when fetching JindoFS
data and needs to add the domain name to IP mapping that appears in the log in
`/ets/hosts`.
+
+2. When reading data is not authorized, use the `hadoop.username` property to
specify the authorized user.
+
+3. The metadata in the DLF Catalog is consistent with the DLF. When DLF is
used to manage metadata, newly imported Hive partitions may not be synchronized
by DLF, resulting in inconsistency between the DLF and Hive metadata. In this
case, ensure firstly that the Hive metadata is fully synchronized by DLF.
diff --git a/docs/en/docs/lakehouse/multi-catalog/dlf.md
b/docs/en/docs/lakehouse/multi-catalog/dlf.md
index 763fa9fdd8c..7fb88bb316b 100644
--- a/docs/en/docs/lakehouse/multi-catalog/dlf.md
+++ b/docs/en/docs/lakehouse/multi-catalog/dlf.md
@@ -67,23 +67,25 @@ Doris supports accessing Hive/Iceberg/Hudi metadata in DLF.
### Use OSS-HDFS as the datasource
1. Enable OSS-HDFS. [Grant access to OSS or
OSS-HDFS](https://www.alibabacloud.com/help/en/e-mapreduce/latest/oss-hdfsnew)
-2. Download the SDK. [JindoData
SDK](https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/5.x/5.0.0-beta7/jindodata_download.md)
-3. Decompress the jindosdk.tar.gz, and then enter its lib directory and put
`jindo-core.jar, jindo-sdk.jar` to both `${DORIS_HOME}/fe/lib` and
`${DORIS_HOME}/be/lib/java_extensions`.
+2. Download the SDK. [JindoData
SDK](https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/5.x/5.0.0-beta7/jindodata_download.md).
If the Jindo SDK directory already exists on the cluster, skip this step.
+3. Decompress the jindosdk.tar.gz or locate the Jindo SDK directory on the
cluster, and then enter its lib directory and put `jindo-core.jar,
jindo-sdk.jar` to both `${DORIS_HOME}/fe/lib` and
`${DORIS_HOME}/be/lib/java_extensions/preload-extensions`.
4. Create DLF Catalog, set `oss.hdfs.enabled` as `true`：
-```sql
-CREATE CATALOG dlf_oss_hdfs PROPERTIES (
- "type"="hms",
- "hive.metastore.type" = "dlf",
- "dlf.proxy.mode" = "DLF_ONLY",
- "dlf.endpoint" = "datalake-vpc.cn-beijing.aliyuncs.com",
- "dlf.region" = "cn-beijing",
- "dlf.uid" = "uid",
- "dlf.access_key" = "ak",
- "dlf.secret_key" = "sk",
- "oss.hdfs.enabled" = "true"
-);
-```
+ ```sql
+ CREATE CATALOG dlf_oss_hdfs PROPERTIES (
+ "type"="hms",
+ "hive.metastore.type" = "dlf",
+ "dlf.proxy.mode" = "DLF_ONLY",
+ "dlf.endpoint" = "datalake-vpc.cn-beijing.aliyuncs.com",
+ "dlf.region" = "cn-beijing",
+ "dlf.uid" = "uid",
+ "dlf.access_key" = "ak",
+ "dlf.secret_key" = "sk",
+ "oss.hdfs.enabled" = "true"
+ );
+ ```
+
+5. When the Jindo SDK version is inconsistent with the version used on the EMR
cluster, will reported `Plugin not found` and the Jindo SDK needs to be
replaced with the corresponding version.
### DLF Iceberg Catalog
diff --git a/docs/zh-CN/docs/lakehouse/faq.md b/docs/zh-CN/docs/lakehouse/faq.md
new file mode 100644
index 00000000000..c1ab720e9e0
--- /dev/null
+++ b/docs/zh-CN/docs/lakehouse/faq.md
@@ -0,0 +1,273 @@
+---
+{
+ "title": "常见问题",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+
+# 常见问题
+
+## Kerberos
+
+1. 连接 Kerberos 认证的 Hive Metastore 报错：`GSS initiate failed`
+
+ 通常是因为 Kerberos 认证信息填写不正确导致的，可以通过以下步骤排查：
+
+ 1. 1.2.1 之前的版本中，Doris 依赖的 libhdfs3 库没有开启 gsasl。请更新至 1.2.2 之后的版本。
+ 2. 确认对各个组件，设置了正确的 keytab 和 principal，并确认 keytab 文件存在于所有 FE、BE 节点上。
+
+ 1. `hadoop.kerberos.keytab`/`hadoop.kerberos.principal`：用于 Hadoop hdfs
访问，填写 hdfs 对应的值。
+ 2. `hive.metastore.kerberos.principal`：用于 hive metastore。
+
+ 3. 尝试将 principal 中的 ip 换成域名（不要使用默认的 `_HOST` 占位符）
+ 4. 确认 `/etc/krb5.conf` 文件存在于所有 FE、BE 节点上。
+
+2. 通过 Hive Catalog 连接 Hive 数据库报错：`RemoteException: SIMPLE authentication is
not enabled. Available:[TOKEN, KERBEROS]`.
+
+ 如果在 `show databases` 和 `show tables` 都是没问题的情况下，查询的时候出现上面的错误，我们需要进行下面两个操作：
+ - fe/conf、be/conf 目录下需放置 core-site.xml 和 hdfs-site.xml
+ - BE 节点执行 Kerberos 的 kinit 然后重启 BE ，然后再去执行查询即可.
+
+3. 查询配置了Kerberos的外表，遇到该报错：`GSSException: No valid credentials provided
(Mechanism level: Failed to find any Kerberos Ticket)`，一般重启FE和BE能够解决该问题。
+
+ -
重启所有节点前可在`"${DORIS_HOME}/be/conf/be.conf"`中的JAVA_OPTS参数里配置`-Djavax.security.auth.useSubjectCredsOnly=false`，通过底层机制去获取JAAS
credentials信息，而不是应用程序。
+ - 在[JAAS
Troubleshooting](https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/Troubleshooting.html)中可获取更多常见JAAS报错的解决方法。
+
+4. 在Catalog中配置Kerberos时，报错`Unable to obtain password from user`的解决方法：
+
+ - 用到的principal必须在klist中存在，使用`klist -kt your.keytab`检查。
+ - 检查catalog配置是否正确，比如漏配`yarn.resourcemanager.principal`。
+ - 若上述检查没问题，则当前系统yum或者其他包管理软件安装的JDK版本存在不支持的加密算法，建议自行安装JDK并设置`JAVA_HOME`环境变量。
+
+5. 使用 KMS 访问 HDFS 时报错：`java.security.InvalidKeyException: Illegal key size`
+
+ 升级 JDK 版本到 >= Java 8 u162 的版本。或者下载安装 JDK 相应的 JCE Unlimited Strength
Jurisdiction Policy Files。
+
+6. 在Catalog中配置Kerberos时，如果报错`SIMPLE authentication is not enabled.
Available:[TOKEN,
KERBEROS]`，那么需要将`core-site.xml`文件放到`"${DORIS_HOME}/be/conf"`目录下。
+
+ 如果访问HDFS报错`No common protection layer between client and
server`，检查客户端和服务端的`hadoop.rpc.protection`属性，使他们保持一致。
+
+ ```
+ <?xml version="1.0" encoding="UTF-8"?>
+ <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+
+ <configuration>
+
+ <property>
+ <name>hadoop.security.authentication</name>
+ <value>kerberos</value>
+ </property>
+
+ </configuration>
+ ```
+7. 在使用Broker Load时，配置了Kerberos，如果报错`Cannot locate default realm.`。
+
+ 将 `-Djava.security.krb5.conf=/your-path` 配置项添加到Broker Load启动脚本的
`start_broker.sh` 的 `JAVA_OPTS`里。
+
+8. 当在Catalog里使用Kerberos配置时，不能同时使用`hadoop.username`属性。
+
+## JDBC Catalog
+
+1. 通过 JDBC Catalog 连接 SQLServer 报错：`unable to find valid certification path to
requested target`
+
+ 请在 `jdbc_url` 中添加 `trustServerCertificate=true` 选项。
+
+2. 通过 JDBC Catalog 连接 MySQL 数据库，中文字符乱码，或中文字符条件查询不正确
+
+ 请在 `jdbc_url` 中添加 `useUnicode=true&characterEncoding=utf-8`
+
+ > 注：1.2.3 版本后，使用 JDBC Catalog 连接 MySQL 数据库，会自动添加这些参数。
+
+3. 通过 JDBC Catalog 连接 MySQL 数据库报错：`Establishing SSL connection without
server's identity verification is not recommended`
+
+ 请在 `jdbc_url` 中添加 `useSSL=true`
+
+4. 使用JDBC
Catalog将MySQL数据同步到Doris中，日期数据同步错误。需要校验下MySQL的版本是否与MySQL的驱动包是否对应，比如MySQL8以上需要使用驱动com.mysql.cj.jdbc.Driver。
+
+
+## Hive Catalog
+
+1. 通过 Hive Metastore 访问 Iceberg 表报错：`failed to get schema` 或 `Storage schema
reading not supported`
+
+ 在 Hive 的 lib/ 目录放上 `iceberg` 运行时有关的 jar 包。
+
+ 在 `hive-site.xml` 配置：
+
+ ```
+
metastore.storage.schema.reader.impl=org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader
+ ```
+
+ 配置完成后需要重启Hive Metastore。
+
+2. 连接 Hive Catalog 报错：`Caused by: java.lang.NullPointerException`
+
+ 如 fe.log 中有如下堆栈：
+
+ ```
+ Caused by: java.lang.NullPointerException
+ at
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:78)
~[hive-exec-3.1.3-core.jar:3.1.3]
+ at
org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:55)
~[hive-exec-3.1.3-core.jar:3.1.3]
+ at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1548)
~[doris-fe.jar:3.1.3]
+ at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:1542)
~[doris-fe.jar:3.1.3]
+ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_181]
+ ```
+
+ 可以尝试在 `create catalog` 语句中添加 `"metastore.filter.hook" =
"org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl"` 解决。
+
+3. 如果创建 Hive Catalog 后能正常`show tables`，但查询时报`java.net.UnknownHostException:
xxxxx`
+
+ 可以在 CATALOG 的 PROPERTIES 中添加
+ ```
+ 'fs.defaultFS' = 'hdfs://<your_nameservice_or_actually_HDFS_IP_and_port>'
+ ```
+4. Hive 1.x 的 orc 格式的表可能会遇到底层 orc 文件 schema 中列名为 `_col0`，`_col1`，`_col2`...
这类系统列名，此时需要在 catalog 配置中添加 `hive.version` 为 1.x.x，这样就会使用 hive 表中的列名进行映射。
+
+ ```sql
+ CREATE CATALOG hive PROPERTIES (
+ 'hive.version' = '1.x.x'
+ );
+ ```
+
+5. 使用Catalog查询表数据时发现与Hive Metastore相关的报错：`Invalid method
name`，需要设置`hive.version`参数。
+
+ ```sql
+ CREATE CATALOG hive PROPERTIES (
+ 'hive.version' = '2.x.x'
+ );
+ ```
+
+6. 查询 ORC 格式的表，FE 报错 `Could not obtain block` 或 `Caused by:
java.lang.NoSuchFieldError: types`
+
+ 对于 ORC 文件，在默认情况下，FE 会访问 HDFS 获取文件信息，进行文件切分。部分情况下，FE 可能无法访问到
HDFS。可以通过添加以下参数解决：
+
+ `"hive.exec.orc.split.strategy" = "BI"`
+
+ 其他选项：HYBRID（默认），ETL。
+
+7. 在hive上可以查到hudi表分区字段的值，但是在doris查不到。
+
+
doris和hive目前查询hudi的方式不一样，doris需要在hudi表结构的avsc文件里添加上分区字段,如果没加，就会导致doris查询partition_val为空（即使设置了hoodie.datasource.hive_sync.partition_fields=partition_val也不可以）
+ ```
+ {
+ "type": "record",
+ "name": "record",
+ "fields": [{
+ "name": "partition_val",
+ "type": [
+ "null",
+ "string"
+ ],
+ "doc": "Preset partition field, empty string when not partitioned",
+ "default": null
+ },
+ {
+ "name": "name",
+ "type": "string",
+ "doc": "名称"
+ },
+ {
+ "name": "create_time",
+ "type": "string",
+ "doc": "创建时间"
+ }
+ ]
+ }
+ ```
+8. 查询hive外表，遇到该报错：`java.lang.ClassNotFoundException: Class
com.hadoop.compression.lzo.LzoCodec not found`
+
+ 去hadoop环境搜索`hadoop-lzo-*.jar`放在`"${DORIS_HOME}/fe/lib/"`目录下并重启fe。
+
+ 从 2.0.2 版本起，可以将这个文件放置在BE的 `custom_lib/` 目录下（如不存在，手动创建即可），以防止升级集群时因为 lib
目录被替换而导致文件丢失。
+
+9. 创建hive表指定serde为
`org.apache.hadoop.hive.contrib.serde2.MultiDelimitserDe`，访问表时报错：`storage
schema reading not supported`
+
+ 在hive-site.xml文件中增加以下配置，并重启hms服务：
+
+ ```
+ <property>
+ <name>metastore.storage.schema.reader.impl</name>
+ <value>org.apache.hadoop.hive.metastore.SerDeStorageSchemaReader</value>
+ </property>
+ ```
+
+10. 报错：java.security.InvalidAlgorithmParameterException: the trustAnchors
parameter must be non-empty
+
+ FE日志中完整报错信息如下：
+ ```
+ org.apache.doris.common.UserException: errCode = 2, detailMessage = S3
list path failed. path=s3://bucket/part-*,msg=errors while get file status
listStatus on s3://bucket: com.amazonaws.SdkClientException: Unable to execute
HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty: Unable to execute HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be [...]
+ org.apache.doris.common.UserException: errCode = 2, detailMessage = S3
list path exception. path=s3://bucket/part-*, err: errCode = 2, detailMessage =
S3 list path failed. path=s3://bucket/part-*,msg=errors while get file status
listStatus on s3://bucket: com.amazonaws.SdkClientException: Unable to execute
HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty: Unable to execute HTTP request: Unexpected error: ja [...]
+ org.apache.hadoop.fs.s3a.AWSClientIOException: listStatus on s3://bucket:
com.amazonaws.SdkClientException: Unable to execute HTTP request: Unexpected
error: java.security.InvalidAlgorithmParameterException: the trustAnchors
parameter must be non-empty: Unable to execute HTTP request: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty
+ Caused by: com.amazonaws.SdkClientException: Unable to execute HTTP
request: Unexpected error: java.security.InvalidAlgorithmParameterException:
the trustAnchors parameter must be non-empty
+ Caused by: javax.net.ssl.SSLException: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty
+ Caused by: java.lang.RuntimeException: Unexpected error:
java.security.InvalidAlgorithmParameterException: the trustAnchors parameter
must be non-empty
+ Caused by: java.security.InvalidAlgorithmParameterException: the
trustAnchors parameter must be non-empty
+ ```
+
+ 尝试更新FE节点CA证书，使用 `update-ca-trust（CentOS/RockyLinux）`，然后重启FE进程即可。
+
+## HDFS
+
+1. 访问 HDFS 3.x 时报错：`java.lang.VerifyError: xxx`
+
+ 1.2.1 之前的版本中，Doris 依赖的 Hadoop 版本为 2.8。需更新至 2.10.2。或更新 Doris 至 1.2.2 之后的版本。
+
+2. 使用 Hedged Read 优化 HDFS 读取慢的问题。
+
+ 在某些情况下，HDFS 的负载较高可能导致读取某个 HDFS 上的数据副本的时间较长，从而拖慢整体的查询效率。HDFS Client 提供了
Hedged Read 功能。
+ 该功能可以在一个读请求超过一定阈值未返回时，启动另一个读线程读取同一份数据，哪个先返回就是用哪个结果。
+
+ 注意：该功能可能会增加 HDFS 集群的负载，请酌情使用。
+
+ 可以通过以下两种方式开启这个功能：
+
+ - 在创建 Catalog 的参数中指定：
+
+ ```
+ create catalog regression properties (
+ 'type'='hms',
+ 'hive.metastore.uris' = 'thrift://172.21.16.47:7004',
+ 'dfs.client.hedged.read.threadpool.size' = '128',
+ 'dfs.client.hedged.read.threshold.millis' = "500"
+ );
+ ```
+
+ `dfs.client.hedged.read.threadpool.size` 表示用于 Hedged Read 的线程数，这些线程由一个
HDFS Client 共享。通常情况下，针对一个 HDFS 集群，BE 节点会共享一个 HDFS Client。
+
+ `dfs.client.hedged.read.threshold.millis`
是读取阈值，单位毫秒。当一个读请求超过这个阈值未返回时，会触发 Hedged Read。
+
+
+ 开启后，可以在 Query Profile 中看到相关参数：
+
+ `TotalHedgedRead`: 发起 Hedged Read 的次数。
+
+ `HedgedReadWins`：Hedged Read 成功的次数（发起并且比原请求更快返回的次数）
+
+ 注意，这里的值是单个 HDFS Client 的累计值，而不是单个查询的数值。同一个 HDFS Client 会被多个查询复用。
+
+## DLF Catalog
+
+1. 使用DLF Catalog时，BE读在取JindoFS数据出现`Invalid
address`，需要在`/ets/hosts`中添加日志中出现的域名到IP的映射。
+
+2. 读取数据无权限时，使用`hadoop.username`属性指定有权限的用户。
+
+3. DLF
Catalog中的元数据和DLF保持一致。当使用DLF管理元数据时，Hive新导入的分区，可能未被DLF同步，导致出现DLF和Hive元数据不一致的情况，对此，需要先保证Hive元数据被DLF完全同步。
diff --git a/docs/zh-CN/docs/lakehouse/multi-catalog/dlf.md
b/docs/zh-CN/docs/lakehouse/multi-catalog/dlf.md
index 822ecff1bb0..43d30ff5e65 100644
--- a/docs/zh-CN/docs/lakehouse/multi-catalog/dlf.md
+++ b/docs/zh-CN/docs/lakehouse/multi-catalog/dlf.md
@@ -66,24 +66,26 @@ CREATE CATALOG dlf PROPERTIES (
### 使用开启了HDFS服务的OSS存储数据
-1.
确认OSS开启了HDFS服务。[开通并授权访问OSS-HDFS服务](https://help.aliyun.com/document_detail/419505.html?spm=a2c4g.2357115.0.i0)
-2. 下载SDK。[JindoData
SDK下载](https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/5.x/5.0.0-beta7/jindodata_download.md)
-3.
解压下载后的jindosdk.tar.gz，将其lib目录下的`jindo-core.jar、jindo-sdk.jar`放到`${DORIS_HOME}/fe/lib`和`${DORIS_HOME}/be/lib/java_extensions`目录下。
+1.
确认OSS开启了HDFS服务。[开通并授权访问OSS-HDFS服务](https://help.aliyun.com/document_detail/419505.html?spm=a2c4g.2357115.0.i0)。
+2. 下载SDK。[JindoData
SDK下载](https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/5.x/5.0.0-beta7/jindodata_download.md)。如果集群上已有SDK目录，忽略这一步。
+3. 解压下载后的jindosdk.tar.gz或者在集群上找到Jindo
SDK的目录，将其lib目录下的`jindo-core.jar、jindo-sdk.jar`放到`${DORIS_HOME}/fe/lib`和`${DORIS_HOME}/be/lib/java_extensions/preload-extensions`目录下。
4. 创建DLF Catalog，并配置`oss.hdfs.enabled`为`true`：
-```sql
-CREATE CATALOG dlf_oss_hdfs PROPERTIES (
- "type"="hms",
- "hive.metastore.type" = "dlf",
- "dlf.proxy.mode" = "DLF_ONLY",
- "dlf.endpoint" = "datalake-vpc.cn-beijing.aliyuncs.com",
- "dlf.region" = "cn-beijing",
- "dlf.uid" = "uid",
- "dlf.access_key" = "ak",
- "dlf.secret_key" = "sk",
- "oss.hdfs.enabled" = "true"
-);
-```
+ ```sql
+ CREATE CATALOG dlf_oss_hdfs PROPERTIES (
+ "type"="hms",
+ "hive.metastore.type" = "dlf",
+ "dlf.proxy.mode" = "DLF_ONLY",
+ "dlf.endpoint" = "datalake-vpc.cn-beijing.aliyuncs.com",
+ "dlf.region" = "cn-beijing",
+ "dlf.uid" = "uid",
+ "dlf.access_key" = "ak",
+ "dlf.secret_key" = "sk",
+ "oss.hdfs.enabled" = "true"
+ );
+ ```
+
+5. 当Jindo SDK版本与EMR集群上所用的版本不一致时，会出现`Plugin not found`的问题，需更换到对应版本。
### 访问DLF Iceberg表
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
index 6380dd2e6e0..850619cb246 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java
@@ -404,8 +404,10 @@ public class HiveMetaStoreCache {
if (uri.getScheme() != null) {
String scheme = uri.getScheme();
updateJobConf("fs." + scheme + ".impl.disable.cache",
"true");
- if (!scheme.equals("hdfs") && !scheme.equals("viewfs")) {
- updateJobConf("fs." + scheme + ".impl",
PropertyConverter.getHadoopFSImplByScheme(scheme));
+ if (jobConf.get("fs." + scheme + ".impl") == null) {
+ if (!scheme.equals("hdfs") &&
!scheme.equals("viewfs")) {
+ updateJobConf("fs." + scheme + ".impl",
PropertyConverter.getHadoopFSImplByScheme(scheme));
+ }
}
}
} catch (Exception e) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
index 174b0808bc2..1b0e3b6d972 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
@@ -332,8 +332,8 @@ public class PropertyConverter {
region + ".oss-dls.aliyuncs.com");
}
}
- ossProperties.put("fs.oss.impl",
"com.aliyun.emr.fs.oss.JindoOssFileSystem");
- ossProperties.put("fs.AbstractFileSystem.oss.impl",
"com.aliyun.emr.fs.oss.OSS");
+ ossProperties.put("fs.oss.impl",
"com.aliyun.jindodata.oss.JindoOssFileSystem");
+ ossProperties.put("fs.AbstractFileSystem.oss.impl",
"com.aliyun.jindodata.oss.OSS");
}
private static Map<String, String> convertToCOSProperties(Map<String,
String> props, CloudCredential credential) {
@@ -454,7 +454,8 @@ public class PropertyConverter {
if (!Strings.isNullOrEmpty(region)) {
boolean hdfsEnabled =
Boolean.parseBoolean(props.getOrDefault(OssProperties.OSS_HDFS_ENABLED,
"false"));
if (hdfsEnabled) {
- props.putIfAbsent("fs.oss.impl",
"com.aliyun.emr.fs.oss.JindoOssFileSystem");
+ props.putIfAbsent("fs.oss.impl",
"com.aliyun.jindodata.oss.JindoOssFileSystem");
+ props.put("fs.AbstractFileSystem.oss.impl",
"com.aliyun.jindodata.oss.OSS");
props.putIfAbsent(OssProperties.REGION, region);
// example: cn-shanghai.oss-dls.aliyuncs.com
// from
https://www.alibabacloud.com/help/en/e-mapreduce/latest/oss-kusisurumen
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
