This is an automated email from the ASF dual-hosted git repository.
suxiaogang223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 0c08c3af933 [fix](fe) Surface Hive table comments from HMS (#69684)
0c08c3af933 is described below
commit 0c08c3af93341b3d68a84249d13766276c644e00
Author: Socrates <[email protected]>
AuthorDate: Wed Oct 28 10:02:05 2026 +0800
[fix](fe) Surface Hive table comments from HMS (#69684)
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary: A Hive catalog already exposes
`information_schema.tables` and `information_schema.columns`, and
ordinary Hive column comments already flow from HMS. However,
`HiveConnectorMetadata.getTableComment()` only read HMS
`parameters["comment"]` for Iceberg-on-HMS tables. It returned an empty
string for ordinary Hive tables even when HMS stored a table comment.
Read that existing HMS parameter for every table format, preserving the
empty result for absent comments or unreadable tables. This adds no
metadata RPC, network destination, or new privilege path.
### Release note
Hive catalog `information_schema.tables.TABLE_COMMENT` and `SHOW TABLE
STATUS` now display table comments stored in HMS.
### Check List (For Author)
- Test: Unit Test
- `./run-fe-ut.sh --run
'org.apache.doris.connector.hive.HiveConnectorMetadataTableCommentTest,org.apache.doris.connector.hive.HiveConnectorMetadataSchemaTest'`:
31 passed; FE reactor build succeeded.
- `mvn -f fe/pom.xml -pl fe-connector/fe-connector-hive checkstyle:check
-DskipTests -q`: passed.
- `build-support/check-fe-connector-imports.sh` and `git diff --check`:
passed.
- `./build.sh --fe` could not start on this machine because LLVM 20 is
not installed. External cluster regression was not run locally.
- Behavior changed: Yes. Plain Hive table comments now appear in table
metadata views.
- Does this need documentation: No
### Self-review
- Scope: Only the Hive connector's table comment lookup changes.
Existing column comment propagation remains intact and is covered by a
schema unit test, including partition columns.
- Compatibility: Iceberg-on-HMS comments and empty or unreadable table
behavior retain their existing tests. No SPI, Thrift, persistence, or
catalog property changes.
- Metadata lifecycle: The lookup uses the existing cached HMS table, so
comment changes follow the existing table metadata refresh path.
---
.../connector/hive/HiveConnectorMetadata.java | 35 +++++-----------------
.../hive/HiveConnectorMetadataSchemaTest.java | 16 ++++++++++
.../HiveConnectorMetadataTableCommentTest.java | 24 ++++++++++-----
3 files changed, 40 insertions(+), 35 deletions(-)
diff --git
a/fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveConnectorMetadata.java
b/fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveConnectorMetadata.java
index 4c8f56c58b5..a588b804927 100644
---
a/fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveConnectorMetadata.java
+++
b/fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveConnectorMetadata.java
@@ -124,11 +124,8 @@ public class HiveConnectorMetadata implements
ConnectorMetadata {
private static final Logger LOG =
LogManager.getLogger(HiveConnectorMetadata.class);
static final String TABLE_INFO_NAMESPACE = "hms.table-info";
- /**
- * The HMS table parameter iceberg writes its table comment into (mirrored
from the iceberg table property
- * of the same name on every commit).
- */
- private static final String ICEBERG_TABLE_COMMENT_PARAM = "comment";
+ /** The HMS table parameter that stores a table comment, including
comments mirrored from Iceberg. */
+ private static final String HMS_TABLE_COMMENT_PARAM = "comment";
// ===== CREATE TABLE / CREATE DATABASE statement property keys =====
// These are statement properties, not catalog properties: they arrive on
the DDL request, and this class
@@ -777,33 +774,17 @@ public class HiveConnectorMetadata implements
ConnectorMetadata {
}
/**
- * Returns the table comment. Overridden here for one reason: an
iceberg-on-HMS table would otherwise report
- * no comment at all.
- *
- * <p>Every other foreign-table operation is diverted to the embedded
iceberg sibling by the concrete handle
- * type, but this method addresses a table by NAME and never receives a
handle, so there is nothing to
- * discriminate on. It is answered directly instead: iceberg's HMS catalog
mirrors the table's iceberg
- * properties into the HMS table parameters on every commit, {@code
comment} included — the same
- * parameter {@code HiveShowCreateTableRenderer} lifts into the {@code
COMMENT} clause — so the single
- * (cached) {@code getTable} this connector already performs carries it,
with no sibling build and no
- * iceberg metadata load. Without this, the same table shows its comment
through a dedicated iceberg catalog
- * and shows nothing through an HMS gateway catalog, in {@code SHOW CREATE
TABLE},
- * {@code information_schema.tables.TABLE_COMMENT} and {@code SHOW TABLE
STATUS}.</p>
- *
- * <p>Deliberately restricted to iceberg tables: a plain hive table keeps
the empty comment that legacy
- * {@code HMSExternalTable.getComment} returned, so nothing about plain
hive changes. A missing or
- * unreadable table degrades to the empty comment rather than failing the
caller, matching
- * {@link #viewExists} — this is an opportunistic display value, not
a user-requested operation.</p>
+ * Returns the comment stored in the HMS table parameters for Hive and
foreign tables alike. This uses the
+ * same {@code comment} parameter as {@link HiveShowCreateTableRenderer},
so a Hive table's comment is also
+ * visible through {@code information_schema.tables} and {@code SHOW TABLE
STATUS}. Iceberg-on-HMS tables
+ * mirror their comment into this parameter, allowing the gateway to
answer without loading the sibling
+ * connector. A missing or unreadable table retains the empty display
value.
*/
@Override
public String getTableComment(ConnectorSession session, String dbName,
String tableName) {
try {
HmsTableInfo tableInfo = hmsClient.getTable(dbName, tableName);
- if (HiveTableFormatDetector.detect(tableInfo) !=
HiveTableType.ICEBERG) {
- return "";
- }
- Map<String, String> params = tableInfo.getParameters();
- String comment = params == null ? null :
params.get(ICEBERG_TABLE_COMMENT_PARAM);
+ String comment =
tableInfo.getParameters().get(HMS_TABLE_COMMENT_PARAM);
return comment == null ? "" : comment;
} catch (HmsClientException e) {
LOG.debug("Table comment lookup: '{}.{}' not readable: {}",
dbName, tableName, e.getMessage());
diff --git
a/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataSchemaTest.java
b/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataSchemaTest.java
index c72faf5ce69..57fee93e387 100644
---
a/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataSchemaTest.java
+++
b/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataSchemaTest.java
@@ -74,6 +74,22 @@ public class HiveConnectorMetadataSchemaTest {
return new ConnectorColumn(name, ConnectorType.of(typeName), null,
true, null);
}
+ @Test
+ public void testHiveColumnCommentsIncludePartitionColumns() {
+ HmsTableInfo table = HmsTableInfo.builder()
+ .dbName("db").tableName("t")
+ .inputFormat(PARQUET_INPUT_FORMAT)
+ .columns(Collections.singletonList(
+ new ConnectorColumn("id", ConnectorType.of("INT"),
"编号's note", true, null)))
+ .partitionKeys(Collections.singletonList(
+ new ConnectorColumn("dt", ConnectorType.of("STRING"),
"分区日期", true, null)))
+ .build();
+
+ ConnectorTableSchema schema = schemaOf(table);
+ Assertions.assertEquals("编号's note", columnByName(schema,
"id").getComment());
+ Assertions.assertEquals("分区日期", columnByName(schema,
"dt").getComment());
+ }
+
private static HmsTableInfo.Builder partitionedTable() {
return HmsTableInfo.builder()
.dbName("db").tableName("t")
diff --git
a/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataTableCommentTest.java
b/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataTableCommentTest.java
index a907ce56d85..a964346f3aa 100644
---
a/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataTableCommentTest.java
+++
b/fe/fe-connector/fe-connector-hive/src/test/java/org/apache/doris/connector/hive/HiveConnectorMetadataTableCommentTest.java
@@ -34,7 +34,7 @@ import java.util.List;
import java.util.Map;
/**
- * Pins {@link HiveConnectorMetadata#getTableComment} for an iceberg-on-HMS
table read through the gateway.
+ * Pins {@link HiveConnectorMetadata#getTableComment} for Hive and Iceberg
tables read through HMS.
*
* <p><b>WHY this matters:</b> the comment accessor takes a database and table
NAME, never a
* {@code ConnectorTableHandle}, so the gateway cannot route it to the iceberg
sibling the way it routes every
@@ -45,8 +45,8 @@ import java.util.Map;
* table. Iceberg's HMS catalog mirrors the comment into the HMS table
parameters on every commit, so the
* gateway answers from the metastore table it already reads.</p>
*
- * <p>The plain-hive assertions are the other half of the contract: a plain
hive table must keep the empty
- * comment legacy returned, so this change cannot alter what any existing hive
deployment displays.</p>
+ * <p>Plain Hive tables use the same HMS {@code comment} parameter. Their
comments must appear in the
+ * gateway's information schema and table status instead of being hidden by
the legacy empty result.</p>
*/
public class HiveConnectorMetadataTableCommentTest {
@@ -73,18 +73,26 @@ public class HiveConnectorMetadataTableCommentTest {
}
@Test
- public void plainHiveTableKeepsItsLegacyEmptyComment() {
+ public void plainHiveTableReportsItsComment() {
Map<String, String> params = new HashMap<>();
- params.put("comment", "a hive table comment that legacy never
surfaced");
+ params.put("comment", "客户表's comment");
HmsTableInfo hive = HmsTableInfo.builder()
.dbName("db").tableName("t").tableType("MANAGED_TABLE")
.inputFormat("org.apache.hadoop.mapred.TextInputFormat")
.parameters(params)
.build();
- Assertions.assertEquals("", metadataFor(hive).getTableComment(session,
"db", "t"),
- "a plain hive table must keep the empty comment legacy
HMSExternalTable returned; surfacing it "
- + "here would change what every existing hive
deployment displays");
+ Assertions.assertEquals("客户表's comment",
metadataFor(hive).getTableComment(session, "db", "t"));
+ }
+
+ @Test
+ public void plainHiveTableWithoutACommentReportsEmpty() {
+ HmsTableInfo hive = HmsTableInfo.builder()
+ .dbName("db").tableName("t").tableType("EXTERNAL_TABLE")
+ .inputFormat("org.apache.hadoop.mapred.TextInputFormat")
+ .build();
+
+ Assertions.assertEquals("", metadataFor(hive).getTableComment(session,
"db", "t"));
}
@Test
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
