Copilot commented on code in PR #15814:
URL: https://github.com/apache/iceberg/pull/15814#discussion_r3035749947
##########
docs/docs/spark-ddl.md:
##########
@@ -259,6 +280,11 @@ ALTER TABLE prod.db.sample
ADD COLUMN nested.new_column bigint FIRST;
```
+!!! warning "Hive Catalog Limitation"
+ When using a Hive catalog, adding a column with `FIRST` or `AFTER` may
fail due to HMS positional
+ schema validation. See the [Hive Catalog
Limitation](#hive-catalog-limitation) above for details
Review Comment:
The link target `#hive-catalog-limitation` likely won’t resolve because the
earlier “Hive Catalog Limitation” text is an admonition, not a heading/anchor
that most MkDocs setups include in the page TOC. Add an explicit anchor (e.g.,
a `### Hive catalog limitation {#hive-catalog-limitation}` heading or an HTML
anchor) so this cross-reference works reliably.
```suggestion
schema validation. See the warning above for details
```
##########
docs/docs/spark-ddl.md:
##########
@@ -302,6 +328,10 @@ ALTER TABLE prod.db.sample ALTER COLUMN col FIRST;
ALTER TABLE prod.db.sample ALTER COLUMN nested.col AFTER other_col;
```
+!!! warning "Hive Catalog Limitation"
+ When using a Hive catalog, reordering columns may fail due to HMS
positional schema validation.
+ See the [Hive Catalog Limitation](#hive-catalog-limitation) above for
details and workaround.
Review Comment:
This cross-reference uses `#hive-catalog-limitation`, but the page doesn’t
currently define that fragment as a heading/anchor (the earlier block is an
admonition). Add an explicit anchor at the definition site or adjust the link
to point to an existing heading so the link isn’t broken.
```suggestion
See the Hive Catalog Limitation note above for details and workaround.
```
##########
docs/docs/spark-ddl.md:
##########
@@ -323,6 +353,11 @@ ALTER TABLE prod.db.sample DROP COLUMN id;
ALTER TABLE prod.db.sample DROP COLUMN point.z;
```
+!!! warning "Hive Catalog Limitation"
+ When using a Hive catalog, dropping a non-last column may fail due to HMS
positional schema
+ validation. See the [Hive Catalog Limitation](#hive-catalog-limitation)
above for details
+ and workaround.
Review Comment:
This warning links to `#hive-catalog-limitation`, but the document doesn’t
define that fragment as a heading/anchor (the earlier “Hive Catalog Limitation”
block is an admonition). Please add an explicit anchor or update the link to an
existing section header to avoid a broken link.
```suggestion
validation. See the earlier Hive Catalog Limitation warning above for
details and
workaround.
```
##########
docs/docs/flink-ddl.md:
##########
@@ -45,6 +45,23 @@ The following properties can be set if using the Hive
catalog:
* `hive-conf-dir`: Path to a directory containing a `hive-site.xml`
configuration file which will be used to provide custom Hive configuration
values. The value of `hive.metastore.warehouse.dir` from
`<hive-conf-dir>/hive-site.xml` (or hive configure file from classpath) will be
overwritten with the `warehouse` value if setting both `hive-conf-dir` and
`warehouse` when creating iceberg catalog.
* `hadoop-conf-dir`: Path to a directory containing `core-site.xml` and
`hdfs-site.xml` configuration files which will be used to provide custom Hadoop
configuration values.
+!!! warning "Hive Catalog Limitation"
+ The Hive Metastore validates schema changes by comparing column types
**positionally**
+ (`hive.metastore.disallow.incompatible.col.type.changes`, default `true`).
When using a Hive catalog,
+ schema evolution operations that change column positions — such as
dropping a non-last column or
+ reordering columns — may fail regardless of which engine performs the
change (Spark, Flink Java API, etc.).
+
+ To work around this, disable the HMS schema compatibility check by setting
Review Comment:
The warning text uses the abbreviation “HMS” later, but it isn’t
introduced/defined here. Either introduce it as “Hive Metastore (HMS)” on first
use or avoid the acronym to keep the docs self-contained.
```suggestion
To work around this, disable the Hive Metastore (HMS) schema
compatibility check by setting
```
##########
docs/docs/flink-ddl.md:
##########
@@ -45,6 +45,23 @@ The following properties can be set if using the Hive
catalog:
* `hive-conf-dir`: Path to a directory containing a `hive-site.xml`
configuration file which will be used to provide custom Hive configuration
values. The value of `hive.metastore.warehouse.dir` from
`<hive-conf-dir>/hive-site.xml` (or hive configure file from classpath) will be
overwritten with the `warehouse` value if setting both `hive-conf-dir` and
`warehouse` when creating iceberg catalog.
* `hadoop-conf-dir`: Path to a directory containing `core-site.xml` and
`hdfs-site.xml` configuration files which will be used to provide custom Hadoop
configuration values.
+!!! warning "Hive Catalog Limitation"
+ The Hive Metastore validates schema changes by comparing column types
**positionally**
+ (`hive.metastore.disallow.incompatible.col.type.changes`, default `true`).
When using a Hive catalog,
+ schema evolution operations that change column positions — such as
dropping a non-last column or
+ reordering columns — may fail regardless of which engine performs the
change (Spark, Flink Java API, etc.).
+
+ To work around this, disable the HMS schema compatibility check by setting
+ `hive.metastore.disallow.incompatible.col.type.changes=false`:
+
+ - **Remote HMS:** Set this property in the HMS server's `hive-site.xml`.
+ - **Embedded HMS:** Add the equivalent property to the Hive catalog
configuration.
Review Comment:
The “Embedded HMS” workaround is unclear in the context of this Flink Hive
catalog section (where `uri` is required and configuration is typically
provided via `hive-conf-dir`/classpath). Consider rephrasing this to explicitly
instruct users to set the property in a `hive-site.xml` picked up via
`hive-conf-dir` (or classpath) rather than suggesting an “embedded” metastore
configuration path that may not apply.
```suggestion
- **When configuring the Hive catalog in Flink:** Set this property in a
`hive-site.xml`
that Flink picks up via `hive-conf-dir` or from the classpath.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
