bitsondatadev commented on code in PR #9819:
URL: https://github.com/apache/iceberg/pull/9819#discussion_r1507058032
##########
docs/docs/configuration.md:
##########
@@ -122,16 +122,16 @@ The value of these properties are not persisted as a part
of the table metadata.
Iceberg catalogs support using catalog properties to configure catalog
behaviors. Here is a list of commonly used catalog properties:
-| Property | Default | Description
|
-| --------------------------------- | ------------------ |
------------------------------------------------------ |
-| catalog-impl | null | a custom `Catalog`
implementation to use by an engine |
-| io-impl | null | a custom `FileIO`
implementation to use in a catalog |
-| warehouse | null | the root path of
the data warehouse |
-| uri | null | a URI string, such
as Hive metastore URI |
-| clients | 2 | client pool size
|
-| cache-enabled | true | Whether to cache
catalog entries |
-| cache.expiration-interval-ms | 30000 | How long catalog
entries are locally cached, in milliseconds; 0 disables caching, negative
values disable expiration |
-| metrics-reporter-impl | org.apache.iceberg.metrics.LoggingMetricsReporter |
Custom `MetricsReporter` implementation to use in a catalog. See the [Metrics
reporting](metrics-reporting) section for additional details |
+| Property | Default | Description
|
+| --------------------------------- | ------------------
|-----------------------------------------------------------------------------------------------------------------------------------------------|
+| catalog-impl | null | a custom `Catalog`
implementation to use by an engine
|
+| io-impl | null | a custom `FileIO`
implementation to use in a catalog
|
+| warehouse | null | the root path of
the data warehouse
|
+| uri | null | a URI string, such
as Hive metastore URI
|
+| clients | 2 | client pool size
|
+| cache-enabled | true | Whether to cache
catalog entries
|
+| cache.expiration-interval-ms | 30000 | How long catalog
entries are locally cached, in milliseconds; 0 disables caching, negative
values disable expiration |
+| metrics-reporter-impl | org.apache.iceberg.metrics.LoggingMetricsReporter |
Custom `MetricsReporter` implementation to use in a catalog. See the [Metrics
reporting](metrics-reporting.md) section for additional details |
Review Comment:
Don't forget these become relative links when you add `.md`. This navigates
to <https://iceberg.apache.org/docs/latest/configuration/metrics-reporting>,
not <https://iceberg.apache.org/docs/latest/metrics-reporting>
```suggestion
| metrics-reporter-impl | org.apache.iceberg.metrics.LoggingMetricsReporter
| Custom `MetricsReporter` implementation to use in a catalog. See the [Metrics
reporting](../metrics-reporting.md) section for additional details |
```
##########
docs/docs/flink-writes.md:
##########
@@ -267,8 +267,8 @@ Check out all the options here:
[write-options](flink-configuration.md#write-opt
## Notes
Flink streaming write jobs rely on snapshot summary to keep the last committed
checkpoint ID, and
-store uncommitted data as temporary files. Therefore, [expiring
snapshots](../tables/maintenance#expire-snapshots)
-and [deleting orphan files](../tables/maintenance#delete-orphan-files) could
possibly corrupt
+store uncommitted data as temporary files. Therefore, [expiring
snapshots](maintenance.md#expire-snapshots)
Review Comment:
Links to
<https://iceberg.apache.org/docs/latest/flink-writes/maintenance/#expire-snapshots>
and not to
<https://iceberg.apache.org/docs/latest/maintenance/#expire-snapshots>.
```suggestion
store uncommitted data as temporary files. Therefore, [expiring
snapshots](../maintenance.md#expire-snapshots)
```
##########
docs/docs/spark-configuration.md:
##########
@@ -176,19 +176,19 @@ df.write
.insertInto("catalog.db.table")
```
-| Spark option | Default | Description
|
-| ---------------------- | -------------------------- |
------------------------------------------------------------ |
-| write-format | Table write.format.default | File format to use for
this write operation; parquet, avro, or orc |
-| target-file-size-bytes | As per table property | Overrides this table's
write.target-file-size-bytes |
-| check-nullability | true | Sets the nullable
check on fields |
-| snapshot-property._custom-key_ | null | Adds an entry with
custom-key and corresponding value in the snapshot summary (the
`snapshot-property.` prefix is only required for DSv2) |
-| fanout-enabled | false | Overrides this table's
write.spark.fanout.enabled |
-| check-ordering | true | Checks if input schema and table schema
are same |
-| isolation-level | null | Desired isolation level for Dataframe overwrite
operations. `null` => no checks (for idempotent writes), `serializable` =>
check for concurrent inserts or deletes in destination partitions, `snapshot`
=> checks for concurrent deletes in destination partitions. |
-| validate-from-snapshot-id | null | If isolation level is set, id of base
snapshot from which to check concurrent write conflicts into a table. Should be
the snapshot before any reads from the table. Can be obtained via [Table
API](../../api#table-metadata) or [Snapshots
table](../spark-queries#snapshots). If null, the table's oldest known snapshot
is used. |
-| compression-codec | Table write.(fileformat).compression-codec |
Overrides this table's compression codec for this write |
-| compression-level | Table write.(fileformat).compression-level |
Overrides this table's compression level for Parquet and Avro tables for this
write |
-| compression-strategy | Table write.orc.compression-strategy |
Overrides this table's compression strategy for ORC tables for this write |
+| Spark option | Default | Description
|
+| ---------------------- | --------------------------
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| write-format | Table write.format.default | File format to use for
this write operation; parquet, avro, or orc
|
+| target-file-size-bytes | As per table property | Overrides this table's
write.target-file-size-bytes
|
+| check-nullability | true | Sets the nullable
check on fields
|
+| snapshot-property._custom-key_ | null | Adds an entry with
custom-key and corresponding value in the snapshot summary (the
`snapshot-property.` prefix is only required for DSv2)
|
+| fanout-enabled | false | Overrides this table's
write.spark.fanout.enabled
|
+| check-ordering | true | Checks if input schema and table schema
are same
|
+| isolation-level | null | Desired isolation level for Dataframe overwrite
operations. `null` => no checks (for idempotent writes), `serializable` =>
check for concurrent inserts or deletes in destination partitions, `snapshot`
=> checks for concurrent deletes in destination partitions.
|
+| validate-from-snapshot-id | null | If isolation level is set, id of base
snapshot from which to check concurrent write conflicts into a table. Should be
the snapshot before any reads from the table. Can be obtained via [Table
API](api.md#table-metadata) or [Snapshots table](spark-queries.md#snapshots).
If null, the table's oldest known snapshot is used. |
Review Comment:
`spark-queries is relative to spark-configuration so no need for `../`
```suggestion
| validate-from-snapshot-id | null | If isolation level is set, id of base
snapshot from which to check concurrent write conflicts into a table. Should be
the snapshot before any reads from the table. Can be obtained via [Table
API](../api.md#table-metadata) or [Snapshots
table](spark-queries.md#snapshots). If null, the table's oldest known snapshot
is used. |
```
##########
docs/docs/flink.md:
##########
@@ -22,22 +22,22 @@ title: "Flink Getting Started"
Apache Iceberg supports both [Apache Flink](https://flink.apache.org/)'s
DataStream API and Table API. See the [Multi-Engine
Support](../../multi-engine-support.md#apache-flink) page for the integration
of Apache Flink.
-| Feature support | Flink | Notes
|
-| -----------------------------------------------------------
|-------|----------------------------------------------------------------------------------------|
-| [SQL create catalog](#creating-catalogs-and-using-catalogs) | ✔️ |
|
-| [SQL create database](#create-database) | ✔️ |
|
-| [SQL create table](#create-table) | ✔️ |
|
-| [SQL create table like](#create-table-like) | ✔️ |
|
-| [SQL alter table](#alter-table) | ✔️ | Only
support altering table properties, column and partition changes are not
supported |
-| [SQL drop_table](#drop-table) | ✔️ |
|
-| [SQL select](#querying-with-sql) | ✔️ |
Support both streaming and batch mode
|
-| [SQL insert into](#insert-into) | ✔️ ️ |
Support both streaming and batch mode
|
-| [SQL insert overwrite](#insert-overwrite) | ✔️ ️ |
|
-| [DataStream read](#reading-with-datastream) | ✔️ ️ |
|
-| [DataStream append](#appending-data) | ✔️ ️ |
|
-| [DataStream overwrite](#overwrite-data) | ✔️ ️ |
|
-| [Metadata tables](#inspecting-tables) | ✔️ |
|
-| [Rewrite files action](#rewrite-files-action) | ✔️ ️ |
|
+| Feature support | Flink |
Notes
|
+|----------------------------------------------------------------|-------|----------------------------------------------------------------------------------------|
+| [SQL create catalog](flink-ddl.md#create-catalog) | ✔️ |
|
+| [SQL create database](flink-ddl.md#create-database) | ✔️ |
|
+| [SQL create table](flink-ddl.md#create-table) | ✔️ |
|
+| [SQL create table like](flink-ddl.md#create-table-like) | ✔️ |
|
+| [SQL alter table](flink-ddl.md#alter-table) | ✔️ |
Only support altering table properties, column and partition changes are not
supported |
+| [SQL drop_table](flink-ddl.md#drop-table) | ✔️ |
|
+| [SQL select](flink-queries.md#reading-with-sql) | ✔️ |
Support both streaming and batch mode
|
+| [SQL insert into](flink-writes.md#insert-into) | ✔️ ️ |
Support both streaming and batch mode
|
+| [SQL insert overwrite](flink-writes.md#insert-overwrite) | ✔️ ️ |
|
+| [DataStream read](flink-queries.md#reading-with-datastream) | ✔️ ️ |
|
+| [DataStream append](flink-writes.md#appending-data) | ✔️ ️ |
|
+| [DataStream overwrite](flink-writes.md#overwrite-data) | ✔️ ️ |
|
+| [Metadata tables](flink-queries.md#inspecting-tables) | ✔️ |
|
+| [Rewrite files action](flink-actions.md#rewrite-files-action) | ✔️ ️ |
|
Review Comment:
```suggestion
| [SQL create catalog](../flink-ddl.md#create-catalog) | ✔️
|
|
| [SQL create database](../flink-ddl.md#create-database) | ✔️
|
|
| [SQL create table](../flink-ddl.md#create-table) | ✔️
|
|
| [SQL create table like](../flink-ddl.md#create-table-like) | ✔️
|
|
| [SQL alter table](../flink-ddl.md#alter-table) | ✔️
| Only support altering table properties, column and partition changes are not
supported |
| [SQL drop_table](../flink-ddl.md#drop-table) | ✔️
|
|
| [SQL select](../flink-queries.md#reading-with-sql) | ✔️
| Support both streaming and batch mode
|
| [SQL insert into](../flink-writes.md#insert-into) | ✔️ ️
| Support both streaming and batch mode
|
| [SQL insert overwrite](../flink-writes.md#insert-overwrite) | ✔️ ️
|
|
| [DataStream read](../flink-queries.md#reading-with-datastream) | ✔️ ️
|
|
| [DataStream append](../flink-writes.md#appending-data) | ✔️ ️
|
|
| [DataStream overwrite](../flink-writes.md#overwrite-data) | ✔️ ️
|
|
| [Metadata tables](../flink-queries.md#inspecting-tables) | ✔️
|
|
| [Rewrite files action](../flink-actions.md#rewrite-files-action) | ✔️ ️
|
|
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
