wgtmac commented on code in PR #18062: URL: https://github.com/apache/iceberg/pull/18062#discussion_r4409157668
########## site/docs/status.md: ########## @@ -27,47 +27,100 @@ capabilities. This section lists the libraries that implement the Apache Iceberg specification. -- [Java](https://mvnrepository.com/artifact/org.apache.iceberg) -- [PyIceberg](https://pypi.org/project/pyiceberg/) -- [Rust](https://crates.io/crates/iceberg) -- [Go](https://pkg.go.dev/github.com/apache/iceberg-go) -- [C++](https://github.com/apache/iceberg-cpp/releases) +| Library | Released Version | +|---------|------------------| +| [Java](https://mvnrepository.com/artifact/org.apache.iceberg) | [1.12.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.12.0) | +| [PyIceberg](https://pypi.org/project/pyiceberg/) | [0.12.0](https://github.com/apache/iceberg-python/releases/tag/pyiceberg-0.12.0) | +| [Rust](https://crates.io/crates/iceberg) | [0.10.1](https://github.com/apache/iceberg-rust/releases/tag/v0.10.1) | +| [Go](https://pkg.go.dev/github.com/apache/iceberg-go) | [0.7.0](https://github.com/apache/iceberg-go/releases/tag/v0.7.0) | +| [C++](https://github.com/apache/iceberg-cpp/releases) | [0.4.0](https://github.com/apache/iceberg-cpp/releases/tag/v0.4.0) | ## Data Types -| Data Type | Java | PyIceberg | Rust | Go | C++ | -|----------------|------|-----------|------|----|-----| -| boolean | Y | Y | Y | Y | Y | -| int | Y | Y | Y | Y | Y | -| long | Y | Y | Y | Y | Y | -| float | Y | Y | Y | Y | Y | -| double | Y | Y | Y | Y | Y | -| decimal | Y | Y | Y | Y | Y | -| date | Y | Y | Y | Y | Y | -| time | Y | Y | Y | Y | Y | -| timestamp | Y | Y | Y | Y | Y | -| timestamptz | Y | Y | Y | Y | Y | -| timestamp_ns | Y | Y | Y | Y | N | -| timestamptz_ns | Y | Y | Y | Y | N | -| unknown | Y | Y | N | Y | N | -| string | Y | Y | Y | Y | Y | -| uuid | Y | Y | Y | Y | N | -| fixed | Y | Y | Y | Y | Y | -| binary | Y | Y | Y | Y | Y | -| variant | Y | Y | Y | Y | N | -| geometry | Y | N | N | N | N | -| geography | Y | N | N | N | N | -| list | Y | Y | Y | Y | Y | -| map | Y | Y | Y | Y | Y | -| struct | Y | Y | Y | Y | Y | +`Y` means the library supports the type in schemas and can both read and write its values in at least one supported data file +format. `N` means one or more of these capabilities is missing. For example, recognizing `geometry` in a schema without being +able to read and write geometry values is `N`. Table-format write support is listed under Table Write Operations. + +### Table Spec + +=== "V1 - V2" + | Data Type | Java | PyIceberg | Rust | Go | C++ | + |-------------|------|-----------|------|----|-----| + | boolean | Y | Y | Y | Y | Y | + | int | Y | Y | Y | Y | Y | + | long | Y | Y | Y | Y | Y | + | float | Y | Y | Y | Y | Y | + | double | Y | Y | Y | Y | Y | + | decimal | Y | Y | Y | Y | Y | + | date | Y | Y | Y | Y | Y | + | time | Y | Y | Y | Y | Y | + | timestamp | Y | Y | Y | Y | Y | + | timestamptz | Y | Y | Y | Y | Y | + | string | Y | Y | Y | Y | Y | + | uuid | Y | Y | Y | Y | Y | + | fixed | Y | Y | Y | Y | Y | + | binary | Y | Y | Y | Y | Y | + | list | Y | Y | Y | Y | Y | + | map | Y | Y | Y | Y | Y | + | struct | Y | Y | Y | Y | Y | + +=== "V3" + | Data Type | Java | PyIceberg | Rust | Go | C++ | + |----------------|------|-----------|------|----|-----| + | boolean | Y | Y | Y | Y | Y | + | int | Y | Y | Y | Y | Y | + | long | Y | Y | Y | Y | Y | + | float | Y | Y | Y | Y | Y | + | double | Y | Y | Y | Y | Y | + | decimal | Y | Y | Y | Y | Y | + | date | Y | Y | Y | Y | Y | + | time | Y | Y | Y | Y | Y | + | timestamp | Y | Y | Y | Y | Y | + | timestamptz | Y | Y | Y | Y | Y | + | timestamp_ns | Y | Y | Y | Y | Y | + | timestamptz_ns | Y | Y | Y | Y | Y | + | unknown | Y | Y | N | Y | Y | + | string | Y | Y | Y | Y | Y | + | uuid | Y | Y | Y | Y | Y | + | fixed | Y | Y | Y | Y | Y | + | binary | Y | Y | Y | Y | Y | + | variant | Y | N | N | Y | N | + | geometry | Y | N | N | Y | Y | + | geography | Y | N | N | Y | Y | + | list | Y | Y | Y | Y | Y | + | map | Y | Y | Y | Y | Y | + | struct | Y | Y | Y | Y | Y | + +## Table Metadata and Features + +### Table Spec + +=== "V1 - V2" + | Capability | Java | PyIceberg | Rust | Go | C++ | + |----------------------|------|-----------|------|----|-----| + | Read table metadata | Y | Y | Y | Y | Y | + | Write table metadata | Y | Y | Y | Y | Y | + +=== "V3" + | Capability | Java | PyIceberg | Rust | Go | C++ | + |---------------------------------|------|-----------|------|----|-----| + | Read table metadata | Y | Y | Y | Y | Y | + | Write table metadata | Y | N | Y | Y | Y | + | Read initial column defaults | Y | Y | Y | Y | Y | + | Read multi-argument transforms | N | N | N | N | N | + | Write multi-argument transforms | N | N | N | N | N | + | Read row lineage columns | Y | N | N | Y | Y | + | Write row lineage | Y | N | N | Y | Y | Review Comment: Rust 0.10.1 writes `first-row-id` in V3 manifest lists and snapshot row ranges. This should be Y. ########## site/docs/status.md: ########## @@ -207,8 +281,8 @@ This section lists the libraries that implement the Apache Iceberg specification | dropView | Y | Y | N | Y | N | | listView | Y | Y | N | Y | N | | viewExists | Y | Y | N | Y | N | -| replaceView | Y | N | N | N | N | -| renameView | Y | N | N | N | N | +| replaceView | Y | N | N | Y | N | Review Comment: PyIceberg 0.12.0 already implements `RestCatalog.create_view()`, so the `createView` cell above should be Y. ########## site/docs/status.md: ########## @@ -76,128 +129,149 @@ This section lists the libraries that implement the Apache Iceberg specification |-------------------|------|-----------|------|----|------| | Local Filesystem | Y | Y | Y | Y | Y | | Hadoop Filesystem | Y | Y | Y | Y | N | Review Comment: Go 0.7.0 has no Hadoop filesystem registration; its feature status lists only local/S3/GCS/Azure, so this should be N. ########## site/docs/status.md: ########## @@ -270,38 +344,38 @@ The sql catalog is a catalog backed by a sql database, which is called jdbc cata | Namespace Operation | Java | PyIceberg | Rust | Go | C++ | |---------------------------|------|-----------|------|----|-----| -| listNamespaces | Y | Y | Y | Y | N | -| createNamespace | Y | Y | Y | Y | N | -| dropNamespace | Y | Y | Y | Y | N | -| namespaceExists | Y | N | Y | Y | N | -| updateNamespaceProperties | Y | Y | Y | Y | N | -| loadNamespaceMetadata | Y | Y | Y | Y | N | +| listNamespaces | Y | Y | Y | Y | Y | +| createNamespace | Y | Y | Y | Y | Y | +| dropNamespace | Y | Y | Y | Y | Y | +| namespaceExists | Y | N | Y | Y | Y | Review Comment: PyIceberg provides `namespace_exists` for SQL and inherits it for Glue/Hive through `MetastoreCatalog`. These SQL, Glue, and Hive namespaceExists cells should all be Y. ########## site/docs/status.md: ########## @@ -27,47 +27,100 @@ capabilities. This section lists the libraries that implement the Apache Iceberg specification. -- [Java](https://mvnrepository.com/artifact/org.apache.iceberg) -- [PyIceberg](https://pypi.org/project/pyiceberg/) -- [Rust](https://crates.io/crates/iceberg) -- [Go](https://pkg.go.dev/github.com/apache/iceberg-go) -- [C++](https://github.com/apache/iceberg-cpp/releases) +| Library | Released Version | +|---------|------------------| +| [Java](https://mvnrepository.com/artifact/org.apache.iceberg) | [1.12.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.12.0) | +| [PyIceberg](https://pypi.org/project/pyiceberg/) | [0.12.0](https://github.com/apache/iceberg-python/releases/tag/pyiceberg-0.12.0) | +| [Rust](https://crates.io/crates/iceberg) | [0.10.1](https://github.com/apache/iceberg-rust/releases/tag/v0.10.1) | +| [Go](https://pkg.go.dev/github.com/apache/iceberg-go) | [0.7.0](https://github.com/apache/iceberg-go/releases/tag/v0.7.0) | +| [C++](https://github.com/apache/iceberg-cpp/releases) | [0.4.0](https://github.com/apache/iceberg-cpp/releases/tag/v0.4.0) | ## Data Types -| Data Type | Java | PyIceberg | Rust | Go | C++ | -|----------------|------|-----------|------|----|-----| -| boolean | Y | Y | Y | Y | Y | -| int | Y | Y | Y | Y | Y | -| long | Y | Y | Y | Y | Y | -| float | Y | Y | Y | Y | Y | -| double | Y | Y | Y | Y | Y | -| decimal | Y | Y | Y | Y | Y | -| date | Y | Y | Y | Y | Y | -| time | Y | Y | Y | Y | Y | -| timestamp | Y | Y | Y | Y | Y | -| timestamptz | Y | Y | Y | Y | Y | -| timestamp_ns | Y | Y | Y | Y | N | -| timestamptz_ns | Y | Y | Y | Y | N | -| unknown | Y | Y | N | Y | N | -| string | Y | Y | Y | Y | Y | -| uuid | Y | Y | Y | Y | N | -| fixed | Y | Y | Y | Y | Y | -| binary | Y | Y | Y | Y | Y | -| variant | Y | Y | Y | Y | N | -| geometry | Y | N | N | N | N | -| geography | Y | N | N | N | N | -| list | Y | Y | Y | Y | Y | -| map | Y | Y | Y | Y | Y | -| struct | Y | Y | Y | Y | Y | +`Y` means the library supports the type in schemas and can both read and write its values in at least one supported data file +format. `N` means one or more of these capabilities is missing. For example, recognizing `geometry` in a schema without being +able to read and write geometry values is `N`. Table-format write support is listed under Table Write Operations. + +### Table Spec + +=== "V1 - V2" + | Data Type | Java | PyIceberg | Rust | Go | C++ | + |-------------|------|-----------|------|----|-----| + | boolean | Y | Y | Y | Y | Y | + | int | Y | Y | Y | Y | Y | + | long | Y | Y | Y | Y | Y | + | float | Y | Y | Y | Y | Y | + | double | Y | Y | Y | Y | Y | + | decimal | Y | Y | Y | Y | Y | + | date | Y | Y | Y | Y | Y | + | time | Y | Y | Y | Y | Y | + | timestamp | Y | Y | Y | Y | Y | + | timestamptz | Y | Y | Y | Y | Y | + | string | Y | Y | Y | Y | Y | + | uuid | Y | Y | Y | Y | Y | + | fixed | Y | Y | Y | Y | Y | + | binary | Y | Y | Y | Y | Y | + | list | Y | Y | Y | Y | Y | + | map | Y | Y | Y | Y | Y | + | struct | Y | Y | Y | Y | Y | + +=== "V3" + | Data Type | Java | PyIceberg | Rust | Go | C++ | + |----------------|------|-----------|------|----|-----| + | boolean | Y | Y | Y | Y | Y | + | int | Y | Y | Y | Y | Y | + | long | Y | Y | Y | Y | Y | + | float | Y | Y | Y | Y | Y | + | double | Y | Y | Y | Y | Y | + | decimal | Y | Y | Y | Y | Y | + | date | Y | Y | Y | Y | Y | + | time | Y | Y | Y | Y | Y | + | timestamp | Y | Y | Y | Y | Y | + | timestamptz | Y | Y | Y | Y | Y | + | timestamp_ns | Y | Y | Y | Y | Y | + | timestamptz_ns | Y | Y | Y | Y | Y | + | unknown | Y | Y | N | Y | Y | + | string | Y | Y | Y | Y | Y | + | uuid | Y | Y | Y | Y | Y | + | fixed | Y | Y | Y | Y | Y | + | binary | Y | Y | Y | Y | Y | + | variant | Y | N | N | Y | N | + | geometry | Y | N | N | Y | Y | + | geography | Y | N | N | Y | Y | + | list | Y | Y | Y | Y | Y | + | map | Y | Y | Y | Y | Y | + | struct | Y | Y | Y | Y | Y | + +## Table Metadata and Features + +### Table Spec + +=== "V1 - V2" + | Capability | Java | PyIceberg | Rust | Go | C++ | + |----------------------|------|-----------|------|----|-----| + | Read table metadata | Y | Y | Y | Y | Y | + | Write table metadata | Y | Y | Y | Y | Y | + +=== "V3" + | Capability | Java | PyIceberg | Rust | Go | C++ | + |---------------------------------|------|-----------|------|----|-----| + | Read table metadata | Y | Y | Y | Y | Y | + | Write table metadata | Y | N | Y | Y | Y | + | Read initial column defaults | Y | Y | Y | Y | Y | + | Read multi-argument transforms | N | N | N | N | N | + | Write multi-argument transforms | N | N | N | N | N | + | Read row lineage columns | Y | N | N | Y | Y | + | Write row lineage | Y | N | N | Y | Y | + | Read encrypted tables | Y | N | N | N | N | + | Write encrypted tables | Y | N | N | N | N | ## Data File Formats | Format | Java | PyIceberg | Rust | Go | C++ | |---------|------|-----------|------|----|-----| | Parquet | Y | Y | Y | Y | Y | | ORC | Y | N | N | N | N | -| Puffin | Y | N | N | N | N | +| Puffin | Y | N | N | Y | Y | Review Comment: Rust 0.10.1 exposes working `PuffinReader` and `PuffinWriter`. The Puffin format cell should be Y. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
