mxm commented on code in PR #18313: URL: https://github.com/apache/iceberg/pull/18313#discussion_r4157327358
########## site/docs/releases.md: ########## @@ -66,6 +67,219 @@ To add a dependency on Iceberg in Maven, add the following to your `pom.xml`: </dependencies> ``` +### 1.12.0 release + +Apache Iceberg 1.12.0 was released on September 29, 2026. + +The 1.12.0 release contains bug fixes and new features. For full release notes visit [Github](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.12.0) + +* Deprecation / End of Support + - Spark: Remove Spark 3.4 support ([\#14122](https://github.com/apache/iceberg/pull/14122)) + - Flink: Remove support for Flink 2.0 + - Remove deprecated methods and fields scheduled for 1.12.0 removal ([\#17700](https://github.com/apache/iceberg/pull/17700)) + - Remove deprecated `DataReader` in favor of `PlannedDataReader` ([\#17699](https://github.com/apache/iceberg/pull/17699)) + - Data: Remove deprecated `GenericAppenderFactory` and `BaseFileWriterFactory` ([\#17696](https://github.com/apache/iceberg/pull/17696)) + - Spark: Remove deprecated `SparkFilters` ([\#17702](https://github.com/apache/iceberg/pull/17702)) + - Flink: Remove deprecated `RewriteDataFiles.Builder.filter(Expression)` ([\#17624](https://github.com/apache/iceberg/pull/17624)) + - Core, Data, Spark, Flink: Remove position delete files with row data ([\#17706](https://github.com/apache/iceberg/pull/17706)) + - AWS: Remove deprecated S3 signer classes and properties ([\#17627](https://github.com/apache/iceberg/pull/17627)) + - Spark 4.0, 4.1: Remove deprecated `SparkTableUtil` methods ([\#17703](https://github.com/apache/iceberg/pull/17703)) + - Spark: Remove deprecated `SparkReadConf`, `SparkWriteConf`, and `SparkSchemaUtil` methods ([\#17626](https://github.com/apache/iceberg/pull/17626)) + - Core: Remove deprecated REST namespace encoding helpers ([\#17697](https://github.com/apache/iceberg/pull/17697)) + - Core: Remove deprecated `HadoopFileIO(SerializableSupplier)` constructor ([\#17704](https://github.com/apache/iceberg/pull/17704)) + - Core, ORC: Remove deprecated partition stats read functionality ([\#14998](https://github.com/apache/iceberg/pull/14998)) + - Kafka Connect: Remove deprecated `TableReference` and `IcebergWriterResult` members ([\#17623](https://github.com/apache/iceberg/pull/17623)) + - BigQuery: Remove deprecated catalog property constants ([\#17625](https://github.com/apache/iceberg/pull/17625)) +* Behavior change + - `GeometryType` and `GeographyType` `toString()` now always include the resolved CRS, and for geography the edge algorithm: `geometry` prints as `geometry(OGC:CRS84)` and `geography` as `geography(OGC:CRS84, spherical)` ([\#16765](https://github.com/apache/iceberg/pull/16765)). + Previously, default instances printed the bare type name (`geometry` / `geography`). + - The default AWS SDK HTTP client migrated to Apache HttpClient 5. Users who provide AWS dependencies separately must switch from `software.amazon.awssdk:apache-client` to `software.amazon.awssdk:apache5-client` ([\#18195](https://github.com/apache/iceberg/pull/18195)). + - The REST client now retries POST requests carrying an `Idempotency-Key` on retriable errors (408, 500, 502, 503, 504) ([\#17947](https://github.com/apache/iceberg/pull/17947)). +* Spec + - Add spec for expressions ([\#16652](https://github.com/apache/iceberg/pull/16652)) + - Add finer grained read restrictions as part of loadTable ([\#13879](https://github.com/apache/iceberg/pull/13879)) + - Add relative paths to the v4 spec ([\#15630](https://github.com/apache/iceberg/pull/15630)) + - Add content stats to the spec ([\#14234](https://github.com/apache/iceberg/pull/14234)) + - Add optional specific-name to UDF definition model ([\#16727](https://github.com/apache/iceberg/pull/16727)) + - Clarify variant type classification and primitive type scoping ([\#16836](https://github.com/apache/iceberg/pull/16836)) + - Clarify decimal type serialization ([\#16798](https://github.com/apache/iceberg/pull/16798)) + - Clarify content file uniqueness within a snapshot ([\#17198](https://github.com/apache/iceberg/pull/17198)) +* API + - Single-value binary serialization for geometry and geography ([\#16607](https://github.com/apache/iceberg/pull/16607)) + - Define `RepairTable` action interface ([\#17399](https://github.com/apache/iceberg/pull/17399)) + - Make variant classes serializable ([\#17260](https://github.com/apache/iceberg/pull/17260)) + - Harden variant binary parsing against malformed input ([\#16568](https://github.com/apache/iceberg/pull/16568)) + - Add `CatalogObjectIdentifier` ([\#16160](https://github.com/apache/iceberg/pull/16160)) + - Implement project() for partition statistics scan API ([\#16569](https://github.com/apache/iceberg/pull/16569)) + - Implement filter() for partition statistics scan API ([\#16582](https://github.com/apache/iceberg/pull/16582)) + - Guard against null in IN/NOT_IN predicates to avoid NPE ([\#17014](https://github.com/apache/iceberg/pull/17014)) +* Core + - Read and write geometry and geography values in Avro ([\#17119](https://github.com/apache/iceberg/pull/17119)) + - Add a read-only Mumbling bitmap implementation ([\#16747](https://github.com/apache/iceberg/pull/16747)) + - Add v4 manifest reader ([\#16958](https://github.com/apache/iceberg/pull/16958)) + - Resolve relative paths in V4 manifest reader ([\#17434](https://github.com/apache/iceberg/pull/17434)) + - Add V4 location relativization utilities ([\#16174](https://github.com/apache/iceberg/pull/16174)) + - Add v4 TrackedFileAdapters to bridge Data and Delete Files ([\#16100](https://github.com/apache/iceberg/pull/16100)) + - Add `format_version` field to TrackedFile ([\#16952](https://github.com/apache/iceberg/pull/16952)) + - Extend V4 DeletionVector with key_metadata field ([\#17438](https://github.com/apache/iceberg/pull/17438)) + - Expose co-located deletion vector through `DataFile` ([\#17928](https://github.com/apache/iceberg/pull/17928)) + - Allow writing Parquet and Avro manifests in the V4 layout ([\#15634](https://github.com/apache/iceberg/pull/15634)) + - Add ManifestFile adapter for v4 tracked files ([\#17932](https://github.com/apache/iceberg/pull/17932)) + - Add `EagerInputFile` and `EagerInputStream` to buffer files below a size threshold ([\#16729](https://github.com/apache/iceberg/pull/16729)) + - Cache manifest list files in manifest content cache ([\#16762](https://github.com/apache/iceberg/pull/16762)) + - Parquet per column dictionary encoding ([\#16713](https://github.com/apache/iceberg/pull/16713)) + - Disallow setting main branch ref to a tag ([\#16753](https://github.com/apache/iceberg/pull/16753)) + - Encrypting IO as a `DelegateFileIO` ([\#14876](https://github.com/apache/iceberg/pull/14876)) + - Preserve DV encryption metadata in merges ([\#15911](https://github.com/apache/iceberg/pull/15911)) + - Commit manifest list encryption keys with the snapshot that uses them ([\#17984](https://github.com/apache/iceberg/pull/17984)) + - Add a scan-based action to remove dangling delete files ([\#15727](https://github.com/apache/iceberg/pull/15727)) + - Fix thread conflict when deleting duplicate files in manifest ([\#16686](https://github.com/apache/iceberg/pull/16686)) + - Fix row lineage last updated sequence inheritance ([\#17039](https://github.com/apache/iceberg/pull/17039)) + - Fix time-travel snapshot lookup to not assume snapshot-log order ([\#17360](https://github.com/apache/iceberg/pull/17360)) + - Fix pruning for negated all_manifests filters ([\#17346](https://github.com/apache/iceberg/pull/17346)) + - Fix incorrect delete manifest pruning in entries metadata tables ([\#17440](https://github.com/apache/iceberg/pull/17440)) + - Fix delete file references for DVs in the same Puffin file ([\#17497](https://github.com/apache/iceberg/pull/17497)) + - Fix commit validation when a data file has multiple DVs across snapshots ([\#17764](https://github.com/apache/iceberg/pull/17764)) + - Read delete manifests correctly during snapshot expiration ([\#17763](https://github.com/apache/iceberg/pull/17763)) + - Fail scans when position deletes or DVs don't match the data file partition ([\#16957](https://github.com/apache/iceberg/pull/16957)) + - Fix Z-order byte encoding for floating-point values ([\#17071](https://github.com/apache/iceberg/pull/17071)) + - Preserve manifest content pruning when ignoring residuals ([\#17443](https://github.com/apache/iceberg/pull/17443)) + - Fix `SerializableTable.sortOrders()` throwing on historical sort orders with dropped fields ([\#16521](https://github.com/apache/iceberg/pull/16521)) + - Fix `RESTMetricsReporter.report()` blocking the calling thread ([\#16695](https://github.com/apache/iceberg/pull/16695)) + - Read catalog labels on the load table and view responses ([\#18045](https://github.com/apache/iceberg/pull/18045)) + - Expose catalog labels on the loaded table via `SupportsLabels` ([\#18046](https://github.com/apache/iceberg/pull/18046)) + - Add `max-file-group-input-files` to valid rewrite options ([\#17544](https://github.com/apache/iceberg/pull/17544)) +* Arrow + - Fix dict-encoded VARCHAR/VARBINARY read for direct ByteBuffers ([\#17055](https://github.com/apache/iceberg/pull/17055)) + - Fix direct memory leak in row lineage vectorized readers ([\#17296](https://github.com/apache/iceberg/pull/17296)) + - Fix `ClassCastException` in the vectorized reader on int-to-long promotion ([\#16343](https://github.com/apache/iceberg/pull/16343)) + - Fix vectorized reads of decimal columns with default values ([\#16501](https://github.com/apache/iceberg/pull/16501)) + - Fix truncation of decimals with precision larger than 18 ([\#16627](https://github.com/apache/iceberg/pull/16627)) + - Fix vectorized read of all-null DELTA-encoded Parquet pages ([\#17017](https://github.com/apache/iceberg/pull/17017)) + - Fix int96 timestamp offset in the Arrow dictionary decode ([\#16435](https://github.com/apache/iceberg/pull/16435)) +* Parquet + - Type uniformity for variant shredding ([\#17424](https://github.com/apache/iceberg/pull/17424)) + - Read and write geometry and geography WKB values ([\#16982](https://github.com/apache/iceberg/pull/16982)) + - Honor column metrics truncate length for variant shredded bounds ([\#17342](https://github.com/apache/iceberg/pull/17342)) + - Fix variant STRING metrics bounds to use UTF-8 byte order ([\#17397](https://github.com/apache/iceberg/pull/17397)) + - Fix variant BINARY upper bound to truncate up ([\#16880](https://github.com/apache/iceberg/pull/16880)) + - Fix variant metrics crash when value column has no stats ([\#16585](https://github.com/apache/iceberg/pull/16585)) + - Fix variant shredding of large decimals (precision > 18) ([\#17002](https://github.com/apache/iceberg/pull/17002)) + - Add adaptive bloom filter sizing (PARQUET-2254) ([\#16363](https://github.com/apache/iceberg/pull/16363)) + - Add opt-in uncompressed row group size tracking ([\#16327](https://github.com/apache/iceberg/pull/16327)) + - Fix timestamp_ns and timestamptz_ns predicate pushdown ([\#16619](https://github.com/apache/iceberg/pull/16619)) + - Fix nested initial default applied when ancestor struct is null ([\#17320](https://github.com/apache/iceberg/pull/17320)) + - Fix incorrect pruning for nested fields in optional structs ([\#18069](https://github.com/apache/iceberg/pull/18069)) + - Fix `notStartsWith` skipping row groups with null values ([\#17656](https://github.com/apache/iceberg/pull/17656)) + - Fix null counting for Parquet files without null_count stats ([\#17557](https://github.com/apache/iceberg/pull/17557)) + - Fix initial-default rows dropped when filtering on the defaulted column ([\#16692](https://github.com/apache/iceberg/pull/16692)) +* ORC + - Fix lower/upper bounds for timestamp_ns columns in OrcMetrics ([\#16922](https://github.com/apache/iceberg/pull/16922)) + - Fix garbled exception message for invalid timestamp unit attribute ([\#17098](https://github.com/apache/iceberg/pull/17098)) + - Fix timestamp nano predicate pushdown ([\#17750](https://github.com/apache/iceberg/pull/17750)) + - Fix filter pushdown on tables with a variant column ([\#17998](https://github.com/apache/iceberg/pull/17998)) +* Spark + - Spark 4.0, 4.1: Add vectorized Parquet reads for unshredded variant columns ([\#16292](https://github.com/apache/iceberg/pull/16292)) + - Spark 4.1: Add Hilbert-curve clustering strategy for `rewrite_data_files` ([\#16827](https://github.com/apache/iceberg/pull/16827)) + - Spark 4.1: Read and write geometry and geography values in Parquet ([\#17073](https://github.com/apache/iceberg/pull/17073)) + - Spark 4.1: Map geometry and geography Spark types ([\#16851](https://github.com/apache/iceberg/pull/16851)) + - Add `rest-catalog-purge` property to delegate DROP TABLE PURGE to REST catalogs ([\#15614](https://github.com/apache/iceberg/pull/15614)) + - Add session-level split size override ([\#16154](https://github.com/apache/iceberg/pull/16154)) + - Add ignore_missing_files to migrate procedure ([\#16643](https://github.com/apache/iceberg/pull/16643)) + - Add ignore_missing_files to snapshot procedure ([\#16710](https://github.com/apache/iceberg/pull/16710)) + - Return session catalog views ([\#16845](https://github.com/apache/iceberg/pull/16845)) + - Spark 4.1: Implement listTableSummaries ([\#16891](https://github.com/apache/iceberg/pull/16891)) + - Spark 3.5, 4.0, 4.1: Add streaming merge-append write config ([\#17347](https://github.com/apache/iceberg/pull/17347), [\#17403](https://github.com/apache/iceberg/pull/17403)) + - Spark 3.5, 4.0, 4.1: Add rewrite option to enable executor cache for delete files ([\#17868](https://github.com/apache/iceberg/pull/17868)) + - Fix Z-order NPE on null booleans and case-insensitive column resolution ([\#17669](https://github.com/apache/iceberg/pull/17669)) + - Fix time-travel filter on renamed columns in distributed planning mode ([\#16523](https://github.com/apache/iceberg/pull/16523)) + - Fix first row ID carry-over for manifest rewrite ([\#16699](https://github.com/apache/iceberg/pull/16699)) + - Spark 3.5, 4.0: Add `sort_by` parameter to the `rewrite_manifests` procedure ([\#18065](https://github.com/apache/iceberg/pull/18065)) + - Encrypt manifests written by `rewrite_manifests` ([\#17987](https://github.com/apache/iceberg/pull/17987)) +* Flink + - Add Flink 2.2 and 2.3 support ([\#17849](https://github.com/apache/iceberg/pull/17849)) + - Add a Flink maintenance task to convert equality deletes into deletion vectors (`ConvertEqualityDeletes`) ([\#16831](https://github.com/apache/iceberg/pull/16831), [\#16844](https://github.com/apache/iceberg/pull/16844), [\#16858](https://github.com/apache/iceberg/pull/16858), [\#16874](https://github.com/apache/iceberg/pull/16874), [\#16889](https://github.com/apache/iceberg/pull/16889), [\#16948](https://github.com/apache/iceberg/pull/16948)) + - Integrate ConvertEqualityDeletes with IcebergSink ([\#17142](https://github.com/apache/iceberg/pull/17142)) + - Fix deleted rows reappearing after a failed equality-delete conversion cycle ([\#17630](https://github.com/apache/iceberg/pull/17630)) + - Resolve unpartitioned equality deletes across all partitions ([\#17018](https://github.com/apache/iceberg/pull/17018)) + - Flink 2.1, 2.2, 2.3: Support variant for Avro readers and writers ([\#17737](https://github.com/apache/iceberg/pull/17737)) + - Flink 2.1, 2.2, 2.3: Support writing shredded variant ([\#15596](https://github.com/apache/iceberg/pull/15596)) + - Flink 2.1, 2.2, 2.3: Add SQL variant Avro dynamic record generator ([\#16450](https://github.com/apache/iceberg/pull/16450)) Review Comment: Makes sense. I didn't realize we didn't add those to Flink 1.20 and Flink 2.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
