ryanworl opened a new pull request, #1830: URL: https://github.com/apache/iceberg-go/pull/1830
## What `DataFileStatsFromMeta` panics on Parquet files that materialize the reserved v3 row-lineage columns `_row_id` (field-id 2147483540) and `_last_updated_sequence_number` (2147483539): writers emit those columns without adding them to the table schema, so they have no entry in the metrics plan, and the zero-value `StatisticsCollector` panics inside the aggregator on its nil Iceberg type. After resolving a column's field id, the collector lookup now checks `ok`. Missing and `iceberg.IsMetadataColumn(fieldID)`: skip the column (no stats). Missing and a real schema column: panic with the field id and column path — a plan/file mismatch is a bug, not a condition to paper over. `MetricModeNone` still skips as before. ## Why Per the spec ([Reserved Field IDs](https://iceberg.apache.org/spec/#reserved-field-ids), [Row Lineage](https://iceberg.apache.org/spec/#row-lineage)) these are reserved metadata columns; a writer that materializes them is not required to put them on the table schema. This repo's own `position_delta_writer.go` does exactly that — it writes `_row_id` into data files without touching the table schema — so stats collection over such a file panics today. One disclosed tightening: a nil or partial `StatsCols` plan now panics deterministically for any real schema column holding values (previously it could pass silently when the column happened to contain only nulls). `TestWriteDataFileErrOnClose` was updated to supply a plan covering the list-element leaf so it reaches the close error it actually tests. ## Tests - Reserved `_row_id` outside the plan is skipped and produces no stats; a non-reserved schema column outside the plan panics with the field id and path. - `go test ./table/...` and `golangci-lint run` are clean. Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
