rdblue commented on code in PR #14234: URL: https://github.com/apache/iceberg/pull/14234#discussion_r3197701376
########## format/spec.md: ########## @@ -707,6 +714,131 @@ For `geography` only, xmin (X value of `lower_bounds`) may be greater than xmax When calculating upper and lower bounds for `geometry` and `geography`, null or NaN values in a coordinate dimension are skipped; for example, POINT (1 NaN) contributes a value to X but no values to Y, Z, or M dimension bounds. If a dimension has only null or NaN values, that dimension is omitted from the bounding box. If either the X or Y dimension is missing then the bounding box itself is not produced. +##### Content Stats + +In Iceberg v4 stats have been redesigned and are represented by using nested structs (`struct<struct<...>>`). The statistics for fields are tracked inside a nested struct of value counts and bounds (described in the next section). Each field-level statistics struct is a field of the `content_stats` struct, which holds all statistics for table fields. + +###### ID assignment for stats fields + +ID assignment follows a deterministic mapping from the **table ID space** to the **stats ID space**, where a given field ID from the **table ID space** gets an ID assigned from the **stats ID space** for each field-level statistics struct. +Each field-level statistic listed in the [field stats types section](#field-stats-types) has a fixed offset. Its stats field ID is the enclosing stats struct's ID plus that offset. Review Comment: I think that this section should not focus on ID assignment, but rather how to construct the struct that holds a field's statistics. To me, that starts with a short description: the stats are assigned an ID range based on the table field ID. The fields of the struct are assigned by adding an offset to the beginning of that range. Then we should have a table with the offsets. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
