aihuaxu commented on code in PR #12658: URL: https://github.com/apache/iceberg/pull/12658#discussion_r2021739556
########## format/spec.md: ########## @@ -1558,6 +1561,7 @@ The binary single-value serialization can be used to store the lower and upper b |------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **`geometry`** | A single point, encoded as a x:y:z:m concatenation of its 8-byte little-endian IEEE 754 coordinate values. x and y are mandatory. This becomes x:y if z and m are both unset, x:y:z if only m is unset, and x:y:NaN:m if only z is unset. | | **`geography`** | A single point, encoded as a x:y:z:m concatenation of its 8-byte little-endian IEEE 754 coordinate values. x and y are mandatory. This becomes x:y if z and m are both unset, x:y:z if only m is unset, and x:y:NaN:m if only z is unset. | +| **`variant`** | A `Variant` object, where each subcolumn path serves as a key and the corresponding bound value as the value. Subcolumn paths follow the JSON path format. | Review Comment: Make sense. Here we want to describe how exactly the data is laid out ########## format/spec.md: ########## @@ -648,6 +648,9 @@ Notes: 5. The `content_offset` and `content_size_in_bytes` fields are used to reference a specific blob for direct access to a deletion vector. For deletion vectors, these values are required and must exactly match the `offset` and `length` stored in the Puffin footer for the deletion vector blob. 6. The following field ids are reserved on `data_file`: 141. +For `variant` type, the `lower_bounds` and `upper_bounds` store the lower and upper bounds for all shredded subcolumns within a file. These bounds are represented as a Variant object, where each subcolumn path serves as a key and the corresponding bound value as the value. The object is then serialized into binary format (see [Variant encoding](https://github.com/apache/parquet-format/blob/master/VariantEncoding.md)). +Subcolumn paths follow the JSON path format to use normalized path, such as `$['location']['latitude']` or `$['user.name']`. If the shredded subcolumn is an array, represent it using the index 0 to indicate the array structure, such as `$['tags'][0]`. Review Comment: I will defer that for now and we can add if needed later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
