advancedxy commented on code in PR #8579:
URL: https://github.com/apache/iceberg/pull/8579#discussion_r1454350089
##########
format/spec.md:
##########
@@ -1145,9 +1191,14 @@ Sort orders are serialized as a list of JSON object,
each of which contains the
Each sort field in the fields list is stored as an object with the following
properties:
-|Field|JSON representation|Example|
-|--- |--- |--- |
-|**`Sort Field`**|`JSON object: {`<br /> `"transform": <transform
JSON>,`<br /> `"source-id": <source id int>,`<br
/> `"direction": <direction string>,`<br
/> `"null-order": <null-order string>`<br />`}`|`{`<br
/> ` "transform": "bucket[4]",`<br /> ` "source-id":
3,`<br /> ` "direction": "desc",`<br /> ` "null-order":
"nulls-last"`<br />`}`|
+| Field | JSON representation
|
Example
|
+|-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **`Sort Field`** [1] | `JSON object: {`<br
/> `"transform": <transform JSON>,`<br /> `"source-id":
<source id int>,`<br /> `"direction": <direction string>,`<br
/> `"null-order": <null-order string>`<br />`}`
| `{`<br /> ` "transform": "bucket[4]",`<br
/> ` "source-id": 3,`<br /> ` "direction": "desc",`<br
/> ` "null-order": "nulls-last"`<br />`}`
|
+| **`Sort Field with multi-arg transform`** [2] | `JSON object: {`<br
/> `"transform": <transform JSON>,`<br /> `"source-id":
-1,`<br /> `"source-ids": <list of ids>,`<br
/> `"direction": <direction string>,`<br
/> `"null-order": <null-order string>`<br />`}` | `{`<br
/> ` "transform": "bucketV2[4]",`<br /> ` "source-id":
-1,`<br /> ` "source-id": [1,2],`<br /> ` "direction":
"desc",`<br /> ` "null-order": "nulls-last"`<br />`}` |
+
+Notes:
+1. For sort fields with a transform with a single argument, the id of the
source field is set on `source-id`, and `source-ids` is omitted.
+2. For sort fields with a transform of multiple arguments, the ids of the
source fields are set on `source-ids`. To preserve backward compatibility,
`source-id` is set to -1.
Review Comment:
Unfortunately, I believe an explicit `source-id` is required to avoid
exception in old versions.
See
https://github.com/apache/iceberg/blob/a1f4642314ba36dd788c3c15838a0b32c9b91963/core/src/main/java/org/apache/iceberg/PartitionSpecParser.java#L135
and
https://github.com/apache/iceberg/blob/ab398a0d5ff195f763f8c7a4358ac98fa38a8de7/core/src/main/java/org/apache/iceberg/SortOrderParser.java#L154
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
