Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1503456330 ## format/spec.md: ## @@ -1170,9 +1170,9 @@ Each sort field in the fields list is stored as an object with the following pro | required | required | required | **`d

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on PR #9661: URL: https://github.com/apache/iceberg/pull/9661#issuecomment-1965505699 Merging this. Thanks, @szehon-ho! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1503432867 ## format/spec.md: ## @@ -1170,9 +1170,9 @@ Each sort field in the fields list is stored as an object with the following pro | required | required | required | **`dire

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue merged PR #9661: URL: https://github.com/apache/iceberg/pull/9661 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1503431015 ## format/spec.md: ## @@ -1170,9 +1170,9 @@ Each sort field in the fields list is stored as an object with the following pro | required | required | required | **`dire

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1503400947 ## format/spec.md: ## @@ -1134,10 +1148,9 @@ In some cases partition specs are stored using only the field list instead of th The `field-id` property was added fo

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1501924946 ## format/spec.md: ## @@ -1314,6 +1331,24 @@ Default values are added to struct fields in v3. Types `timestamp_ns` and `timestamptz_ns` are added in v3. +Writing V3

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1501924360 ## format/spec.md: ## @@ -1314,6 +1330,24 @@ Default values are added to struct fields in v3. Types `timestamp_ns` and `timestamptz_ns` are added in v3. +Writing V3

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1501924078 ## format/spec.md: ## @@ -1134,10 +1148,9 @@ In some cases partition specs are stored using only the field list instead of th The `field-id` property was added for e

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1501923473 ## format/spec.md: ## @@ -1128,16 +1142,14 @@ Each partition field in the fields list is stored as an object. See the table fo |**`month`**|`JSON string: "month"`|`"mo

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1501923252 ## format/spec.md: ## @@ -1134,10 +1148,9 @@ In some cases partition specs are stored using only the field list instead of th The `field-id` property was added for e

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1501922331 ## format/spec.md: ## @@ -301,7 +303,7 @@ Tables are configured with a **partition spec** that defines how to produce a tu * A **transform** that is applied to the s

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

advancedxy commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1494418622 ## format/spec.md: ## @@ -1134,10 +1148,9 @@ In some cases partition specs are stored using only the field list instead of th The `field-id` property was added f

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1487243929 ## format/spec.md: ## @@ -1130,14 +1142,10 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"h

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1487243929 ## format/spec.md: ## @@ -1130,14 +1142,10 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"h

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1487244304 ## format/spec.md: ## @@ -1314,6 +1324,25 @@ Default values are added to struct fields in v3. Types `timestamp_ns` and `timestamptz_ns` are added in v3. +Sort or

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1487243929 ## format/spec.md: ## @@ -1130,14 +1142,10 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"h

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1487234117 ## format/spec.md: ## @@ -1117,7 +1119,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1487233897 ## format/spec.md: ## @@ -301,7 +301,7 @@ Tables are configured with a **partition spec** that defines how to produce a tu * A **transform** that is applied to th

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1486950821 ## format/spec.md: ## @@ -1314,6 +1324,25 @@ Default values are added to struct fields in v3. Types `timestamp_ns` and `timestamptz_ns` are added in v3. +Sort order

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1486949549 ## format/spec.md: ## @@ -1130,14 +1142,10 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"hour

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1486925845 ## format/spec.md: ## @@ -1117,7 +1119,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`**|`

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1486915191 ## format/spec.md: ## @@ -301,7 +301,7 @@ Tables are configured with a **partition spec** that defines how to produce a tu * A **transform** that is applied to the s

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

rdblue commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1486910753 ## format/spec.md: ## @@ -27,7 +27,7 @@ This is a specification for the Iceberg table format that is designed to manage ## Format Versioning -Versions 1 and 2 of t

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484905812 ## format/spec.md: ## @@ -1117,7 +1121,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484906498 ## format/spec.md: ## @@ -301,12 +301,14 @@ Tables are configured with a **partition spec** that defines how to produce a tu * A **transform** that is applied to

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484906929 ## format/spec.md: ## @@ -1130,14 +1140,11 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"h

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484906656 ## format/spec.md: ## @@ -1150,13 +1161,17 @@ Sort orders are serialized as a list of JSON object, each of which contains the Each sort field in the fields list i

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484906174 ## format/spec.md: ## @@ -1117,7 +1117,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484905812 ## format/spec.md: ## @@ -1117,7 +1121,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484905812 ## format/spec.md: ## @@ -1117,7 +1121,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

aokolnychyi commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1484814023 ## format/spec.md: ## @@ -301,12 +301,14 @@ Tables are configured with a **partition spec** that defines how to produce a tu * A **transform** that is applied t

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1483639368 ## format/spec.md: ## @@ -1117,7 +1117,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1483630271 ## format/spec.md: ## @@ -1130,14 +1140,11 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"h

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

advancedxy commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1482479258 ## format/spec.md: ## @@ -1130,14 +1140,11 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1481028685 ## format/spec.md: ## @@ -1130,14 +1140,11 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"h

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

advancedxy commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1480822510 ## format/spec.md: ## @@ -1130,14 +1140,11 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

advancedxy commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1480813301 ## format/spec.md: ## @@ -1117,7 +1117,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1480608962 ## format/spec.md: ## @@ -1117,7 +1117,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

manuzhang commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1479364416 ## format/spec.md: ## @@ -1117,7 +1117,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1479355794 ## format/spec.md: ## @@ -1117,7 +1117,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

manuzhang commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1479179241 ## format/spec.md: ## @@ -1117,7 +1117,17 @@ Partition specs are serialized as a JSON object with the following fields: |**`spec-id`**|`JSON int`|`0`| |**`fields`*

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

manuzhang commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1479178945 ## format/spec.md: ## @@ -1150,13 +1157,17 @@ Sort orders are serialized as a list of JSON object, each of which contains the Each sort field in the fields list i

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

manuzhang commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1479178945 ## format/spec.md: ## @@ -1150,13 +1157,17 @@ Sort orders are serialized as a list of JSON object, each of which contains the Each sort field in the fields list i

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

advancedxy commented on PR #9661: URL: https://github.com/apache/iceberg/pull/9661#issuecomment-1928683803 Thanks for taking this over @szehon-ho, I will review it in today or tomorrow. > hence wanted to get this change in before the 1.5 release I do agree that we should get t

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on PR #9661: URL: https://github.com/apache/iceberg/pull/9661#issuecomment-1928666370 @manuzhang i believe #8579 is not published yet, hence wanted to get this change in before the 1.5 release, if we want to add the clarification. -- This is an automated message from t

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

manuzhang commented on PR #9661: URL: https://github.com/apache/iceberg/pull/9661#issuecomment-1928640445 Do we also need to update https://github.com/apache/iceberg/blob/main/site/docs/spec.md? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on PR #9661: URL: https://github.com/apache/iceberg/pull/9661#issuecomment-1928524876 @rdblue @advancedxy @aokolnychyi @emkornfield , wanted to get the conversation started, on this proposal to clarify for V1-V3 behaviors for multi-arg transforms as discussed, let me kn

Re: [PR] Spec: Clarify multi-arg transform behavior for different versions [iceberg]

szehon-ho commented on code in PR #9661: URL: https://github.com/apache/iceberg/pull/9661#discussion_r1479063142 ## format/spec.md: ## @@ -1130,14 +1140,11 @@ Each partition field in the fields list is stored as an object. See the table fo |**`hour`**|`JSON string: "hour"`|`"h