moomindani commented on code in PR #3630:
URL: https://github.com/apache/iceberg-python/pull/3630#discussion_r3562618973
##########
pyiceberg/table/sorting.py:
##########
@@ -107,11 +108,31 @@ def map_source_ids_onto_source_id(cls, data: Any) -> Any:
if len(source_ids) == 0:
raise ValueError("Empty source-ids is not allowed")
if len(source_ids) > 1:
- raise ValueError("Multi argument transforms are not
yet supported")
+ if data.get("transform") is None:
+ raise ValueError("Transform is required for a
multi-argument field")
+ # Multi-argument transforms cannot be evaluated; per
the spec, v3 readers
+ # must read tables with such transforms, ignoring them
+ data["transform"] =
UnknownTransform(transform=str(data["transform"]))
+ else:
+ data.pop("source-ids", None)
data["source-id"] = source_ids[0]
return data
+ @model_serializer(mode="wrap")
+ def _serialize_source_ids(self, handler: Any) -> Any:
+ serialized = handler(self)
+ # Per the spec, single-argument transforms write only source-id and
+ # multi-argument transforms write only source-ids
+ if self.source_ids is not None and len(self.source_ids) > 1:
+ serialized.pop("source-id", None)
+ serialized.pop("source_id", None)
+ else:
+ serialized.pop("source-ids", None)
+ serialized.pop("source_ids", None)
Review Comment:
Agreed, removed them in 818e6f99. For context, they were a defensive measure
for `model_dump(by_alias=False)`, where pydantic emits field names
(`source_id`) instead of aliases (`source-id`) — but since `IcebergBaseModel`
always forces `by_alias=True` for metadata serialization, that case never
produces spec JSON and the extra pops were unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
