syun64 commented on code in PR #921:
URL: https://github.com/apache/iceberg-python/pull/921#discussion_r1676463963
##########
pyiceberg/io/pyarrow.py:
##########
@@ -1450,14 +1451,17 @@ def field_partner(self, partner_struct:
Optional[pa.Array], field_id: int, _: st
except ValueError:
return None
- if isinstance(partner_struct, pa.StructArray):
- return partner_struct.field(name)
- elif isinstance(partner_struct, pa.Table):
- return partner_struct.column(name).combine_chunks()
- elif isinstance(partner_struct, pa.RecordBatch):
- return partner_struct.column(name)
- else:
- raise ValueError(f"Cannot find {name} in expected
partner_struct type {type(partner_struct)}")
+ try:
+ if isinstance(partner_struct, pa.StructArray):
+ return partner_struct.field(name)
+ elif isinstance(partner_struct, pa.Table):
+ return partner_struct.column(name).combine_chunks()
+ elif isinstance(partner_struct, pa.RecordBatch):
+ return partner_struct.column(name)
+ else:
+ raise ValueError(f"Cannot find {name} in expected
partner_struct type {type(partner_struct)}")
+ except KeyError:
Review Comment:
This change is necessary to support writing dataframes / recordbatches with
a subset of the schema. Otherwise, the `ArrowAccessor` throws a `KeyError`.
This way, we return a `None` and the `ArrowProjectionVisitor` is responsible
for [checking if the field is
nullable](https://github.com/apache/iceberg-python/blob/b11cdb54b1a05cce0ade34af4ce81a94c34b2650/pyiceberg/io/pyarrow.py#L1394),
and can be filled in with a null array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
