syun64 commented on code in PR #921:
URL: https://github.com/apache/iceberg-python/pull/921#discussion_r1676463963
##########
pyiceberg/io/pyarrow.py:
##########
@@ -1450,14 +1451,17 @@ def field_partner(self, partner_struct:
Optional[pa.Array], field_id: int, _: st
except ValueError:
return None
- if isinstance(partner_struct, pa.StructArray):
- return partner_struct.field(name)
- elif isinstance(partner_struct, pa.Table):
- return partner_struct.column(name).combine_chunks()
- elif isinstance(partner_struct, pa.RecordBatch):
- return partner_struct.column(name)
- else:
- raise ValueError(f"Cannot find {name} in expected
partner_struct type {type(partner_struct)}")
+ try:
+ if isinstance(partner_struct, pa.StructArray):
+ return partner_struct.field(name)
+ elif isinstance(partner_struct, pa.Table):
+ return partner_struct.column(name).combine_chunks()
+ elif isinstance(partner_struct, pa.RecordBatch):
+ return partner_struct.column(name)
+ else:
+ raise ValueError(f"Cannot find {name} in expected
partner_struct type {type(partner_struct)}")
+ except KeyError:
Review Comment:
This change is necessary to support dataframes / recordbatches with a subset
of the schema. Otherwise, the ArrowAccessor throws a KeyError. This way, we
return a None and the ArrowProjectionVisitor is responsible for checking if the
field is nullable, and can be filled in with a null array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
