HonahX commented on code in PR #8267:
URL: https://github.com/apache/iceberg/pull/8267#discussion_r1288168026
##########
python/pyiceberg/io/pyarrow.py:
##########
@@ -451,7 +451,7 @@ def visit_binary(self, _: BinaryType) -> pa.DataType:
def _convert_scalar(value: Any, iceberg_type: IcebergType) -> pa.scalar:
if not isinstance(iceberg_type, PrimitiveType):
raise ValueError(f"Expected primitive type, got: {iceberg_type}")
- return pa.scalar(value).cast(schema_to_pyarrow(iceberg_type))
+ return pa.scalar(value=value, type=schema_to_pyarrow(iceberg_type))
Review Comment:
Changing this because I hit:
```
pyarrow.lib.ArrowNotImplementedError: casting scalars of type binary to type
fixed_size_binary[25]
```
when trying to apply a row filter on either UUIDType or FixedType:
```python
fixed_table = catalog.load_table("default.test_uuid_and_fixed_unpartitioned")
arrow_table = fixed_table.scan(row_filter=EqualTo("fixed_col",
b"1234567890123456789012345")).to_arrow()
```
Seems `pa.scalar` can specify type during creation and doing so can solve
this problem. However, I am not sure if there is any implication for this
change. Please let me know if there is any issue with this change. Thank you in
advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
