rdblue commented on code in PR #6775:
URL: https://github.com/apache/iceberg/pull/6775#discussion_r1111318769
##########
python/pyiceberg/io/pyarrow.py:
##########
@@ -515,6 +529,14 @@ def _file_to_table(
if pyarrow_filter is not None:
arrow_table = arrow_table.filter(pyarrow_filter)
+ if len(positional_deletes) > 0:
+ # When there are positional deletes, create a filter mask
+ mask = [True] * len(arrow_table)
+ for buffer in positional_deletes:
+ for pos in buffer:
+ mask[pos.as_py()] = False
+ arrow_table = arrow_table.filter(mask)
Review Comment:
It seems wasteful to create a Python list for this, but I'm having trouble
finding a better solution. It looks like we can create an array of nulls and
replace null with True, like this:
```python
null_arr = pa.nulls(10, type=pa.bool_())
true_arr = pa.fill_null(True)
```
Unfortunately, I don't see an easy way to set individual values to False in
the resulting array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
