syahra712 opened a new pull request, #4118:
URL: https://github.com/apache/iceberg-python/pull/4118
# Rationale for this change
Closes #4114.
When `ArrowProjectionVisitor.map()` rebuilds a map whose values are structs,
it calls `pa.MapArray.from_arrays(offsets, keys, values)`. That drops the
validity bitmap of the original array, so null maps come back as empty maps
(`{}`) after projection. This is the map counterpart of the list issue handled
in #4065.
The fix keeps the validity and offsets buffers of the original map array and
only swaps in the projected child array, so nulls (and slicing offsets) are
preserved.
## Are these changes tested?
Yes. Added `test__to_requested_schema_null_map_of_structs` in
`tests/io/test_pyarrow.py`, covering a map with struct values, a map with
primitive values, and a sliced batch. The test fails on `main` (null rows
become `[]`) and passes with the fix.
## Are there any user-facing changes?
Null map columns with struct values are now read back as null instead of an
empty map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
