eitsupi opened a new issue, #743:
URL: https://github.com/apache/arrow-nanoarrow/issues/743
I don't know if this is a bug in polars or a problem with nanoarrow, but I
think it could be a bug in nanoarrow, since Arrow C++, the reference
implementation of Apache Arrow, seems to read it correctly (I'm using I noticed
this bug on R, so it is not a Python problem).
The binary representation of the Polars DataFrame should be an uncompressed
Arrow IPC stream, but nanoarrow does not seem to read this correctly.
```python
>>> import polars as pl
>>> import nanoarrow as na
>>> import pyarrow as pa
>>>
>>> df = pl.DataFrame({"foo": [1, 2, 3]})
>>> bytes = df.serialize()
>>>
>>> pa.ipc.open_stream(bytes).read_all() # Ok
pyarrow.Table
foo: int64
----
foo: [[1,2,3]]
>>> na.ArrayStream.from_readable(bytes).to_pylist() # Error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/home/username/.local/lib/python3.12/site-packages/nanoarrow/visitor.py", line
51, in to_pylist
return ToPyListConverter.visit(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/username/.local/lib/python3.12/site-packages/nanoarrow/visitor.py", line
233, in visit
for array in stream:
File "src/nanoarrow/_array_stream.pyx", line 249, in
nanoarrow._array_stream.CArrayStream.__next__
File "src/nanoarrow/_array_stream.pyx", line 238, in
nanoarrow._array_stream.CArrayStream.get_next
File "src/nanoarrow/_utils.pyx", line 155, in
nanoarrow._utils.Error.raise_message_not_ok
File "src/nanoarrow/_utils.pyx", line 149, in
nanoarrow._utils.Error.raise_message
nanoarrow._utils.NanoarrowException: ArrowArrayStream::get_next() failed
(22): Message flatbuffer verification failed (16) vector header out of range or
unaligned
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
