kylebarron opened a new issue, #43410: URL: https://github.com/apache/arrow/issues/43410
### Describe the enhancement requested Now that the PyCapsule Interface is starting to gain more traction (https://github.com/apache/arrow/issues/39195), I think it would be great if some of pyarrow's functional APIs accepted any PyCapsule Interface object, and not _just_ pyarrow objects. Do people have opinions on what functions should or should not check for these objects? I'd argue that file format writers should check for them, because it's only a couple lines of code, and the input stream will be fully iterated over regardless. E.g. looking at the Parquet writer: the high level API doesn't currently accept a `RecordBatchReader` either, so support for both can come at the same time. ```py from dataclasses import dataclass from typing import Any import pyarrow as pa import pyarrow.parquet as pq @dataclass class ArrowCStream: obj: Any def __arrow_c_stream__(self, requested_schema=None): return self.obj.__arrow_c_stream__(requested_schema=requested_schema) table = pa.table({"a": [1, 2, 3, 4]}) pq.write_table(table, "test.parquet") # works reader = pa.RecordBatchReader.from_stream(table) pq.write_table(reader, "test.parquet") # fails pq.write_table(ArrowCStream(table), "test.parquet") # fails ``` I'd argue that the writer should be generalized to accept any object with an `__arrow_c_stream__` dunder, and to ensure the stream is not materialized as a table. ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
