chitralverma commented on issue #7067: URL: https://github.com/apache/iceberg/issues/7067#issuecomment-1496983955
Hi all, Polars contributor here. I did the integration for DeltaIO recently :) I was looking to do this integration over the weekend and it will be a quick addition because py-iceberg already allows a table to be converted to a pyarrow table which can be fed to Polars' eager `read` API. No need to rely on `to_pandas` which will incur additional overhead. However, it would be great to support the lazy `scan` API as well, because most internal optimisation takes over there. This will require an iceberg table to be converted lazily into a pyarrow dataset with a `.to_arrow_dataset()` addition. I can open a PR for this if it makes sense to the contributors/ members of Iceberg. Once this addition is in place, I can open a PR to support iceberg on Polars side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
