Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-06 Thread via GitHub
yigal-rozenberg commented on code in PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#discussion_r1944993648 ## pyiceberg/table/__init__.py: ## @@ -1624,6 +1628,19 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arr

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-06 Thread via GitHub
yigal-rozenberg commented on code in PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#discussion_r1944925640 ## pyiceberg/table/__init__.py: ## @@ -1624,6 +1628,19 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arr

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
kevinjqliu commented on code in PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#discussion_r1944069870 ## pyiceberg/table/__init__.py: ## @@ -1624,6 +1628,19 @@ def to_ray(self) -> ray.data.dataset.Dataset: return ray.data.from_arrow(self.to_arrow())

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
yigal-rozenberg commented on PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#issuecomment-2638211161 Thanks for the comment! I can add this to the documentation as an alternative. The motivations was to extend existing PyIceberg table API to align with 'to_pandas' as

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
corleyma commented on PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#issuecomment-2638079043 At first glance, this approach seems fairly inferior to using the Polars [scan_iceberg ](https://docs.pola.rs/api/python/dev/reference/api/polars.scan_iceberg.html) functionality

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
yigal-rozenberg commented on code in PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#discussion_r1943690450 ## pyproject.toml: ## @@ -1183,6 +1184,766 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.override

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
yigal-rozenberg commented on code in PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#discussion_r1943690450 ## pyproject.toml: ## @@ -1183,6 +1184,766 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.override

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
yigal-rozenberg commented on PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#issuecomment-2637856975 I will provide with the relevant documentation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
Fokko commented on code in PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#discussion_r1943508147 ## pyproject.toml: ## @@ -1183,6 +1184,766 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.overrides]] Revie

Re: [PR] Added support for Polars DataFrame [iceberg-python]

2025-02-05 Thread via GitHub
amitgilad3 commented on PR #1614: URL: https://github.com/apache/iceberg-python/pull/1614#issuecomment-2637711734 Really nice - i really like working with polars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR