Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-20 Thread via GitHub
Fokko merged PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-20 Thread via GitHub
Fokko commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1722881230 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_fi

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-19 Thread via GitHub
sungwy commented on PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#issuecomment-2297755108 > LGTM! I added a few comments. @Fokko @HonahX do you mind taking a look when you get a chance? Yes, would love to get your blessings on this refactoring @Fokko and @HonahX -

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-15 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1719085905 ## pyiceberg/io/pyarrow.py: ## @@ -1398,6 +1588,11 @@ def project_table( return result +@deprecated( +deprecated_in="0.8.0", +removed_in="0.9

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-15 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1718336793 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717833002 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717825245 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
corleyma commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717763537 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
ndrluis commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717675207 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717696677 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717696470 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717682714 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717681434 ## pyiceberg/io/pyarrow.py: ## @@ -1304,6 +1305,195 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717678600 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
ndrluis commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717675207 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
ndrluis commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717670683 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717653154 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717484170 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717483870 ## pyiceberg/io/__init__.py: ## @@ -354,3 +356,14 @@ def load_file_io(properties: Properties = EMPTY_DICT, location: Optional[str] = raise ModuleNotFoun

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-14 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1717275634 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716215131 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,192 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716223848 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716165411 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716153306 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716153306 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716152079 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716137872 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716137872 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716137061 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716135524 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
sungwy commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716135206 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_per_f

Re: [PR] Refactor PyArrow DataFiles Projection functions [iceberg-python]

2024-08-13 Thread via GitHub
kevinjqliu commented on code in PR #1043: URL: https://github.com/apache/iceberg-python/pull/1043#discussion_r1716128868 ## pyiceberg/io/pyarrow.py: ## @@ -1308,6 +1309,138 @@ def _read_all_delete_files(fs: FileSystem, tasks: Iterable[FileScanTask]) -> Dic return deletes_p