Re: [PR] Support partial deletes [iceberg-python]

2025-02-08 Thread via GitHub
kevinjqliu commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1947926049 ## pyiceberg/table/__init__.py: ## @@ -292,7 +303,13 @@ def _apply(self, updates: Tuple[TableUpdate, ...], requirements: Tuple[TableRequ requireme

Re: [PR] Support partial deletes [iceberg-python]

2025-02-07 Thread via GitHub
ZENOTME commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1946187013 ## pyiceberg/table/__init__.py: ## @@ -292,7 +303,13 @@ def _apply(self, updates: Tuple[TableUpdate, ...], requirements: Tuple[TableRequ requirement.

Re: [PR] Support partial deletes [iceberg-python]

2024-07-09 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1670138135 ## pyiceberg/table/__init__.py: ## @@ -539,6 +578,81 @@ def overwrite( for data_file in data_files: update_snapshot.append_data

Re: [PR] Support partial deletes [iceberg-python]

2024-07-09 Thread via GitHub
Fokko merged PR #569: URL: https://github.com/apache/iceberg-python/pull/569 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Support partial deletes [iceberg-python]

2024-07-08 Thread via GitHub
HonahX commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1669726783 ## pyiceberg/table/__init__.py: ## @@ -539,6 +578,81 @@ def overwrite( for data_file in data_files: update_snapshot.append_dat

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666905217 ## pyiceberg/table/__init__.py: ## @@ -498,7 +524,10 @@ def append(self, df: pa.Table, snapshot_properties: Dict[str, str] = EMPTY_DICT) update

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666901857 ## pyiceberg/table/__init__.py: ## @@ -538,6 +566,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_data

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666900390 ## pyiceberg/table/__init__.py: ## @@ -517,9 +546,6 @@ def overwrite( if not isinstance(df, pa.Table): raise ValueError(f"Expected PyArrow tabl

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666880488 ## pyiceberg/table/__init__.py: ## @@ -3882,7 +4161,7 @@ def _get_table_partitions( return table_partitions -def _determine_partitions(spec: PartitionSpec,

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666876690 ## tests/integration/test_deletes.py: ## @@ -0,0 +1,368 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666875275 ## pyiceberg/table/__init__.py: ## @@ -237,6 +245,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit" WRITE_PARTIT

Re: [PR] Support partial deletes [iceberg-python]

2024-07-05 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1666868594 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_data

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637353561 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingS

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637353561 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingS

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637347402 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingS

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_dat

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_dat

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_dat

Re: [PR] Support partial deletes [iceberg-python]

2024-06-12 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1637279841 ## pyiceberg/table/__init__.py: ## @@ -454,6 +482,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_dat

Re: [PR] Support partial deletes [iceberg-python]

2024-06-09 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1632276771 ## pyiceberg/table/__init__.py: ## @@ -434,6 +456,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615942072 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), ( AssertTable

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615924919 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingSn

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615924919 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingSn

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615912422 ## pyiceberg/table/__init__.py: ## @@ -443,6 +471,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_data

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1599183250 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), ( AssertTabl

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1599183250 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), ( AssertTabl

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1599183250 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), ( AssertTabl

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1598823358 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingS

Re: [PR] Support partial deletes [iceberg-python]

2024-05-13 Thread via GitHub
syun64 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1598626280 ## pyiceberg/table/__init__.py: ## @@ -443,6 +471,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_dat

Re: [PR] Support partial deletes [iceberg-python]

2024-05-10 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1597144084 ## pyiceberg/table/__init__.py: ## @@ -434,6 +460,8 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-10 Thread via GitHub
jqin61 commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1597063514 ## pyiceberg/table/__init__.py: ## @@ -434,6 +460,8 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594704486 ## pyiceberg/table/__init__.py: ## @@ -434,6 +458,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594703680 ## pyiceberg/table/__init__.py: ## @@ -434,6 +458,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594058803 ## tests/integration/test_deletes.py: ## @@ -0,0 +1,257 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594057885 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2959,161 @@ def _commit(self) -> UpdatesAndRequirements: ), ( AssertTable

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594055639 ## pyiceberg/table/__init__.py: ## @@ -443,6 +468,54 @@ def overwrite( for data_file in data_files: update_snapshot.append_data

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594053750 ## pyiceberg/table/__init__.py: ## @@ -443,6 +468,54 @@ def overwrite( for data_file in data_files: update_snapshot.append_data

Re: [PR] Support partial deletes [iceberg-python]

2024-05-08 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1594048917 ## pyiceberg/table/__init__.py: ## @@ -292,7 +303,13 @@ def _apply(self, updates: Tuple[TableUpdate, ...], requirements: Tuple[TableRequ requirement.va

Re: [PR] Support partial deletes [iceberg-python]

2024-05-04 Thread via GitHub
HonahX commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1584249768 ## pyiceberg/table/__init__.py: ## @@ -443,6 +468,54 @@ def overwrite( for data_file in data_files: update_snapshot.append_dat

Re: [PR] Support partial deletes [iceberg-python]

2024-05-02 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1588290938 ## pyiceberg/table/__init__.py: ## @@ -434,6 +456,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema) +

Re: [PR] Support partial deletes [iceberg-python]

2024-05-02 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1588286537 ## pyiceberg/table/__init__.py: ## @@ -292,7 +303,13 @@ def _apply(self, updates: Tuple[TableUpdate, ...], requirements: Tuple[TableRequ requirement.va

Re: [PR] Support partial deletes [iceberg-python]

2024-05-02 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1588266506 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit" WRITE_PARTIT

Re: [PR] Support partial deletes [iceberg-python]

2024-04-26 Thread via GitHub
kevinjqliu commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1581278513 ## pyiceberg/table/__init__.py: ## @@ -434,6 +456,9 @@ def overwrite( if table_arrow_schema != df.schema: df = df.cast(table_arrow_schema)

Re: [PR] Support partial deletes [iceberg-python]

2024-04-25 Thread via GitHub
MehulBatra commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1580084812 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit" WRITE_P

Re: [PR] Support partial deletes [iceberg-python]

2024-04-25 Thread via GitHub
MehulBatra commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1580084812 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit" WRITE_P

Re: [PR] Support partial deletes [iceberg-python]

2024-04-25 Thread via GitHub
MehulBatra commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1580084812 ## pyiceberg/table/__init__.py: ## @@ -235,6 +242,10 @@ class TableProperties: WRITE_PARTITION_SUMMARY_LIMIT = "write.summary.partition-limit" WRITE_P

Re: [PR] Support partial deletes [iceberg-python]

2024-04-16 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1566870388 ## pyiceberg/io/pyarrow.py: ## @@ -1912,3 +1920,55 @@ def _get_parquet_writer_kwargs(table_properties: Properties) -> Dict[str, Any]: default=TableProp