Re: [PR] prevent adding duplicate files [iceberg-python]

sungwy merged PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] prevent adding duplicate files [iceberg-python]

sungwy commented on PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#issuecomment-2306881850 > Hey @sungwy , fixed all the comments and tests pass is there anything else or can we merge? I will take another pass at it today. Thank you @amitgilad3 ! -- This is an a

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#issuecomment-2306566373 Hey @sungwy , fixed all the comments and tests pass is there anything else or can we merge? -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#issuecomment-2294096857 Hey @sungwy + @kevinjqliu , again thanks for all the help and guidance , i went over all the comments and fixed them -- This is an automated message from the Apache Git Serv

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1720268289 ## tests/integration/test_add_files.py: ## @@ -732,3 +732,76 @@ def test_add_files_subset_of_schema(spark: SparkSession, session_catalog: Catalo for col

Re: [PR] prevent adding duplicate files [iceberg-python]

sungwy commented on PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#issuecomment-2292280939 @amitgilad3 - this looks almost ready to merge. @kevinjqliu 's made some great suggestions here, so I'm thinking that we can take another round of reviews once we take a pass throug

Re: [PR] prevent adding duplicate files [iceberg-python]

sungwy commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1715220578 ## tests/integration/test_add_files.py: ## @@ -732,3 +732,76 @@ def test_add_files_subset_of_schema(spark: SparkSession, session_catalog: Catalo for column

Re: [PR] prevent adding duplicate files [iceberg-python]

kevinjqliu commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1714447553 ## tests/integration/test_add_files.py: ## @@ -732,3 +732,76 @@ def test_add_files_subset_of_schema(spark: SparkSession, session_catalog: Catalo for col

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1714418419 ## pyiceberg/table/__init__.py: ## @@ -630,7 +630,20 @@ def add_files(self, file_paths: List[str], snapshot_properties: Dict[str, str] = Raises:

Re: [PR] prevent adding duplicate files [iceberg-python]

sungwy commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1714392026 ## pyiceberg/table/__init__.py: ## @@ -630,7 +630,20 @@ def add_files(self, file_paths: List[str], snapshot_properties: Dict[str, str] = Raises:

Re: [PR] prevent adding duplicate files [iceberg-python]

sungwy commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1714391548 ## pyiceberg/table/__init__.py: ## @@ -630,7 +630,20 @@ def add_files(self, file_paths: List[str], snapshot_properties: Dict[str, str] = Raises:

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1714382905 ## pyiceberg/table/__init__.py: ## @@ -621,6 +621,13 @@ def delete(self, delete_filter: Union[str, BooleanExpression], snapshot_properti if not dele

Re: [PR] prevent adding duplicate files [iceberg-python]

amitgilad3 commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1714382905 ## pyiceberg/table/__init__.py: ## @@ -621,6 +621,13 @@ def delete(self, delete_filter: Union[str, BooleanExpression], snapshot_properti if not dele

Re: [PR] prevent adding duplicate files [iceberg-python]

sungwy commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1714372318 ## pyiceberg/table/__init__.py: ## @@ -621,6 +621,13 @@ def delete(self, delete_filter: Union[str, BooleanExpression], snapshot_properti if not delete_s

Re: [PR] prevent adding duplicate files [iceberg-python]

kevinjqliu commented on code in PR #1036: URL: https://github.com/apache/iceberg-python/pull/1036#discussion_r1713025181 ## pyiceberg/table/__init__.py: ## @@ -630,7 +648,15 @@ def add_files(self, file_paths: List[str], snapshot_properties: Dict[str, str] = Raises:

[PR] prevent adding duplicate files [iceberg-python]

amitgilad3 opened a new pull request, #1036: URL: https://github.com/apache/iceberg-python/pull/1036 This resolves #998 , where duplicate files are added with add_files method, handles 2 cases: 1. Files list is not unique 2. One of the files added is already referenced by current