Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
kevinjqliu merged PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
kevinjqliu commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661577715 LGTM! Thanks @Fokko and thanks @mattmartin14 for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
Fokko commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661563124 > I honestly didn't even know about the iceberg specific identifier fields until you had recently mentioned it. I can't imagine many have. I see situations where teams have already b

Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
kevinjqliu commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661524962 > Not to sound blunt but the example above seems odd TBH @mattmartin14 it is an odd example! I had a feeling this can break the uniqueness constraint so I crafted an examp

Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
kevinjqliu commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661523189 > Yes, that is an issue, but we don't respect this for any of the operations (append, etc). Doing this would make the operations expensive so we could leave this up to the user.

Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
mattmartin14 commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661420332 > actually this doesnt respect [`identifier_field_ids`](https://iceberg.apache.org/spec/#identifier-field-ids) columns' uniqueness > > > > For example, >

Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
mattmartin14 commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661415391 > @kevinjqliu Yes, that is an issue, but we don't respect this for any of the operations (`append`, etc). Doing this would make the operations expensive so we could leave this

Re: [PR] Add upsert docs [iceberg-python]

2025-02-16 Thread via GitHub
Fokko commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661321735 @kevinjqliu Yes, that is an issue, but we don't respect this for any of the operations (`append`, etc). Doing this would make the operations expensive so we could leave this up to th

Re: [PR] Add upsert docs [iceberg-python]

2025-02-15 Thread via GitHub
kevinjqliu commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661288902 > jon_cols seems focused on the primary key. How do we specify the partition column to enable partition pruning @ananthdurai the partition columns are part of the Iceberg

Re: [PR] Add upsert docs [iceberg-python]

2025-02-15 Thread via GitHub
ananthdurai commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661196077 jon_cols seems focused on the primary key. How do we specify the partition column to enable partition pruning? -- This is an automated message from the Apache Git Service. T

Re: [PR] Add upsert docs [iceberg-python]

2025-02-15 Thread via GitHub
kevinjqliu commented on code in PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#discussion_r1957222927 ## pyiceberg/table/__init__.py: ## @@ -1148,6 +1148,15 @@ def upsert( """ from pyiceberg.table import upsert_util +if join_cols is

Re: [PR] Add upsert docs [iceberg-python]

2025-02-15 Thread via GitHub
soumilshah1995 commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661156798 lovely -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Add upsert docs [iceberg-python]

2025-02-15 Thread via GitHub
soumilshah1995 commented on PR #1665: URL: https://github.com/apache/iceberg-python/pull/1665#issuecomment-2661154101 Hi im trying an example I am getting no method upsert I using 0.9.0 version I read the docs attached am I missing something ``` import os import pyarrow as pa

[PR] Add upsert docs [iceberg-python]

2025-02-15 Thread via GitHub
Fokko opened a new pull request, #1665: URL: https://github.com/apache/iceberg-python/pull/1665 And make the join-cols optional using the identifier fields. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above