Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-03-02 Thread via GitHub
Fokko commented on issue #1679: URL: https://github.com/apache/iceberg-python/issues/1679#issuecomment-2692857608 @kevinjqliu Yes, let's close this 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-03-02 Thread via GitHub
Fokko closed issue #1679: Push upsert changes detection to Arrow URL: https://github.com/apache/iceberg-python/issues/1679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-02-24 Thread via GitHub
kevinjqliu commented on issue #1679: URL: https://github.com/apache/iceberg-python/issues/1679#issuecomment-2679985290 I think this is done in #1685, right? @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-02-24 Thread via GitHub
mattmartin14 commented on issue #1679: URL: https://github.com/apache/iceberg-python/issues/1679#issuecomment-2679973969 @Fokko - good stuff here :). I see you all are starting to ramp up the efforts with working on the upsert stuff. -- This is an automated message from the Apache Git

Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-02-19 Thread via GitHub
Fokko commented on issue #1679: URL: https://github.com/apache/iceberg-python/issues/1679#issuecomment-2669693184 @corleyma I was incorporating your code, but then I realized that it is even simpler: https://github.com/apache/iceberg-python/pull/1685/files#diff-cdc27639285a3481a96d65f8844d

Re: [I] Push upsert changes detection to Arrow [iceberg-python]

2025-02-18 Thread via GitHub
corleyma commented on issue #1679: URL: https://github.com/apache/iceberg-python/issues/1679#issuecomment-2666511285 As a temporary workaround, would something like this suffice? ```python def force_non_null_fields(table: pa.Table, fields_to_force: list[str]) -> pa.Table: n

[I] Push upsert changes detection to Arrow [iceberg-python]

2025-02-18 Thread via GitHub
Fokko opened a new issue, #1679: URL: https://github.com/apache/iceberg-python/issues/1679 ### Feature Request / Improvement Right now we iterate [over all the rows](https://github.com/apache/iceberg-python/blob/main/pyiceberg/table/upsert_util.py) to find the rows that are different