Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-15 Thread via GitHub
Fokko commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2661107134 @soumilshah1995 I've raised a PR here https://github.com/apache/iceberg-python/pull/1665 -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-15 Thread via GitHub
soumilshah1995 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2661067345 Hi can you please help us with some example code ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-13 Thread via GitHub
mattmartin14 closed pull request #1534: Feature: MERGE/Upsert Support URL: https://github.com/apache/iceberg-python/pull/1534 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-13 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2657686000 Final stage of this PR was moved to new [PR #1660](https://github.com/apache/iceberg-python/pull/1660) due to some infrastructure challenges on my end. Closing this one now.

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2655333694 > looks like there are some changes that we previous removed that are now back :/ > > like changes in `pyproject.toml` > > > > @mattmartin14 do you mind

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2655235254 looks like there are some changes that we previous removed that are now back :/ like changes in `pyproject.toml` @mattmartin14 do you mind if i push these commits to a

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654988421 i've tried something a little different to resolve the lock file conflict; i'll see if it works; added my company's repo with the option of priority==explicit; it looks like i

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654939057 @Fokko @kevinjqliu - for context on this poetry lock file issue that keeps coming up: If i run poetry.lock locally, it wil embed my company's private repo throughout t

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654870836 @Fokko , @kevinjqliu - i think we are good on teh vendor files; i just need your help one more time computing the poetry.lock file, since if i do it on my end, it will embed

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1953415874 ## vendor/hive_metastore/ThriftHiveMetastore.py: ## @@ -72957,4 +72957,4 @@ def __ne__(self, other): ), # 1 ) fix_spec(all_structs) -del all_structs

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1953389250 ## vendor/fb303/FacebookService.py: ## @@ -2417,4 +2417,4 @@ def __ne__(self, other): all_structs.append(shutdown_args) shutdown_args.thrift_spec = () fix_spec(

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654758986 > ah thats odd. lets do this the manual way > > for these files > > * vendor/fb303/FacebookService.py > * vendor/hive_metastore/ThriftHiveMetastore.py > * ve

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654749976 ah thats odd. lets do this the manual way for these files * vendor/fb303/FacebookService.py * vendor/hive_metastore/ThriftHiveMetastore.py * vendor/hive_metastore/

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654708653 > in this case, the right alias is `github` since > > ``` > github g...@github.com:apache-iceberg-python.git (fetch) > ``` > > so > > ``` > git

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654645955 > the changes to `vendor/` is still there. > > could you try something like > > ``` > git fetch origin main > git checkout origin/main vendor > ``` >

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654628155 the changes to `vendor/` is still there. could you try something like ``` git fetch origin main git checkout origin/main vendor ``` assuming that `orig

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654502142 @kevinjqliu - I saw i pushed the hive metastore and ttypes, i didnt push your changes on fb303; LMK if you still need me to run a checkout -- This is an automated message f

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654489434 > woot! Thanks! can you run `git checkout main vendor`? This will revert the unrelated changes from the `vendor/` dir I thought I pushed the fixes on the vendor facebook

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654487627 woot! Thanks! can you run `git checkout main vendor`? This will revert the unrelated changes from the `vendor/` dir -- This is an automated message from the Apache Git Servic

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654483094 FINALLY! ![victory](https://github.com/user-attachments/assets/f0a7705b-e8d2-4a27-b838-a5c830dbaad3) -- This is an automated message from the Apache Git Service.

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654297109 trying another push on the table init -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654287154 @kevinjqliu - i'm at a loss. the "make lint" command ran with zero issues on my end before i ran the latest push. Do you knw what's going on? -- This is an automated messag

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654266558 thank you @kevinjqliu - pushed latest fixes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654210328 Heres the branch with the relevant changes https://github.com/apache/iceberg-python/compare/main...kevinjqliu:iceberg-python:StateFarmIns/main?expand=1 I had to * r

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654023990 did a rebase on the poetry stuff; hopefull this is it @kevinjqliu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654054288 @kevinjqliu - can you add datafusion as a dev dependency and rerun the poetry lock file hash with it included? This is what we included before in poetry for datafusion:

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654031811 my hunch @kevinjqliu is the content hash will fail on the lock file since i manually added the data fusion stuff back in. -- This is an automated message from the Apache Gi

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2654028531 @kevinjqliu - if this latest update fails, i might need your help again to run a poetry lock that includes datafusion. Every time i add in our State Farm private repo, it tur

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2653974783 > ![Screenshot 2025-02-12 at 6 54 23  AM](https://private-user-images.githubusercontent.com/9057843/412476375-9ec54428-975a-4fd2-a528-79359d807198.jpg?jwt=eyJhbGciOiJIUzI1NiIsIn

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2653953775 ![Screenshot 2025-02-12 at 6 54 23  AM](https://github.com/user-attachments/assets/9ec54428-975a-4fd2-a528-79359d807198) @mattmartin14 im afraid we need to rebase off `main` ag

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2653856593 @Fokko @kevinjqliu @marcoaanogueira - i ran the "make docs-build" commmand locally after applying the recommended changes and we should be all grean lights at this point. Hopi

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-12 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1952434838 ## pyiceberg/table/__init__.py: ## @@ -1086,6 +1094,78 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMappi

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
kevinjqliu commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951735621 ## pyiceberg/table/__init__.py: ## @@ -1086,6 +1094,78 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMapping

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
marcoaanogueira commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951669632 ## pyiceberg/table/__init__.py: ## @@ -1086,6 +1094,78 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMa

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2652066676 @Fokko - I saw the only thing that failed in the CI was the docs build. Any idea why and if it's something I can fix? -- This is an automated message from the Apache Git Ser

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
marcoaanogueira commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951669632 ## pyiceberg/table/__init__.py: ## @@ -1086,6 +1094,78 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMa

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951530296 ## package-lock.json: ## @@ -0,0 +1,1420 @@ +{ Review Comment: removed that file; it was created by npm when i ran "make lint" -- This is an automat

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2651967387 @Fokko - thank you for all the help on this. I'm hoping this is finally it. Make lint ran locally for me with no errors -- This is an automated message from the Apache Git

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951530493 ## pyproject.toml: ## @@ -1183,6 +1184,766 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.overrides]]

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951530027 ## package.json: ## @@ -0,0 +1,5 @@ +{ Review Comment: removed that file -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951523089 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951521796 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951518221 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951480372 ## package.json: ## @@ -0,0 +1,5 @@ +{ Review Comment: We should leave out this one as well, then we can also revert the changes in the `.gitignore` -- Th

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951479991 ## package-lock.json: ## @@ -0,0 +1,1420 @@ +{ Review Comment: I don't think we should commit this one -- This is an automated message from the Apache Git

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951478851 ## pyproject.toml: ## @@ -1183,6 +1184,766 @@ ignore_missing_imports = true module = "tenacity.*" ignore_missing_imports = true +[[tool.mypy.overrides]] +modul

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951477462 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951471685 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1951471241 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2651806164 @mattmartin14 Thanks, let me dig into the remaining issues and see what's going on 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2651624948 @kevinjqliu - I also updated the original description of this PR. Please let me know if you have any more recommendations/tweaks you would like to do on it. Thanks,

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2651601160 @Fokko @kevinjqliu - i was able to run the "make lint" command and I've resolved i think 99% of it; the only thing that make lint is flagging is: ```bash pyiceberg/t

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2651174761 > > otherwise, given my company's firewall rules, i won't ever be able to reach them. > > That's strict! > > I'm also happy to create a PR to the `StateFarmIns:ma

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
Fokko commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2651078297 > otherwise, given my company's firewall rules, i won't ever be able to reach them. That's strict! I'm also happy to create a PR to the `StateFarmIns:main` branch to fix

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2650876043 @kevinjqliu - I manually added all the lint fixes you called out as well as updated my the poetry toml and lock files with the new ones you provided. thank you so much and i'm

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-11 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2650818707 > @mattmartin14 i tried to fix as many linter issue as possible, the changes are in https://github.com/apache/iceberg-python/compare/main...kevinjqliu:iceberg-python:kevinjqli

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649504621 @mattmartin14 i tried to fix as many linter issue as possible, the changes are in https://github.com/apache/iceberg-python/compare/main...kevinjqliu:iceberg-python:kevinjqliu/15

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649313405 i was able to run: ```bash ruff check --fix ``` hope that helped -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649305878 @kevinjqliu - also, i'm seeing that the CI is failing to pull datafusion; how do i fix that, since i"m using datafusion in my test files? I did run the command: ```bas

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649303687 > Looks like CI failed due to the linter, can you run `make lint` locally? > > > i updated my original description to have a link to PR 402; > > It would be great

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649278852 Looks like CI failed due to the linter, can you run `make lint` locally? > i updated my original description to have a link to PR 402; It would be great to rewrite

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649271636 i saw there was an error on the ci on trying to pull in datafusion; i'm using datafusion for my test cases, which i was told is fine. It is used to build the test datasets. Is

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649252673 > Thanks! a couple conflicts from rebasing. would be good to address those before running CI. > > Could you also edit the PR description? And include references to #402

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649246609 Thanks! a couple conflicts from rebasing. would be good to address those before running CI. Could you also edit the PR description? And include references to #402 -- T

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949895088 ## pyiceberg/expressions/literals.py: ## @@ -29,6 +29,7 @@ from math import isnan from typing import Any, Generic, Type from uuid import UUID +from datet

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649245843 please let me know if you all have any additional issues on the workflows. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649242299 @Fokko , @bitsondatadev and i resolved the conflicts (i think 😀). You should be good to go on kicking off the workflows; we are rebased to the current main branch -- This i

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
kevinjqliu commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949892377 ## pyiceberg/expressions/literals.py: ## @@ -29,6 +29,7 @@ from math import isnan from typing import Any, Generic, Type from uuid import UUID +from datetim

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
kevinjqliu commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949893406 ## pyproject.toml: ## @@ -1409,4 +1409,4 @@ module = "tenacity.*" ignore_missing_imports = true [tool.coverage.run] -source = ['pyiceberg/'] +source = ['p

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
Fokko commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649172716 @mattmartin14 There are conflicts with the main branch, which prohibits me from merging this: https://github.com/user-attachments/assets/b842a6f4-807b-429e-be56-17346d5ac53b";

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649131822 > @mattmartin14 We need to get the tests green before we can merge this into main @Fokko - my tests are all green; is there something else i need to do at the moment? o

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
Fokko commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649130857 Let me add my preferred way of doing this: ``` git remote add github g...@github.com:apache/iceberg-python.git git fetch --all git pull github main git push origin m

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
Fokko commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649129009 @mattmartin14 We need to get the tests green before we can merge this into main -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649124452 > @mattmartin14 you need to rebase off of the `upstream` repo and push to `origin`. Assuming that `upstream` is the `apache/iceberg` repository and `origin` is `mattmartin14/i

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
bitsondatadev commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2649081273 @mattmartin14 you need to rebase off of the `upstream` repo and push to `origin`. Assuming that `upstream` is the `apache/iceberg` repository and `origin` is `mattmartin14/ic

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2648953956 > Could you rebase off the main to pick up the latest set of changes? > > Alternatively, I'm happy to help push out the fix for poetry lock and pyproject if needed

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
kevinjqliu commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2648912093 Could you rebase off the main to pick up the latest set of changes? Alternatively, I'm happy to help push out the fix for poetry lock and pyproject if needed -- This i

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
tscottcoombes1 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2648769812 Not in my power to approve, but LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949594241 ## tests/table/test_upsert.py: ## @@ -0,0 +1,281 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreement

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949178473 ## tests/table/test_upsert.py: ## @@ -0,0 +1,281 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949180237 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,153 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949178890 ## pyproject.toml: ## @@ -95,21 +96,29 @@ pyspark = "3.5.3" cython = "3.0.11" deptry = ">=0.14,<0.23" docutils = "!=0.21.post1" # https://github.com/p

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1949176655 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,146 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2648176899 Alright @Fokko @tscottcoombes1 , I've rebased the pyproject.toml file and poetry.lock file to match what you all have as of today on this repo, re-ran my tests, and all passed

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-10 Thread via GitHub
Fokko commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2647949821 I agree with @tscottcoombes1 that we're very close on this one. Could you rebase and make sure that the tests pass? -- This is an automated message from the Apache Git Service. To

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
tscottcoombes1 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2644227791 Thanks @mattmartin14 appreciate you keep plugging away. I think it looks really good. I've left a couple more comments (sorry). IMO only the pyproject.toml is a blocker, eve

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1947262347 ## tests/table/test_upsert.py: ## @@ -0,0 +1,327 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreement

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1947257232 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,153 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1947249998 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,146 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agree

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
tscottcoombes1 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1947207894 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,158 @@ + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agr

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
kevinjqliu commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1946708505 ## pyiceberg/expressions/literals.py: ## @@ -145,6 +148,8 @@ def literal(value: L) -> Literal[L]: return BinaryLiteral(value) elif isinstance(va

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2643139582 Hi @Fokko @tscottcoombes1 @kevinjqliu , Pushed one more (and I hope 😂) final round of updates; i fixed some formatting, replaced the SqlCatalog with the InMemory catalog

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1946631889 ## pyproject.toml: ## @@ -99,17 +101,24 @@ docutils = "!=0.21.post1" # https://github.com/python-poetry/poetry/issues/924 [tool.poetry.group.docs.depend

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1946630356 ## tests/table/test_upsert.py: ## @@ -0,0 +1,327 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-07 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1946392951 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,153 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-06 Thread via GitHub
corleyma commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1946056996 ## pyiceberg/table/upsert_util.py: ## @@ -0,0 +1,153 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-06 Thread via GitHub
abrunner94 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2641591748 Looking forward to this in the upcoming release! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-06 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1945302934 ## tests/table/test_upsert.py: ## @@ -0,0 +1,327 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-06 Thread via GitHub
mattmartin14 commented on PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#issuecomment-2640746041 @Fokko @tscottcoombes1 @kevinjqliu - another round of updates completed. As of now, i think these are the only ouststanding items: - the TOML lock file for poetry might

Re: [PR] Feature: MERGE/Upsert Support [iceberg-python]

2025-02-06 Thread via GitHub
mattmartin14 commented on code in PR #1534: URL: https://github.com/apache/iceberg-python/pull/1534#discussion_r1945259484 ## pyiceberg/table/__init__.py: ## @@ -1064,6 +1067,78 @@ def name_mapping(self) -> Optional[NameMapping]: """Return the table's field-id NameMappi

  1   2   3   >