Re: [PR] Flink: Maintenance - MonitorSource [iceberg]

2024-05-18 Thread via GitHub
pvary commented on code in PR #10308: URL: https://github.com/apache/iceberg/pull/10308#discussion_r1605953735 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/MonitorSource.java: ## @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software Founda

[PR] Cherrypick and Publish Changes Snapshot Management Operations [iceberg-python]

2024-05-18 Thread via GitHub
chinmay-bhat opened a new pull request, #750: URL: https://github.com/apache/iceberg-python/pull/750 PR in Progress, **not ready for review** Create Cherry pick and Publish changes APIs and classes. PR depends on unmerged changes in other PRs - _deleted_data_files for _Merg

[PR] Build: Bump software.amazon.awssdk:bom from 2.25.50 to 2.25.55 [iceberg]

2024-05-18 Thread via GitHub
dependabot[bot] opened a new pull request, #10355: URL: https://github.com/apache/iceberg/pull/10355 Bumps software.amazon.awssdk:bom from 2.25.50 to 2.25.55. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=soft

[PR] Build: Bump org.springframework:spring-web from 5.3.34 to 5.3.35 [iceberg]

2024-05-18 Thread via GitHub
dependabot[bot] opened a new pull request, #10354: URL: https://github.com/apache/iceberg/pull/10354 Bumps [org.springframework:spring-web](https://github.com/spring-projects/spring-framework) from 5.3.34 to 5.3.35. Release notes Sourced from https://github.com/spring-projects/spr

[PR] Build: Bump mkdocs-material from 9.5.21 to 9.5.23 [iceberg]

2024-05-18 Thread via GitHub
dependabot[bot] opened a new pull request, #10353: URL: https://github.com/apache/iceberg/pull/10353 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.21 to 9.5.23. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mkdoc

Re: [PR] Core: Prevent dropping column which is referenced by active partition… [iceberg]

2024-05-18 Thread via GitHub
amogh-jahagirdar commented on code in PR #10352: URL: https://github.com/apache/iceberg/pull/10352#discussion_r1605884398 ## core/src/main/java/org/apache/iceberg/SchemaUpdate.java: ## @@ -533,6 +537,34 @@ private static Schema applyChanges( } } +Map> specToDel

Re: [PR] Core: Prevent dropping column which is referenced by active partition… [iceberg]

2024-05-18 Thread via GitHub
amogh-jahagirdar commented on code in PR #10352: URL: https://github.com/apache/iceberg/pull/10352#discussion_r1605826546 ## core/src/main/java/org/apache/iceberg/SchemaUpdate.java: ## @@ -533,6 +537,34 @@ private static Schema applyChanges( } } +Map> specToDel

Re: [PR] support python 3.12 [iceberg-python]

2024-05-18 Thread via GitHub
MehulBatra commented on PR #254: URL: https://github.com/apache/iceberg-python/pull/254#issuecomment-2118894495 Tentative ray support for 3.12 -> https://github.com/ray-project/ray/issues/40211#issuecomment-2102962786 -- This is an automated message from the Apache Git Service. To respond

Re: [PR] Core: Prevent dropping column which is referenced by active partition… [iceberg]

2024-05-18 Thread via GitHub
amogh-jahagirdar commented on code in PR #10352: URL: https://github.com/apache/iceberg/pull/10352#discussion_r1605820719 ## core/src/main/java/org/apache/iceberg/SchemaUpdate.java: ## @@ -533,6 +537,34 @@ private static Schema applyChanges( } } +Map> specToDel

Re: [PR] Core: Prevent dropping column which is referenced by active partition… [iceberg]

2024-05-18 Thread via GitHub
amogh-jahagirdar commented on PR #10352: URL: https://github.com/apache/iceberg/pull/10352#issuecomment-2118884126 ![Screenshot from 2024-05-18 11-11-56](https://github.com/apache/iceberg/assets/87500546/d3f594fa-37b1-457c-abe4-3ed16b327005) Needs tests but did some local testing on spark

Re: [PR] Core: Prevent dropping column which is referenced by active partition… [iceberg]

2024-05-18 Thread via GitHub
amogh-jahagirdar commented on code in PR #10352: URL: https://github.com/apache/iceberg/pull/10352#discussion_r1605819844 ## core/src/main/java/org/apache/iceberg/SchemaUpdate.java: ## @@ -533,6 +537,34 @@ private static Schema applyChanges( } } +Map> specToDel

Re: [PR] Core: Replace deprecated Roaring64Bitmap#add call with addRange [iceberg]

2024-05-18 Thread via GitHub
amogh-jahagirdar merged PR #10350: URL: https://github.com/apache/iceberg/pull/10350 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] Core: Replace deprecated Roaring64Bitmap#add call with addRange [iceberg]

2024-05-18 Thread via GitHub
amogh-jahagirdar commented on PR #10350: URL: https://github.com/apache/iceberg/pull/10350#issuecomment-2118878167 Thanks for the review @flyrain ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Spark 3.5: Parallelize reading files in snapshot and migrate procedures [iceberg]

2024-05-18 Thread via GitHub
manuzhang commented on code in PR #10037: URL: https://github.com/apache/iceberg/pull/10037#discussion_r1605809547 ## data/src/main/java/org/apache/iceberg/data/MigrationService.java: ## @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [I] Add support for Python 3.12 [iceberg-python]

2024-05-18 Thread via GitHub
MehulBatra commented on issue #28: URL: https://github.com/apache/iceberg-python/issues/28#issuecomment-2118855200 https://github.com/apache/iceberg-python/pull/254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] Implement caching of manifest-files [iceberg-python]

2024-05-18 Thread via GitHub
MehulBatra commented on issue #595: URL: https://github.com/apache/iceberg-python/issues/595#issuecomment-2118850969 Hi @swapdewalkar I wanted to check in and see if you have any updates on this task. If you need any assistance or if there are any obstacles, please let me know—I will be hap

Re: [PR] support python 3.12 [iceberg-python]

2024-05-18 Thread via GitHub
MehulBatra commented on PR #254: URL: https://github.com/apache/iceberg-python/pull/254#issuecomment-2118850057 > @MehulBatra can you rebase? rebased it but It will still fail for 3.12 due to ray support, I am keeping a close eye on it. -- This is an automated message from the Apac

[PR] Kafka Connect: Commit coordination [iceberg]

2024-05-18 Thread via GitHub
bryanck opened a new pull request, #10351: URL: https://github.com/apache/iceberg/pull/10351 This PR is the next stage in submitting the Kafka Connect Iceberg sink connector, and is a follow up to https://github.com/apache/iceberg/pull/8701, https://github.com/apache/iceberg/pull/9466, and

Re: [I] `parquet_path_to_id_mapping` generates incorrect path for List types [iceberg-python]

2024-05-18 Thread via GitHub
felixscherz commented on issue #716: URL: https://github.com/apache/iceberg-python/issues/716#issuecomment-2118806030 Hi, I investigated this a bit further and it seems to be related to the way the visitor works as @cgbur suggested. Here is what I tried: ```python def test_parquet_

Re: [I] Rename IO traits to `IcebergFileRead` or `IcebergRead`? [iceberg-rust]

2024-05-18 Thread via GitHub
Xuanwo commented on issue #368: URL: https://github.com/apache/iceberg-rust/issues/368#issuecomment-2118771090 > To be honest I'm netural to this, is there any reason why we need to do this? I'm fine with this too. I'm going to let this issue open for more feedback. -- This is an a

Re: [I] Rename IO traits to `IcebergFileRead` or `IcebergRead`? [iceberg-rust]

2024-05-18 Thread via GitHub
liurenjie1024 commented on issue #368: URL: https://github.com/apache/iceberg-rust/issues/368#issuecomment-2118724538 To be honest I'm netural to this, is there any reason why we need to do this? -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] Core: Introduce AuthConfig [iceberg]

2024-05-18 Thread via GitHub
nastra commented on code in PR #10161: URL: https://github.com/apache/iceberg/pull/10161#discussion_r1605699574 ## core/src/main/java/org/apache/iceberg/rest/auth/AuthConfig.java: ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more