Re: [PR] [PoC] Test Flink and Spark 4.0 using Hive 4 metastore [iceberg]

2025-06-08 Thread via GitHub
pan3793 commented on PR #13262: URL: https://github.com/apache/iceberg/pull/13262#issuecomment-2954854688 According to changes in `hive-metastore/src/main`, I suppose the iceberg-hive-metastore.jar compiled by Hive 2.3.10 is also binary compatible with Hive 3.x and 4.x runtime jars? -- T

Re: [PR] [#13239]Remove extra \ in Kafka Connect Configuration docs [iceberg]

2025-06-08 Thread via GitHub
manuzhang commented on PR #13240: URL: https://github.com/apache/iceberg/pull/13240#issuecomment-2954746849 @raveendra11 It looks you are adding unrelated commits to this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Update `Iceberg Java API` doc [iceberg]

2025-06-08 Thread via GitHub
KyleLin0927 commented on issue #13249: URL: https://github.com/apache/iceberg/issues/13249#issuecomment-2954639775 ok, thanks for others’ contributions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Upgrade junit 5 to 5.13.x [iceberg]

2025-06-08 Thread via GitHub
manuzhang commented on issue #13278: URL: https://github.com/apache/iceberg/issues/13278#issuecomment-2954588139 @raveendra11 Please open a PR when it's ready. We avoid assigning in general. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [I] Upgrade junit 5 to 5.13.x [iceberg]

2025-06-08 Thread via GitHub
raveendra11 commented on issue #13278: URL: https://github.com/apache/iceberg/issues/13278#issuecomment-2954510948 @manuzhang I'll check on this, cloud you assign? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[I] Upgrade junit 5 to 5.13.x [iceberg]

2025-06-08 Thread via GitHub
manuzhang opened a new issue, #13278: URL: https://github.com/apache/iceberg/issues/13278 ### Feature Request / Improvement It looks junit 5.13.x has breaking changes compared with 5.12.x https://github.com/apache/iceberg/pull/13272 ``` JUnit Jupiter > initializati

Re: [PR] SPARK: Port prefix listing option in remove orphan files to Spark 3.4 and Spark 4.0 [iceberg]

2025-06-08 Thread via GitHub
liziyan-lzy commented on PR #13264: URL: https://github.com/apache/iceberg/pull/13264#issuecomment-2954464633 Hi @pvary, this is the follow-up PR that ports #12254 to Spark 3.4 and Spark 4.0. Please help me review it when you have some time. Thanks! -- This is an automated message from th

Re: [I] [Docs] reorganize tabs on the main website [iceberg]

2025-06-08 Thread via GitHub
manuzhang commented on issue #13261: URL: https://github.com/apache/iceberg/issues/13261#issuecomment-2954414409 We already have "Community" under the "Project" tab. "Blogs", "Talks", "Vendors" look broader, maybe "Ecosystem"? -- This is an automated message from the Apache Git Service. T

Re: [I] AWS: Support setting description for Glue table within buildTable [iceberg]

2025-06-08 Thread via GitHub
github-actions[bot] commented on issue #11735: URL: https://github.com/apache/iceberg/issues/11735#issuecomment-2954354807 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Kafka Connect: Add delta writer support [iceberg]

2025-06-08 Thread via GitHub
github-actions[bot] commented on PR #12070: URL: https://github.com/apache/iceberg/pull/12070#issuecomment-2954354883 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Core,REST: extend httpClient builder to support tls factory [iceberg]

2025-06-08 Thread via GitHub
github-actions[bot] commented on PR #11979: URL: https://github.com/apache/iceberg/pull/11979#issuecomment-2954354849 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Reading a table without schema IDs [iceberg]

2025-06-08 Thread via GitHub
github-actions[bot] commented on issue #11740: URL: https://github.com/apache/iceberg/issues/11740#issuecomment-2954354824 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Add partition expire for Iceberg [iceberg]

2025-06-08 Thread via GitHub
github-actions[bot] commented on issue #11686: URL: https://github.com/apache/iceberg/issues/11686#issuecomment-2954354769 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Build: Bump datamodel-code-generator from 0.30.1 to 0.30.2 [iceberg]

2025-06-08 Thread via GitHub
Fokko merged PR #13274: URL: https://github.com/apache/iceberg/pull/13274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] add iceberg datafusion integration [iceberg-python]

2025-06-08 Thread via GitHub
Fokko commented on code in PR #2075: URL: https://github.com/apache/iceberg-python/pull/2075#discussion_r2134819803 ## pyiceberg/table/__init__.py: ## @@ -1428,6 +1428,51 @@ def to_polars(self) -> pl.LazyFrame: return pl.scan_iceberg(self) +def __datafusion_tabl

[PR] add iceberg datafusion integration [iceberg-python]

2025-06-08 Thread via GitHub
kevinjqliu opened a new pull request, #2075: URL: https://github.com/apache/iceberg-python/pull/2075 # Rationale for this change Add pyiceberg table integration so that pyiceberg `Table` can be pass in directly to datafusion's `register_table_provider` # Are th

Re: [PR] Incremental Append Scan [iceberg-python]

2025-06-08 Thread via GitHub
smaheshwar-pltr commented on code in PR #2031: URL: https://github.com/apache/iceberg-python/pull/2031#discussion_r2134818675 ## pyiceberg/table/__init__.py: ## @@ -1507,16 +1566,17 @@ def _parse_row_filter(expr: Union[str, BooleanExpression]) -> BooleanExpression: return

Re: [PR] Fix projected fields predicate evaluation [iceberg-python]

2025-06-08 Thread via GitHub
Erigara commented on code in PR #2029: URL: https://github.com/apache/iceberg-python/pull/2029#discussion_r2134813546 ## pyiceberg/expressions/visitors.py: ## @@ -894,12 +895,17 @@ def visit_unbound_predicate(self, predicate: UnboundPredicate[L]) -> BooleanExpr def visit

Re: [PR] Build: Bump openapi-spec-validator from 0.7.1 to 0.7.2 [iceberg]

2025-06-08 Thread via GitHub
Fokko merged PR #13275: URL: https://github.com/apache/iceberg/pull/13275 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] Add Python 3.13 to the test matrix [iceberg-python]

2025-06-08 Thread via GitHub
jayceslesar commented on issue #1372: URL: https://github.com/apache/iceberg-python/issues/1372#issuecomment-2954246503 Pretty sure blocked on 3.13 because of https://github.com/ray-project/ray/issues/52819 -- This is an automated message from the Apache Git Service. To respond to the me

[PR] Bump mypy in pre-commit [iceberg-python]

2025-06-08 Thread via GitHub
Fokko opened a new pull request, #2074: URL: https://github.com/apache/iceberg-python/pull/2074 # Rationale for this change # Are these changes tested? # Are there any user-facing changes? -- This is an automated message from the Apache Git Ser

[PR] Bump Ruff to the latest version [iceberg-python]

2025-06-08 Thread via GitHub
Fokko opened a new pull request, #2073: URL: https://github.com/apache/iceberg-python/pull/2073 Makes the code look nicer # Rationale for this change # Are these changes tested? # Are there any user-facing changes? -- This is an automate

Re: [PR] only add `.db` suffix in warehouse location for dynamo/hive/glue catalogs [iceberg-python]

2025-06-08 Thread via GitHub
Fokko commented on code in PR #2059: URL: https://github.com/apache/iceberg-python/pull/2059#discussion_r2134806252 ## pyiceberg/catalog/dynamodb.py: ## @@ -652,6 +654,19 @@ def _convert_dynamo_table_item_to_iceberg_table(self, dynamo_table_item: Dict[st catalog=se

Re: [PR] Incremental Append Scan [iceberg-python]

2025-06-08 Thread via GitHub
Fokko commented on PR #2031: URL: https://github.com/apache/iceberg-python/pull/2031#issuecomment-2954232003 @smaheshwar-pltr First of all, sorry for the long wait, and thanks for picking this up. I'll have to look into this in more detail next week. It would be great to break the reviews i

Re: [PR] Incremental Append Scan [iceberg-python]

2025-06-08 Thread via GitHub
Fokko commented on code in PR #2031: URL: https://github.com/apache/iceberg-python/pull/2031#discussion_r2134798610 ## pyiceberg/table/__init__.py: ## @@ -1507,16 +1566,17 @@ def _parse_row_filter(expr: Union[str, BooleanExpression]) -> BooleanExpression: return parser.par

Re: [PR] Fix projected fields predicate evaluation [iceberg-python]

2025-06-08 Thread via GitHub
Fokko commented on code in PR #2029: URL: https://github.com/apache/iceberg-python/pull/2029#discussion_r2134797616 ## pyiceberg/expressions/visitors.py: ## @@ -894,12 +895,17 @@ def visit_unbound_predicate(self, predicate: UnboundPredicate[L]) -> BooleanExpr def visit_b

Re: [PR] fix: add metadata_properties to _construct_parameters when update hive table [iceberg-python]

2025-06-08 Thread via GitHub
Fokko commented on PR #2013: URL: https://github.com/apache/iceberg-python/pull/2013#issuecomment-2954221613 @kadai0308, there is an issue with the code formatting, can you run `make lint`? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] fix: add metadata_properties to _construct_parameters when update hive table [iceberg-python]

2025-06-08 Thread via GitHub
Fokko commented on code in PR #2013: URL: https://github.com/apache/iceberg-python/pull/2013#discussion_r2134796119 ## tests/integration/test_reads.py: ## @@ -112,6 +113,27 @@ def test_table_properties(catalog: Catalog) -> None: assert "None type is not a supported value in

Re: [PR] Part 1: Support Scan Planning in Rest Client [iceberg]

2025-06-08 Thread via GitHub
singhpk234 commented on code in PR #13004: URL: https://github.com/apache/iceberg/pull/13004#discussion_r2134789754 ## core/src/main/java/org/apache/iceberg/rest/RESTObjectMapper.java: ## @@ -26,7 +26,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxm

Re: [PR] Part 1: Support Scan Planning in Rest Client [iceberg]

2025-06-08 Thread via GitHub
singhpk234 commented on code in PR #13004: URL: https://github.com/apache/iceberg/pull/13004#discussion_r2134789754 ## core/src/main/java/org/apache/iceberg/rest/RESTObjectMapper.java: ## @@ -26,7 +26,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxm

Re: [PR] feat(transaction): Add update statistics [iceberg-rust]

2025-06-08 Thread via GitHub
CTTY commented on PR #1359: URL: https://github.com/apache/iceberg-rust/pull/1359#issuecomment-2954187575 I'm converting this to draft and will rebase this on the new transaction API later: https://github.com/apache/iceberg-rust/pull/1400 -- This is an automated message from the Apache Gi

Re: [I] Improve performance of filter pre-processing [iceberg-python]

2025-06-08 Thread via GitHub
jayceslesar commented on issue #2037: URL: https://github.com/apache/iceberg-python/issues/2037#issuecomment-2954118828 Potentially related? https://github.com/apache/iceberg-python/pull/1830 if you run your test before this commit was merged (try 0.9.0) (You might need to increase the rec

Re: [PR] Robustify boto3 session handling (DynamoDB, RestCatalog) [iceberg-python]

2025-06-08 Thread via GitHub
jayceslesar commented on code in PR #2071: URL: https://github.com/apache/iceberg-python/pull/2071#discussion_r2134724259 ## pyiceberg/catalog/rest/__init__.py: ## @@ -390,11 +392,17 @@ class SigV4Adapter(HTTPAdapter): def __init__(self, **properties: str):

Re: [I] `RestCatalog` with sigv4 enabled doesn't pick up creds provided as arguments [iceberg-python]

2025-06-08 Thread via GitHub
jayceslesar commented on issue #2070: URL: https://github.com/apache/iceberg-python/issues/2070#issuecomment-2954103989 also a duplicate of #2008 -- I can add to existing PR I have up -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Flink: Dynamic Iceberg Sink: Add HashKeyGenerator / RowDataEvolver / TableUpdateOperator [iceberg]

2025-06-08 Thread via GitHub
mxm opened a new pull request, #13277: URL: https://github.com/apache/iceberg/pull/13277 This change adds the following components for the Flink Dynamic Iceberg Sink: ### HashKeyGenerator A hash key generator which will be used in DynamicIcebergSink class (next PR) to implement

Re: [PR] Azure: Support multiple storage credential prefixes [iceberg]

2025-06-08 Thread via GitHub
ChaladiMohanVamsi commented on code in PR #13241: URL: https://github.com/apache/iceberg/pull/13241#discussion_r2134679023 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/PrefixedADLSFileClientBuilder.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Azure: Support multiple storage credential prefixes [iceberg]

2025-06-08 Thread via GitHub
ChaladiMohanVamsi commented on code in PR #13241: URL: https://github.com/apache/iceberg/pull/13241#discussion_r2134679809 ## azure/src/main/java/org/apache/iceberg/azure/adlsv2/PrefixedADLSFileClientBuilder.java: ## @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] AWS: KeyManagementClient implementation that works with AWS KMS [iceberg]

2025-06-08 Thread via GitHub
smaheshwar-pltr commented on code in PR #13136: URL: https://github.com/apache/iceberg/pull/13136#discussion_r2134663195 ## aws/src/main/java/org/apache/iceberg/aws/AwsKeyManagementClient.java: ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

[PR] chore(deps): Bump crate-ci/typos from 1.32.0 to 1.33.1 [iceberg-rust]

2025-06-08 Thread via GitHub
dependabot[bot] opened a new pull request, #1424: URL: https://github.com/apache/iceberg-rust/pull/1424 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.32.0 to 1.33.1. Release notes Sourced from https://github.com/crate-ci/typos/releases";>crate-ci/typos's release

[PR] chore(deps): Bump aws-sdk-glue from 1.98.0 to 1.99.0 [iceberg-rust]

2025-06-08 Thread via GitHub
dependabot[bot] opened a new pull request, #1423: URL: https://github.com/apache/iceberg-rust/pull/1423 Bumps [aws-sdk-glue](https://github.com/awslabs/aws-sdk-rust) from 1.98.0 to 1.99.0. Commits See full diff in https://github.com/awslabs/aws-sdk-rust/commits";>compare view

[PR] chore(deps): Bump aws-sdk-s3tables from 1.23.0 to 1.25.0 [iceberg-rust]

2025-06-08 Thread via GitHub
dependabot[bot] opened a new pull request, #1422: URL: https://github.com/apache/iceberg-rust/pull/1422 Bumps [aws-sdk-s3tables](https://github.com/awslabs/aws-sdk-rust) from 1.23.0 to 1.25.0. Commits See full diff in https://github.com/awslabs/aws-sdk-rust/commits";>compare vi

Re: [PR] AWS: KeyManagementClient implementation that works with AWS KMS [iceberg]

2025-06-08 Thread via GitHub
smaheshwar-pltr commented on code in PR #13136: URL: https://github.com/apache/iceberg/pull/13136#discussion_r2134655837 ## core/src/main/java/org/apache/iceberg/encryption/KeyManagementClient.java: ## @@ -24,7 +24,7 @@ import java.util.Map; /** A minimum client interface to

[PR] AWS: Fix double-checked-locking pattern in S3FileIO. [iceberg]

2025-06-08 Thread via GitHub
ChaladiMohanVamsi opened a new pull request, #13276: URL: https://github.com/apache/iceberg/pull/13276 ## Fix: double-checked-locking pattern in S3FileIO. ### Ensure Thread-Safe Initialization of `clientByPrefix` This PR addresses a thread-safety issue during the lazy initializa

[PR] build(deps): bump the gomod_updates group with 14 updates [iceberg-go]

2025-06-08 Thread via GitHub
dependabot[bot] opened a new pull request, #457: URL: https://github.com/apache/iceberg-go/pull/457 Bumps the gomod_updates group with 14 updates: | Package | From | To | | --- | --- | --- | | [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.29.1

Re: [I] Variant Data Type Support [iceberg]

2025-06-08 Thread via GitHub
Visorgood commented on issue #10392: URL: https://github.com/apache/iceberg/issues/10392#issuecomment-2953724003 Any update on what is the current state of the implementation of both part1 and part2? -- This is an automated message from the Apache Git Service. To respond to the message, p