Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1832195417 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -146,6 +151,26 @@ private Iterable materialize(CloseableIterable iterable) { @Over

Re: [PR] TableMetadataBuilder [iceberg-rust]

c-thiel commented on PR #587: URL: https://github.com/apache/iceberg-rust/pull/587#issuecomment-2461442365 @Xuanwo, @liurenjie1024 this PR is ready for another round of review. It's now rebased on the 6 PRs we merged during the last months. The core logic is ~1100 lines of code, including q

Re: [I] MinIO + Spark + hive metadata + iceberg format [iceberg]

mustafaaykon commented on issue #10222: URL: https://github.com/apache/iceberg/issues/10222#issuecomment-2461433487 Hi @rychu151 , I saw you passed you passed 'fs.s3a.endpoint' as localhost. I think, It shouldn't be localhost because hive is working on different container and MinIO is wo

[PR] feat: Add ViewUpdate to catalog [iceberg-rust]

c-thiel opened a new pull request, #690: URL: https://github.com/apache/iceberg-rust/pull/690 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] feat: Implement TableRequirement checks [iceberg-rust]

c-thiel commented on code in PR #689: URL: https://github.com/apache/iceberg-rust/pull/689#discussion_r1832067510 ## crates/iceberg/src/catalog/mod.rs: ## @@ -312,29 +312,29 @@ pub enum TableRequirement { LastAssignedFieldIdMatch { /// The last assigned field id of

Re: [PR] API: Add Variant data type [iceberg]

aihuaxu commented on code in PR #11324: URL: https://github.com/apache/iceberg/pull/11324#discussion_r1832033887 ## api/src/main/java/org/apache/iceberg/types/Type.java: ## @@ -92,6 +93,10 @@ default boolean isListType() { return false; } + default boolean isVariantTy

Re: [PR] AWS: Enable RetryMode for AWS KMS client [iceberg]

hsiang-c commented on PR #11420: URL: https://github.com/apache/iceberg/pull/11420#issuecomment-2461269631 @danielcweeks Thank you for your review. > Is this really necessary to make configurable? I am also working on an enhancement to Iceberg S3 doc: https://github.com/apache/

[PR] feat: TableMetadata accessors for current ids of Schema, Snapshot and SortOrder [iceberg-rust]

c-thiel opened a new pull request, #688: URL: https://github.com/apache/iceberg-rust/pull/688 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Spark 3.5: Iceberg parser should passthrough unsupported procedure to delegate [iceberg]

pan3793 commented on PR #11480: URL: https://github.com/apache/iceberg/pull/11480#issuecomment-2461261860 cc @RussellSpitzer @Fokko @aokolnychyi could you please take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] `system.add_files` utility does not support updated Partition Spec [iceberg]

github-actions[bot] closed issue #10008: `system.add_files` utility does not support updated Partition Spec URL: https://github.com/apache/iceberg/issues/10008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Improvements for manifest file caching [iceberg]

github-actions[bot] closed issue #9991: Improvements for manifest file caching URL: https://github.com/apache/iceberg/issues/9991 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Spark: Reconcile derived partitioning from source table with target table specs in AddFilesProcedure [iceberg]

github-actions[bot] closed pull request #10133: Spark: Reconcile derived partitioning from source table with target table specs in AddFilesProcedure URL: https://github.com/apache/iceberg/pull/10133 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [I] Cannot access table endpoint in REST catalog when table name contains a slash character (`/`) [iceberg-python]

github-actions[bot] commented on issue #710: URL: https://github.com/apache/iceberg-python/issues/710#issuecomment-2461063167 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [PR] Flink-1.19: Fix the file offset mismatch when Flink reader first seek… [iceberg]

github-actions[bot] commented on PR #10567: URL: https://github.com/apache/iceberg/pull/10567#issuecomment-2461061156 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Flink: Maintenance - CommitConverter [iceberg]

github-actions[bot] commented on issue #10302: URL: https://github.com/apache/iceberg/issues/10302#issuecomment-2461060754 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Flink: Maintenance - RewriteManifestFiles [iceberg]

github-actions[bot] commented on issue #10305: URL: https://github.com/apache/iceberg/issues/10305#issuecomment-2461060840 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Flink: Maintenance - DeleteOrphanFiles [iceberg]

github-actions[bot] commented on issue #10306: URL: https://github.com/apache/iceberg/issues/10306#issuecomment-2461060871 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] Flink: Maintenance - RewriteDataFiles [iceberg]

github-actions[bot] commented on issue #10303: URL: https://github.com/apache/iceberg/issues/10303#issuecomment-2461060792 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [I] how do you guys back up your iceberg table? [iceberg]

github-actions[bot] commented on issue #10299: URL: https://github.com/apache/iceberg/issues/10299#issuecomment-2461060712 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] Spec: Make NDV blob metadata property required [iceberg]

github-actions[bot] commented on PR #10549: URL: https://github.com/apache/iceberg/pull/10549#issuecomment-2461061133 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] `system.add_files` utility does not support updated Partition Spec [iceberg]

github-actions[bot] commented on issue #10008: URL: https://github.com/apache/iceberg/issues/10008#issuecomment-2461060375 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] Doc: Spark quickstart needs to create context directory first [iceberg]

github-actions[bot] commented on PR #10572: URL: https://github.com/apache/iceberg/pull/10572#issuecomment-2461061180 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Track uncompressed data size for column metrics [iceberg]

github-actions[bot] closed issue #9966: Track uncompressed data size for column metrics URL: https://github.com/apache/iceberg/issues/9966 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] Improvements for manifest file caching [iceberg]

github-actions[bot] commented on issue #9991: URL: https://github.com/apache/iceberg/issues/9991#issuecomment-2461060336 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] S3 InputsStream: Reopen connection on Connection Reset [iceberg]

github-actions[bot] commented on PR #10470: URL: https://github.com/apache/iceberg/pull/10470#issuecomment-2461061102 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] Spark can not delete table metadata and data when drop table [iceberg]

github-actions[bot] commented on issue #9990: URL: https://github.com/apache/iceberg/issues/9990#issuecomment-2461060320 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] An error occurred when the iceberg parquet file was loaded in the hive external table [iceberg]

github-actions[bot] closed issue #10005: An error occurred when the iceberg parquet file was loaded in the hive external table URL: https://github.com/apache/iceberg/issues/10005 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Spark: Reconcile derived partitioning from source table with target table specs in AddFilesProcedure [iceberg]

github-actions[bot] commented on PR #10133: URL: https://github.com/apache/iceberg/pull/10133#issuecomment-2461060460 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [I] An error occurred when the iceberg parquet file was loaded in the hive external table [iceberg]

github-actions[bot] commented on issue #10005: URL: https://github.com/apache/iceberg/issues/10005#issuecomment-2461060358 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [I] Spark can not delete table metadata and data when drop table [iceberg]

github-actions[bot] closed issue #9990: Spark can not delete table metadata and data when drop table URL: https://github.com/apache/iceberg/issues/9990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] Track uncompressed data size for column metrics [iceberg]

github-actions[bot] commented on issue #9966: URL: https://github.com/apache/iceberg/issues/9966#issuecomment-2461060294 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Is there any performance report on iceberg（Merge on Read） [iceberg]

github-actions[bot] closed issue #9959: Is there any performance report on iceberg（Merge on Read） URL: https://github.com/apache/iceberg/issues/9959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Is there any way to define Iceberg catalog and share it between DataStream API and Table/SQL API? [iceberg]

github-actions[bot] closed issue #9954: Is there any way to define Iceberg catalog and share it between DataStream API and Table/SQL API? URL: https://github.com/apache/iceberg/issues/9954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Is there any way to define Iceberg catalog and share it between DataStream API and Table/SQL API? [iceberg]

github-actions[bot] commented on issue #9954: URL: https://github.com/apache/iceberg/issues/9954#issuecomment-2461060235 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] AWS: Change S3FileIO to use SHA1 based checksums [iceberg]

muddyfish closed pull request #10293: AWS: Change S3FileIO to use SHA1 based checksums URL: https://github.com/apache/iceberg/pull/10293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] API: Add Variant data type [iceberg]

aihuaxu commented on code in PR #11324: URL: https://github.com/apache/iceberg/pull/11324#discussion_r1831845888 ## api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java: ## @@ -562,7 +563,7 @@ private static String sanitize(Literal literal, long now, int today)

Re: [I] Support dynamic overwrite [iceberg-python]

koenvo commented on issue #1287: URL: https://github.com/apache/iceberg-python/issues/1287#issuecomment-2460958032 Ah good question. In our normal process the Iceberg tables are only queried using our own application. The application will always (for now at least) use the latest snapshot.

Re: [PR] Flink: Fix config key typo in error message of SplitComparators [iceberg]

szehon-ho commented on PR #11482: URL: https://github.com/apache/iceberg/pull/11482#issuecomment-2460945441 Thanks @liuml07 and @huaxingao for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Flink: Fix config key typo in error message of SplitComparators [iceberg]

szehon-ho merged PR #11482: URL: https://github.com/apache/iceberg/pull/11482 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

danielcweeks commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831802198 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -146,6 +151,26 @@ private Iterable materialize(CloseableIterable iterable) { @Ove

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

danielcweeks commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831797637 ## data/src/test/java/org/apache/iceberg/io/TestDVWriters.java: ## @@ -100,6 +114,211 @@ public void testBasicDVs() throws IOException { .contains(dataF

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

danielcweeks commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831795720 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -259,4 +284,46 @@ private long estimateEqDeletesSize(DeleteFile deleteFile, Schema pr

[PR] Bump getdaft from 0.3.9 to 0.3.10 [iceberg-python]

dependabot[bot] opened a new pull request, #1303: URL: https://github.com/apache/iceberg-python/pull/1303 Bumps [getdaft](https://github.com/Eventual-Inc/Daft) from 0.3.9 to 0.3.10. Release notes Sourced from https://github.com/Eventual-Inc/Daft/releases";>getdaft's releases.

[PR] Bump mkdocstrings-python from 1.11.1 to 1.12.2 [iceberg-python]

dependabot[bot] opened a new pull request, #1302: URL: https://github.com/apache/iceberg-python/pull/1302 Bumps [mkdocstrings-python](https://github.com/mkdocstrings/python) from 1.11.1 to 1.12.2. Release notes Sourced from https://github.com/mkdocstrings/python/releases";>mkdocstr

[I] Cannot connect and read glue iceberg tables with hyphens [iceberg]

noah-instructure opened a new issue, #11483: URL: https://github.com/apache/iceberg/issues/11483 ### Apache Iceberg version 1.6.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 I have several glue databases with hyphens in them, create

Re: [PR] REST: Docker file for Rest catalog adapter image [iceberg]

Fokko commented on code in PR #11283: URL: https://github.com/apache/iceberg/pull/11283#discussion_r1831747697 ## docker/iceberg-rest-adapter-image/README.md: ## @@ -0,0 +1,87 @@ + + +# Iceberg rest adapter image + +For converting different catalog implementations into a rest on

Re: [PR] REST: Docker file for Rest catalog adapter image [iceberg]

Fokko commented on code in PR #11283: URL: https://github.com/apache/iceberg/pull/11283#discussion_r1831744527 ## docker/iceberg-rest-adapter-image/README.md: ## @@ -0,0 +1,87 @@ + + +# Iceberg rest adapter image + +For converting different catalog implementations into a rest on

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831747517 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -259,4 +284,46 @@ private long estimateEqDeletesSize(DeleteFile deleteFile, Schema pro

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831746989 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -259,4 +284,46 @@ private long estimateEqDeletesSize(DeleteFile deleteFile, Schema pro

Re: [PR] REST: Docker file for Rest catalog adapter image [iceberg]

Fokko commented on code in PR #11283: URL: https://github.com/apache/iceberg/pull/11283#discussion_r1831747118 ## docker/iceberg-rest-adapter-image/README.md: ## @@ -0,0 +1,87 @@ + + +# Iceberg rest adapter image + +For converting different catalog implementations into a rest on

Re: [PR] REST: Docker file for Rest catalog adapter image [iceberg]

Fokko commented on code in PR #11283: URL: https://github.com/apache/iceberg/pull/11283#discussion_r1831746438 ## docker/iceberg-rest-adapter-image/README.md: ## @@ -0,0 +1,87 @@ + + +# Iceberg rest adapter image + +For converting different catalog implementations into a rest on

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831735172 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -146,6 +151,26 @@ private Iterable materialize(CloseableIterable iterable) { @Over

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831746215 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -146,6 +151,26 @@ private Iterable materialize(CloseableIterable iterable) { @Over

Re: [PR] REST: Docker file for Rest catalog adapter image [iceberg]

Fokko commented on code in PR #11283: URL: https://github.com/apache/iceberg/pull/11283#discussion_r1831745673 ## docker/iceberg-rest-adapter-image/README.md: ## @@ -0,0 +1,87 @@ + + +# Iceberg rest adapter image Review Comment: ```suggestion # Iceberg Technology Compatib

Re: [PR] REST: Docker file for Rest catalog adapter image [iceberg]

Fokko commented on code in PR #11283: URL: https://github.com/apache/iceberg/pull/11283#discussion_r1831743889 ## docker/iceberg-rest-adapter-image/Dockerfile: ## @@ -0,0 +1,43 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agr

[PR] [flink] Fix config key typo in error message of SplitComparators [iceberg]

liuml07 opened a new pull request, #11482: URL: https://github.com/apache/iceberg/pull/11482 `split-open-file-cost` should be `split-file-open-cost`. Or simply use the const variable for constructing the error message. Also updated the test that asserts error message. -- This is an autom

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831737522 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -146,6 +151,26 @@ private Iterable materialize(CloseableIterable iterable) { @Over

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831735172 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -146,6 +151,26 @@ private Iterable materialize(CloseableIterable iterable) { @Over

Re: [PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi commented on code in PR #11481: URL: https://github.com/apache/iceberg/pull/11481#discussion_r1831735172 ## data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java: ## @@ -146,6 +151,26 @@ private Iterable materialize(CloseableIterable iterable) { @Over

[PR] Core: Support DVs in DeleteLoader [iceberg]

aokolnychyi opened a new pull request, #11481: URL: https://github.com/apache/iceberg/pull/11481 This PR adds support for reading DVs in `BaseDeleteLoader`, the only loader implementation we have. This work is part of #11122. -- This is an automated message from the Apache Git Serv

Re: [PR] Modify S3 config naming convention [iceberg-python]

kevinjqliu merged PR #1301: URL: https://github.com/apache/iceberg-python/pull/1301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] API: Removes Explicit Parameterization of Schema Tests [iceberg]

RussellSpitzer commented on code in PR #11444: URL: https://github.com/apache/iceberg/pull/11444#discussion_r1831723691 ## api/src/test/java/org/apache/iceberg/TestSchema.java: ## @@ -95,14 +140,21 @@ public void testUnsupportedInitialDefault(int formatVersion) { f

Re: [PR] Add support for boolean expressions and quoted columns [iceberg-python]

Fokko merged PR #1286: URL: https://github.com/apache/iceberg-python/pull/1286 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] API: Removes Explicit Parameterization of Schema Tests [iceberg]

RussellSpitzer commented on code in PR #11444: URL: https://github.com/apache/iceberg/pull/11444#discussion_r1831723193 ## api/src/test/java/org/apache/iceberg/TestSchema.java: ## @@ -18,32 +18,27 @@ */ package org.apache.iceberg; +import static org.apache.iceberg.Schema.DE

Re: [PR] Do not deprecate `botocore_session` [iceberg-python]

kevinjqliu commented on code in PR #1300: URL: https://github.com/apache/iceberg-python/pull/1300#discussion_r1831713834 ## pyiceberg/catalog/dynamodb.py: ## @@ -98,6 +99,7 @@ def __init__(self, name: str, **properties: str): session = boto3.Session( profil

Re: [PR] Core, Puffin: Add DV file writer [iceberg]

aokolnychyi commented on PR #11476: URL: https://github.com/apache/iceberg/pull/11476#issuecomment-2460726490 Thanks for reviewing, @nastra @danielcweeks @jbonofre @rdblue! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Modify S3 config naming convention [iceberg-python]

Fokko commented on PR #1301: URL: https://github.com/apache/iceberg-python/pull/1301#issuecomment-2460696317 @kevinjqliu Great catch, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Core, Puffin: Add DV file writer [iceberg]

aokolnychyi merged PR #11476: URL: https://github.com/apache/iceberg/pull/11476 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Do not deprecate `botocore_session` [iceberg-python]

Fokko commented on code in PR #1300: URL: https://github.com/apache/iceberg-python/pull/1300#discussion_r1831655564 ## pyiceberg/catalog/dynamodb.py: ## @@ -98,6 +99,7 @@ def __init__(self, name: str, **properties: str): session = boto3.Session( profile_nam

Re: [PR] Core, Puffin: Add DV file writer [iceberg]

aokolnychyi commented on code in PR #11476: URL: https://github.com/apache/iceberg/pull/11476#discussion_r1831651242 ## data/src/test/java/org/apache/iceberg/io/TestDVWriters.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

[PR] Modify S3 config naming convention [iceberg-python]

kevinjqliu opened a new pull request, #1301: URL: https://github.com/apache/iceberg-python/pull/1301 Add `role-` prefix to `session-name` (`session-name` -> `role-session-name`) Modified `aws.` prefix to `client.` prefix (`aws.role-arn` -> `client.role-arn`) `role-arn` and `ses

Re: [I] ERROR when executing UPDATE/DELETE queries in Iceberg 1.6.0: "Cannot add fieldId 1 as an identifier field" [iceberg]

meatheadmike commented on issue #11341: URL: https://github.com/apache/iceberg/issues/11341#issuecomment-2460554308 This bug does not appear to be limited to AWS nor Flink. I'm getting the same error with the following: ``` spark.sql(f""" CREATE EXTERNAL TABLE IF NOT EXISTS

Re: [PR] Add support for boolean expressions and quoted columns [iceberg-python]

MoSheikh commented on PR #1286: URL: https://github.com/apache/iceberg-python/pull/1286#issuecomment-2460417417 > we're in the process of releasing 0.8. We **could** include this PR Would appreciate that, thank you! I just removed that comment I missed. -- This is an automated messa

Re: [PR] Add support for boolean expressions and quoted columns [iceberg-python]

MoSheikh commented on code in PR #1286: URL: https://github.com/apache/iceberg-python/pull/1286#discussion_r1831475292 ## pyiceberg/expressions/parser.py: ## @@ -77,7 +79,10 @@ NAN = CaselessKeyword("nan") LIKE = CaselessKeyword("like") -identifier = Word(alphas, alphanums +

Re: [PR] Rename `gcs.endpoint` to `gcs.service.host` [iceberg-python]

kevinjqliu merged PR #1007: URL: https://github.com/apache/iceberg-python/pull/1007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@i

Re: [PR] Glue: Allow for assuming role for Glue [iceberg-python]

kevinjqliu commented on code in PR #1299: URL: https://github.com/apache/iceberg-python/pull/1299#discussion_r1831454666 ## pyiceberg/io/__init__.py: ## @@ -61,7 +61,7 @@ AWS_SECRET_ACCESS_KEY = "client.secret-access-key" AWS_SESSION_TOKEN = "client.session-token" AWS_ROLE_AR

Re: [PR] Glue: Allow for assuming role for Glue [iceberg-python]

kevinjqliu commented on code in PR #1299: URL: https://github.com/apache/iceberg-python/pull/1299#discussion_r1831452825 ## pyiceberg/catalog/glue.py: ## @@ -296,13 +306,48 @@ class GlueCatalog(MetastoreCatalog): def __init__(self, name: str, **properties: Any): su

Re: [I] PyIceberg Cookbook [iceberg-python]

kevinjqliu commented on issue #1201: URL: https://github.com/apache/iceberg-python/issues/1201#issuecomment-2460351486 hey @francocalvo the `MERGE` operation is not yet support (https://github.com/apache/iceberg-python/issues/402) For write, pyiceberg currently supports `append` and

Re: [PR] Add support for boolean expressions and quoted columns [iceberg-python]

kevinjqliu commented on PR #1286: URL: https://github.com/apache/iceberg-python/pull/1286#issuecomment-2460336376 we're in the process of releasing 0.8. We **could** include this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Add support for boolean expressions and quoted columns [iceberg-python]

kevinjqliu commented on code in PR #1286: URL: https://github.com/apache/iceberg-python/pull/1286#discussion_r1830333733 ## pyiceberg/expressions/parser.py: ## @@ -77,7 +79,10 @@ NAN = CaselessKeyword("nan") LIKE = CaselessKeyword("like") -identifier = Word(alphas, alphanums

Re: [I] Support dynamic overwrite [iceberg-python]

kevinjqliu commented on issue #1287: URL: https://github.com/apache/iceberg-python/issues/1287#issuecomment-2460326291 yes, thats right. This will create 2 snapshots and if you time travel to the first one, you will only see the table with the data deleted. What if your use case here to

Re: [I] [feat] Support update table's sort order [iceberg-python]

kevinjqliu commented on issue #1245: URL: https://github.com/apache/iceberg-python/issues/1245#issuecomment-2460323090 i didnt find any similar ticket. renamed and assigned to you. Cheers! Please LMK if you have any questions -- This is an automated message from the Apache Git Service.

Re: [PR] Glue: Allow for assuming role for Glue [iceberg-python]

cshenrik commented on code in PR #1299: URL: https://github.com/apache/iceberg-python/pull/1299#discussion_r1831031255 ## pyiceberg/catalog/glue.py: ## @@ -296,13 +306,48 @@ class GlueCatalog(MetastoreCatalog): def __init__(self, name: str, **properties: Any): supe

Re: [PR] Add support for boolean expressions and quoted columns [iceberg-python]

MoSheikh commented on PR #1286: URL: https://github.com/apache/iceberg-python/pull/1286#issuecomment-2459604586 @Fokko Not to rush, but was wondering was the timeline typically is for a fix like this to get merged in and released? Thank you :) -- This is an automated message from the Apac

Re: [PR] Core: Try create Iceberg metadata table for Jdbc catalog in initialization [iceberg]

FANNG1 commented on code in PR #11427: URL: https://github.com/apache/iceberg/pull/11427#discussion_r1830702389 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -123,7 +123,7 @@ enum SchemaVersion { + JdbcTableOperations.METADATA_LOCATION_PROP

Re: [PR] Glue: Allow for assuming role for Glue [iceberg-python]

Fokko commented on code in PR #1299: URL: https://github.com/apache/iceberg-python/pull/1299#discussion_r1831238850 ## pyiceberg/catalog/glue.py: ## @@ -296,13 +306,48 @@ class GlueCatalog(MetastoreCatalog): def __init__(self, name: str, **properties: Any): super()

Re: [I] Support Vended Credentials for Azure Data Lake Store [iceberg-python]

Fokko commented on issue #1146: URL: https://github.com/apache/iceberg-python/issues/1146#issuecomment-2460085783 Hey @sfc-gh-tbenroeck, thanks for uncovering this bug. Are you interested in creating a PR to fix this? It looks like you're almost there. -- This is an automated message fro

Re: [PR] Glue: Allow for assuming role for Glue [iceberg-python]

Fokko commented on code in PR #1299: URL: https://github.com/apache/iceberg-python/pull/1299#discussion_r1831238850 ## pyiceberg/catalog/glue.py: ## @@ -296,13 +306,48 @@ class GlueCatalog(MetastoreCatalog): def __init__(self, name: str, **properties: Any): super()

Re: [PR] Docs: Fix verifying release candidate with Spark and Flink [iceberg]

nastra commented on code in PR #11461: URL: https://github.com/apache/iceberg/pull/11461#discussion_r1831223316 ## site/docs/how-to-release.md: ## @@ -422,7 +422,7 @@ spark-runtime jar for the Spark installation): ```bash spark-shell \ --conf spark.jars.repositories=${MAV

Re: [PR] Glue: Allow for assuming role for Glue [iceberg-python]

cshenrik commented on code in PR #1299: URL: https://github.com/apache/iceberg-python/pull/1299#discussion_r1831036603 ## pyiceberg/catalog/glue.py: ## @@ -296,13 +306,48 @@ class GlueCatalog(MetastoreCatalog): def __init__(self, name: str, **properties: Any): supe

Re: [PR] Docs: Fix verifying release candidate with Spark and Flink [iceberg]

manuzhang commented on code in PR #11461: URL: https://github.com/apache/iceberg/pull/11461#discussion_r1831149038 ## site/docs/how-to-release.md: ## @@ -422,7 +422,7 @@ spark-runtime jar for the Spark installation): ```bash spark-shell \ --conf spark.jars.repositories=${

Re: [PR] Docs: Fix verifying release candidate with Spark and Flink [iceberg]

nastra commented on code in PR #11461: URL: https://github.com/apache/iceberg/pull/11461#discussion_r1831126084 ## site/docs/how-to-release.md: ## @@ -422,7 +422,7 @@ spark-runtime jar for the Spark installation): ```bash spark-shell \ --conf spark.jars.repositories=${MAV

Re: [PR] Docs: Fix verifying release candidate with Spark and Flink [iceberg]

manuzhang commented on code in PR #11461: URL: https://github.com/apache/iceberg/pull/11461#discussion_r1831092850 ## site/docs/how-to-release.md: ## @@ -422,7 +422,7 @@ spark-runtime jar for the Spark installation): ```bash spark-shell \ --conf spark.jars.repositories=${

Re: [I] PyIceberg Cookbook [iceberg-python]

francocalvo commented on issue #1201: URL: https://github.com/apache/iceberg-python/issues/1201#issuecomment-2459743291 Hey! I'm creating a PoC using PyIceberg for a project. I'm quite interested in incremental processing. For this, what I've used before were MERGE operations to upda

Re: [PR] Spark-3.5: make `where` sql case sensitive setting alterable in rewrite data files procedure [iceberg]

ludlows commented on PR #11439: URL: https://github.com/apache/iceberg/pull/11439#issuecomment-2459142794 1. It seems to me that the tests don't really test the changes in this PR; they would pass even without the fix. I think we should add some tests that would fail without the fix but can

Re: [PR] Core, Puffin: Add DV file writer [iceberg]

aokolnychyi commented on code in PR #11476: URL: https://github.com/apache/iceberg/pull/11476#discussion_r1830585211 ## data/src/test/java/org/apache/iceberg/io/TestDVWriters.java: ## @@ -0,0 +1,129 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mor

Re: [PR] REST: Docker file for Rest catalog adapter image [iceberg]

ajantha-bhat commented on code in PR #11283: URL: https://github.com/apache/iceberg/pull/11283#discussion_r1830654063 ## build.gradle: ## @@ -985,6 +985,15 @@ project(':iceberg-open-api') { exclude group: 'org.apache.commons', module: 'commons-configuration2' exclu

Re: [I] Adaptive retry for Glue Catalog calls to fix Glue throttling [iceberg-python]

mark-major commented on issue #1294: URL: https://github.com/apache/iceberg-python/issues/1294#issuecomment-2459021591 @kevinjqliu Yes, we'll need the standard retry mode and the ability to configure the max attempts. -- This is an automated message from the Apache Git Service. To respon

Re: [I] Support for Nessie Rest s3 signer [iceberg-python]

Fokko closed issue #1028: Support for Nessie Rest s3 signer URL: https://github.com/apache/iceberg-python/issues/1028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] Core: Try create Iceberg metadata table for Jdbc catalog in initialization [iceberg]

Fokko commented on code in PR #11427: URL: https://github.com/apache/iceberg/pull/11427#discussion_r1830596375 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -123,7 +123,7 @@ enum SchemaVersion { + JdbcTableOperations.METADATA_LOCATION_PROP

  1   2   >