[PR] Upgrade to Gradle 8.14.1 [iceberg]

2025-05-24 Thread via GitHub
jbonofre opened a new pull request, #13149: URL: https://github.com/apache/iceberg/pull/13149 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[PR] Build: Bump software.amazon.awssdk:bom from 2.31.45 to 2.31.50 [iceberg]

2025-05-24 Thread via GitHub
dependabot[bot] opened a new pull request, #13148: URL: https://github.com/apache/iceberg/pull/13148 Bumps software.amazon.awssdk:bom from 2.31.45 to 2.31.50. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=soft

[PR] Build: Bump com.google.cloud:libraries-bom from 26.60.0 to 26.61.0 [iceberg]

2025-05-24 Thread via GitHub
dependabot[bot] opened a new pull request, #13146: URL: https://github.com/apache/iceberg/pull/13146 Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.60.0 to 26.61.0. Release notes Sourced from https://github.com/googleapis/java-cloud-bo

[PR] Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4.4 to 5.5 [iceberg]

2025-05-24 Thread via GitHub
dependabot[bot] opened a new pull request, #13147: URL: https://github.com/apache/iceberg/pull/13147 Bumps [org.apache.httpcomponents.client5:httpclient5](https://github.com/apache/httpcomponents-client) from 5.4.4 to 5.5. Changelog Sourced from https://github.com/apache/httpcompo

[PR] Build: Bump kafka from 3.9.0 to 3.9.1 [iceberg]

2025-05-24 Thread via GitHub
dependabot[bot] opened a new pull request, #13145: URL: https://github.com/apache/iceberg/pull/13145 Bumps `kafka` from 3.9.0 to 3.9.1. Updates `org.apache.kafka:kafka-clients` from 3.9.0 to 3.9.1 Updates `org.apache.kafka:connect-api` from 3.9.0 to 3.9.1 Updates `org.apache.

[PR] Build: Bump com.palantir.gradle.gitversion:gradle-git-version from 3.2.0 to 3.3.0 [iceberg]

2025-05-24 Thread via GitHub
dependabot[bot] opened a new pull request, #13144: URL: https://github.com/apache/iceberg/pull/13144 Bumps [com.palantir.gradle.gitversion:gradle-git-version](https://github.com/palantir/gradle-git-version) from 3.2.0 to 3.3.0. Release notes Sourced from https://github.com/palanti

[PR] test: Add missing tests for update_namespace method in sql catalog [iceberg-rust]

2025-05-24 Thread via GitHub
kyteware opened a new pull request, #1373: URL: https://github.com/apache/iceberg-rust/pull/1373 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Adds tests for the `update_namespace` method in the SQL catalog crate t

Re: [I] `ALTER TABLE ... DROP COLUMN` allows dropping a column used by old PartitionSpecs [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on issue #4563: URL: https://github.com/apache/iceberg/issues/4563#issuecomment-2907520159 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Iceberg time type fix [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] closed pull request #12725: Iceberg time type fix URL: https://github.com/apache/iceberg/pull/12725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] AVRO: Support UUID logical type on string fields in Avro schema [iceberg]

2025-05-24 Thread via GitHub
vanshb03 commented on PR #12877: URL: https://github.com/apache/iceberg/pull/12877#issuecomment-2907520623 Bumping branch for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Spark: RewriteTablePath: filter content files by snapshotId [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on PR #12885: URL: https://github.com/apache/iceberg/pull/12885#issuecomment-2907520293 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Iceberg time type fix [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on PR #12725: URL: https://github.com/apache/iceberg/pull/12725#issuecomment-2907520254 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] AVRO: Support UUID logical type on string fields in Avro schema [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on PR #12877: URL: https://github.com/apache/iceberg/pull/12877#issuecomment-2907520282 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] `ALTER TABLE ... DROP COLUMN` allows dropping a column used by old PartitionSpecs [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] closed issue #4563: `ALTER TABLE ... DROP COLUMN` allows dropping a column used by old PartitionSpecs URL: https://github.com/apache/iceberg/issues/4563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] How to convert STRING fields to STRUCT? [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on issue #11512: URL: https://github.com/apache/iceberg/issues/11512#issuecomment-2907520197 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache

Re: [PR] Doc: Hint implications on use write.data.path and orphan removal together [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on PR #12890: URL: https://github.com/apache/iceberg/pull/12890#issuecomment-2907520303 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] How to convert STRING fields to STRUCT? [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] closed issue #11512: How to convert STRING fields to STRUCT? URL: https://github.com/apache/iceberg/issues/11512 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [PR] Reduce code duplication in VectorizedParquetDefinitionLevelReader [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on PR #11661: URL: https://github.com/apache/iceberg/pull/11661#issuecomment-2907520222 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on PR #11039: URL: https://github.com/apache/iceberg/pull/11039#issuecomment-2907520184 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] GCP: Add Iceberg Catalog for GCP BigQuery Metastore [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] closed pull request #11039: GCP: Add Iceberg Catalog for GCP BigQuery Metastore URL: https://github.com/apache/iceberg/pull/11039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Review new DangerousJavaDeserialization error-prone check [iceberg]

2025-05-24 Thread via GitHub
github-actions[bot] commented on issue #10853: URL: https://github.com/apache/iceberg/issues/10853#issuecomment-2907520166 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] fix(table): ensure manifest length matches file size [iceberg-go]

2025-05-24 Thread via GitHub
alexandre-normand commented on PR #439: URL: https://github.com/apache/iceberg-go/pull/439#issuecomment-2907485381 Confirmed the manifest's length matches the actual file size in s3 now. Querying the manifests from trino now shows files that aren't all the same fixed length (which we had pr

Re: [PR] fix(table): ensure manifest length matches file size [iceberg-go]

2025-05-24 Thread via GitHub
alexandre-normand commented on PR #439: URL: https://github.com/apache/iceberg-go/pull/439#issuecomment-2907020971 > @alexandre-normand @alexjo2144 can one or both of you please try out this PR and confirm it fixes your issue? I added tests and hit the spots I could find, but it would be gr

[PR] fix(table): ensure manifest length matches file size [iceberg-go]

2025-05-24 Thread via GitHub
zeroshade opened a new pull request, #439: URL: https://github.com/apache/iceberg-go/pull/439 fixes #438 We need to ensure that we flush the manifest that we're writing before we construct the `ManifestFile` object we want to add to the snapshot so that our length is correct when we

Re: [PR] fix(table): ensure manifest length matches file size [iceberg-go]

2025-05-24 Thread via GitHub
zeroshade commented on PR #439: URL: https://github.com/apache/iceberg-go/pull/439#issuecomment-2907015622 @alexandre-normand @alexjo2144 can one or both of you please try out this PR and confirm it fixes your issue? I added tests and hit the spots I could find, but it would be great to get

Re: [I] Manifest list written by go-iceberg contains incorrect file lengths [iceberg-go]

2025-05-24 Thread via GitHub
zeroshade commented on issue #438: URL: https://github.com/apache/iceberg-go/issues/438#issuecomment-2907015860 I put up the PR and tagged you guys! Go try it out and just confirm it fixes the issue for you guys please! :smile: -- This is an automated message from the Apache Git Service.

Re: [I] Manifest list written by go-iceberg contains incorrect file lengths [iceberg-go]

2025-05-24 Thread via GitHub
alexandre-normand commented on issue #438: URL: https://github.com/apache/iceberg-go/issues/438#issuecomment-2907008517 > *UPDATE:* Yup, I've confirmed this is the issue. We're passing the value of Count before the flush occurs so the value is incorrect when we write it. I'll put together a

Re: [I] Manifest list written by go-iceberg contains incorrect file lengths [iceberg-go]

2025-05-24 Thread via GitHub
zeroshade commented on issue #438: URL: https://github.com/apache/iceberg-go/issues/438#issuecomment-2907007693 *UPDATE:* Yup, I've confirmed this is the issue. We're passing the value of Count before the flush occurs so the value is incorrect when we write it. I'll put together a fix and p

Re: [PR] REST: Avoid table corruption on unhandled RestException [iceberg]

2025-05-24 Thread via GitHub
amogh-jahagirdar commented on code in PR #12818: URL: https://github.com/apache/iceberg/pull/12818#discussion_r2105889910 ## core/src/main/java/org/apache/iceberg/rest/ErrorHandlers.java: ## @@ -91,9 +91,22 @@ public void accept(ErrorResponse error) { case 404:

Re: [I] Manifest list written by go-iceberg contains incorrect file lengths [iceberg-go]

2025-05-24 Thread via GitHub
zeroshade commented on issue #438: URL: https://github.com/apache/iceberg-go/issues/438#issuecomment-2906997287 So, looking at the code a bit. One thing that stands out to me as a potential cause of this is that we do `defer counter.W.(io.Closer).Close()` while passing `counter.Count` to th

Re: [PR] Core: Make pageToken query parameter optional [iceberg]

2025-05-24 Thread via GitHub
kevinjqliu commented on code in PR #13129: URL: https://github.com/apache/iceberg/pull/13129#discussion_r2105907785 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -565,13 +568,16 @@ public List listNamespaces(SessionContext context, Namespace names

Re: [PR] Core: Make pageToken query parameter optional [iceberg]

2025-05-24 Thread via GitHub
kevinjqliu commented on code in PR #13129: URL: https://github.com/apache/iceberg/pull/13129#discussion_r2105907785 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -565,13 +568,16 @@ public List listNamespaces(SessionContext context, Namespace names

Re: [PR] Change `FieldSummary` `{upper,lower}_bound` to `ByteBuf` [iceberg-rust]

2025-05-24 Thread via GitHub
Fokko commented on code in PR #1369: URL: https://github.com/apache/iceberg-rust/pull/1369#discussion_r2105904364 ## crates/iceberg/src/spec/manifest_list.rs: ## @@ -577,7 +575,7 @@ pub struct ManifestFile { /// A list of field summaries for each partition field in the spec

Re: [PR] Core: Make pageToken query parameter optional [iceberg]

2025-05-24 Thread via GitHub
elphastori commented on PR #13129: URL: https://github.com/apache/iceberg/pull/13129#issuecomment-2906965351 @nastra Could you please review and help me merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [I] REST Catalog fixture is particular about query params [iceberg]

2025-05-24 Thread via GitHub
elphastori commented on issue #13119: URL: https://github.com/apache/iceberg/issues/13119#issuecomment-2906957307 > Specifically when pageToken is omitted and pageSize is set (http://localhost:8181/v1/namespaces/default/tables?pageSize=1), the server with pagination support should return at

Re: [PR] Core: Make pageToken query parameter optional [iceberg]

2025-05-24 Thread via GitHub
elphastori commented on code in PR #13129: URL: https://github.com/apache/iceberg/pull/13129#discussion_r2105893412 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -565,13 +568,16 @@ public List listNamespaces(SessionContext context, Namespace names

Re: [PR] REST: Avoid table corruption on unhandled RestException [iceberg]

2025-05-24 Thread via GitHub
singhpk234 commented on code in PR #12818: URL: https://github.com/apache/iceberg/pull/12818#discussion_r2105889104 ## core/src/main/java/org/apache/iceberg/rest/ErrorHandlers.java: ## @@ -91,9 +91,22 @@ public void accept(ErrorResponse error) { case 404: thr

Re: [PR] REST: Avoid table corruption on unhandled RestException [iceberg]

2025-05-24 Thread via GitHub
amogh-jahagirdar commented on code in PR #12818: URL: https://github.com/apache/iceberg/pull/12818#discussion_r2105886775 ## core/src/main/java/org/apache/iceberg/rest/ErrorHandlers.java: ## @@ -91,9 +91,22 @@ public void accept(ErrorResponse error) { case 404:

Re: [PR] REST: Avoid table corruption on unhandled RestException [iceberg]

2025-05-24 Thread via GitHub
amogh-jahagirdar commented on code in PR #12818: URL: https://github.com/apache/iceberg/pull/12818#discussion_r2105886775 ## core/src/main/java/org/apache/iceberg/rest/ErrorHandlers.java: ## @@ -91,9 +91,22 @@ public void accept(ErrorResponse error) { case 404:

Re: [PR] REST: Avoid table corruption on unhandled RestException [iceberg]

2025-05-24 Thread via GitHub
singhpk234 commented on code in PR #12818: URL: https://github.com/apache/iceberg/pull/12818#discussion_r2105887401 ## core/src/main/java/org/apache/iceberg/rest/ErrorHandlers.java: ## @@ -91,9 +91,22 @@ public void accept(ErrorResponse error) { case 404: thr

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-24 Thread via GitHub
liamzwbao commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2105884381 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { *

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-24 Thread via GitHub
liamzwbao commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2105884381 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { *

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-24 Thread via GitHub
liamzwbao commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2105884381 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/FlinkSchemaUtil.java: ## @@ -192,7 +222,9 @@ public static Type convert(LogicalType flinkType) { *

[PR] refactor: use nesting enum for DataFile and ManifestFile content [iceberg-cpp]

2025-05-24 Thread via GitHub
wgtmac opened a new pull request, #110: URL: https://github.com/apache/iceberg-cpp/pull/110 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[PR] feat: add avro schema projection [iceberg-cpp]

2025-05-24 Thread via GitHub
wgtmac opened a new pull request, #109: URL: https://github.com/apache/iceberg-cpp/pull/109 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] Build: Run SPARK CI on include path patterns [iceberg]

2025-05-24 Thread via GitHub
manuzhang commented on PR #13033: URL: https://github.com/apache/iceberg/pull/13033#issuecomment-2906894081 I will submit PRs for other modules if this goes well for a while. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

2025-05-24 Thread via GitHub
amitgilad3 commented on PR #12257: URL: https://github.com/apache/iceberg/pull/12257#issuecomment-2906838102 Hey @lliangyu-lin , i already started the work. Hope to finish it soon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-24 Thread via GitHub
liamzwbao commented on PR #13072: URL: https://github.com/apache/iceberg/pull/13072#issuecomment-2906825283 @pvary Yep, my plan is to first remove `TableSchema` from `IcebergSource`. After that, I’ll deprecate the remaining methods in `FlinkSchemaUtil`, followed by deprecations in `FlinkSin

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

2025-05-24 Thread via GitHub
pan3793 commented on PR #13106: URL: https://github.com/apache/iceberg/pull/13106#issuecomment-2906824449 CI is green now, cc @aokolnychyi @huaxiangsun @szehon-ho would you mind taking a look? -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

2025-05-24 Thread via GitHub
pan3793 commented on code in PR #13106: URL: https://github.com/apache/iceberg/pull/13106#discussion_r2105811400 ## spark/v4.0/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestWriteAborts.java: ## @@ -121,7 +121,7 @@ public void testBatchAppend() throws IOE

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

2025-05-24 Thread via GitHub
pan3793 commented on code in PR #13106: URL: https://github.com/apache/iceberg/pull/13106#discussion_r2105812210 ## spark/v4.0/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAncestorsOfProcedure.java: ## @@ -147,14 +147,16 @@ public void testAncestorOfUsi

Re: [PR] Spark 4.0: Migrate Iceberg Stored Procedures to Spark built-in implementations [iceberg]

2025-05-24 Thread via GitHub
pan3793 commented on code in PR #13106: URL: https://github.com/apache/iceberg/pull/13106#discussion_r2105811400 ## spark/v4.0/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestWriteAborts.java: ## @@ -121,7 +121,7 @@ public void testBatchAppend() throws IOE

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-24 Thread via GitHub
tustvold commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2906690237 > Also another concern is that, if we delegate a core abstraction like FileIO to object_store, we may experience unnecessary breaking changes introduces when object_store evolv

[PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]

2025-05-24 Thread via GitHub
coded9 opened a new pull request, #13143: URL: https://github.com/apache/iceberg/pull/13143 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m