[PR] Revert "feat: Introduce snapshot summary properties (#1336)" [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 opened a new pull request, #1390: URL: https://github.com/apache/iceberg-rust/pull/1390 ## Which issue does this PR close? This reverts commit 8bb9a882c7e5f6cfd2a9762136e874421cd7fb59. See https://github.com/apache/iceberg-rust/pull/1336#issuecomment-2915710134

Re: [I] Add a method in TableCommit to apply changes to TableMetadata to create a new TableMetadata [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on issue #1386: URL: https://github.com/apache/iceberg-rust/issues/1386#issuecomment-2915785708 cc @CTTY -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Change `FieldSummary` `{upper,lower}_bound` to `ByteBuf` [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on code in PR #1369: URL: https://github.com/apache/iceberg-rust/pull/1369#discussion_r2111508289 ## crates/iceberg/src/expr/visitors/manifest_evaluator.rs: ## @@ -154,9 +155,19 @@ impl BoundPredicateVisitor for ManifestFilterVisitor<'_> { _predi

Re: [PR] Change `FieldSummary` `{upper,lower}_bound` to `ByteBuf` [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 merged PR #1369: URL: https://github.com/apache/iceberg-rust/pull/1369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

Re: [PR] feat: add retryable property for Error [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on code in PR #1383: URL: https://github.com/apache/iceberg-rust/pull/1383#discussion_r2111516594 ## crates/iceberg/src/error.rs: ## @@ -134,6 +134,8 @@ pub struct Error { source: Option, backtrace: Backtrace, + +retryable: bool, Review C

Re: [PR] Flink: Migrate Flink TableSchema for IcebergSource [iceberg]

2025-05-28 Thread via GitHub
mxm commented on PR #13072: URL: https://github.com/apache/iceberg/pull/13072#issuecomment-2915875085 What are the obstacles for completely getting rid of TableSchema? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] Feature request: support to update snapshot summary [iceberg-rust]

2025-05-28 Thread via GitHub
Xuanwo closed issue #1329: Feature request: support to update snapshot summary URL: https://github.com/apache/iceberg-rust/issues/1329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Hive: Throw exception for when listing a non-existing namespace [iceberg]

2025-05-28 Thread via GitHub
pvary commented on code in PR #13130: URL: https://github.com/apache/iceberg/pull/13130#discussion_r2110933234 ## hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveCatalog.java: ## @@ -1209,11 +1208,4 @@ public void testDatabaseLocationWithSlashInWarehouseDir() {

Re: [I] Running MERGE INTO with more than one WHEN condition fails if the number of columns in the target table is > 321 [iceberg]

2025-05-28 Thread via GitHub
andreaschiappacasse commented on issue #10294: URL: https://github.com/apache/iceberg/issues/10294#issuecomment-2915516882 +1 on the issue since is still a problem in our workloads. @nastra would you think is it possible to re-open it? Even if trino solved the upstream issue, it seems that

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-05-28 Thread via GitHub
pvary commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2111329707 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -107,9 +115,12 @@ public abstract class TestRemoveOrphanFiles

[I] Running MERGE INTO with more than one WHEN condition fails if the number of columns in the target table is > 321 [iceberg]

2025-05-28 Thread via GitHub
andreaschiappacasse opened a new issue, #10294: URL: https://github.com/apache/iceberg/issues/10294 ### Apache Iceberg version None ### Query engine Athena (engine v3) ### Please describe the bug 🐞 Hello everyone, today my team incurred in a very strange bug

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#issuecomment-2915710134 Hi, @Xuanwo @dentiny We need to revert this pr, the new added update doesn't exist in openapi spec: https://github.com/apache/iceberg/blob/cab0decbb0e32bf314039e30807eb033c5066

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#issuecomment-2915721862 Also I don't think we should allow user to udpate snapshot summaries alone, it's dangerous. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-28 Thread via GitHub
dentiny commented on PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#issuecomment-2915720658 > Hi, @Xuanwo @dentiny We need to revert this pr, the new added update doesn't exist in openapi spec: https://github.com/apache/iceberg/blob/cab0decbb0e32bf314039e30807eb033c50665d5/

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-28 Thread via GitHub
dentiny commented on PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#issuecomment-2915729660 > Also I don't think we should allow user to udpate snapshot summaries alone, it's dangerous. HI @liurenjie1024, curious if you have other suggestions to update snapshot summa

Re: [PR] Revert "feat: Introduce snapshot summary properties (#1336)" [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 merged PR #1390: URL: https://github.com/apache/iceberg-rust/pull/1390 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@

[I] Data race in `inclusiveMetricsEval` [iceberg-go]

2025-05-28 Thread via GitHub
psarna opened a new issue, #443: URL: https://github.com/apache/iceberg-go/issues/443 ### Apache Iceberg version None ### Please describe the bug 🐞 When running the `TestMergeManifests` test with `-race` flag, a data report is created (attached to the end of the message)

Re: [PR] Docs: add column descriptions for entries metadata table [iceberg]

2025-05-28 Thread via GitHub
manuzhang commented on code in PR #13104: URL: https://github.com/apache/iceberg/pull/13104#discussion_r2111095523 ## docs/docs/spark-queries.md: ## @@ -301,6 +301,16 @@ SELECT * FROM prod.db.table.entries; | -- | -- | -- | -- | -- | -- | | 2 | 57897183625154 | 0 | 0 | {"cont

[I] Swith metadata pointer to new location for different catalogs(sql, hive, glue, etc) [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 opened a new issue, #1389: URL: https://github.com/apache/iceberg-rust/issues/1389 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[I] Glue catalog: avoid reloading table in `commit_table` [iceberg-python]

2025-05-28 Thread via GitHub
psavalle opened a new issue, #2051: URL: https://github.com/apache/iceberg-python/issues/2051 ### Feature Request / Improvement The Glue catalog implementation [always reloads the table](https://github.com/apache/iceberg-python/blob/main/pyiceberg/catalog/glue.py#L505) in `GlueCatalo

Re: [PR] feat: add asan and ubsan support to cmake [iceberg-cpp]

2025-05-28 Thread via GitHub
wgtmac commented on code in PR #107: URL: https://github.com/apache/iceberg-cpp/pull/107#discussion_r2111250416 ## .github/workflows/sanitizer_test.yml: ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] feat: add asan and ubsan support to cmake [iceberg-cpp]

2025-05-28 Thread via GitHub
raulcd commented on PR #107: URL: https://github.com/apache/iceberg-cpp/pull/107#issuecomment-2915302693 I am slightly confused, the log output from the job, see the artifact uploaded here: https://github.com/apache/iceberg-cpp/actions/runs/15293677546?pr=107 suggest there are several

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

2025-05-28 Thread via GitHub
b-rick commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r2111204802 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/RowDataEvolver.java: ## @@ -0,0 +1,169 @@ +/* + * Licensed to the Apache Software Foundation (

[PR] feat: implement avro file reader [iceberg-cpp]

2025-05-28 Thread via GitHub
wgtmac opened a new pull request, #113: URL: https://github.com/apache/iceberg-cpp/pull/113 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on code in PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#discussion_r2111438158 ## crates/iceberg/src/catalog/mod.rs: ## @@ -495,6 +495,12 @@ pub enum TableUpdate { /// Schema IDs to remove. schema_ids: Vec, }, +

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336#issuecomment-2915765649 > > Also I don't think we should allow user to udpate snapshot summaries alone, it's dangerous. > > HI @liurenjie1024, curious if you have other suggestions to update sn

Re: [PR] Revert "feat: Introduce snapshot summary properties (#1336)" [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on PR #1390: URL: https://github.com/apache/iceberg-rust/pull/1390#issuecomment-2915775716 cc @Xuanwo @sdd @Fokko @kevinjqliu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] [EPIC] Support for appending data to iceberg table. [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on issue #1382: URL: https://github.com/apache/iceberg-rust/issues/1382#issuecomment-2915783715 > Hi [@liurenjie1024](https://github.com/liurenjie1024) , thanks for having this! > > Would it be ok if I take up the commit path work? I can start working on a POC

Re: [I] [EPIC] Support for appending data to iceberg table. [iceberg-rust]

2025-05-28 Thread via GitHub
liurenjie1024 commented on issue #1382: URL: https://github.com/apache/iceberg-rust/issues/1382#issuecomment-2915783358 > I don't get why we need this. Is this used by the catalog implementation, e.g. SqlCatalog::update_table? Could you elaborate more the commit process and where we need th

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-05-28 Thread via GitHub
liziyan-lzy commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r237312 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -303,39 +310,89 @@ private Dataset listedFileDS() {

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-05-28 Thread via GitHub
liziyan-lzy commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r237789 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -344,6 +374,8 @@ public void testWapFilesAreKept() {

Re: [PR] AWS: KeyManagementClient implementation that works with AWS KMS [iceberg]

2025-05-28 Thread via GitHub
jbonofre commented on PR #13136: URL: https://github.com/apache/iceberg/pull/13136#issuecomment-2915277304 I will take a look as well. Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] AWS: KeyManagementClient implementation that works with AWS KMS [iceberg]

2025-05-28 Thread via GitHub
jbonofre commented on PR #13136: URL: https://github.com/apache/iceberg/pull/13136#issuecomment-2915276780 @nastra @Fokko ^^ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-05-28 Thread via GitHub
liziyan-lzy commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r245839 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -107,9 +115,12 @@ public abstract class TestRemoveOrpha

Re: [PR] Flink: Dynamic Iceberg Sink Contribution [iceberg]

2025-05-28 Thread via GitHub
mxm commented on code in PR #12424: URL: https://github.com/apache/iceberg/pull/12424#discussion_r265456 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/RowDataEvolver.java: ## @@ -0,0 +1,169 @@ +/* + * Licensed to the Apache Software Foundation (ASF

Re: [PR] Core: Fix incremental compute of partition stats with COW deletes [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2111663422 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -336,16 +336,29 @@ private static PartitionMap computeStatsDiff( Sets.newHas

Re: [PR] feat: Introduce snapshot summary properties [iceberg-rust]

2025-05-28 Thread via GitHub
Xuanwo merged PR #1336: URL: https://github.com/apache/iceberg-rust/pull/1336 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [I] Incrementally computing partition stats can miss deleted files [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on issue #13155: URL: https://github.com/apache/iceberg/issues/13155#issuecomment-2916012247 I found a clean way. Existing testacase was failing because I included same manifest twice when reused. Now, I added a snapshot id check. So, each snapshot only considers the

Re: [PR] Core: Fix incremental compute of partition stats [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2111647311 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -336,16 +336,29 @@ private static PartitionMap computeStatsDiff( Sets.newHas

Re: [PR] Core: Fix incremental compute of partition stats [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2111640567 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -336,16 +336,22 @@ private static PartitionMap computeStatsDiff( Sets.newHas

Re: [PR] Core: Fix incremental compute of partition stats [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2111642709 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -276,7 +275,8 @@ private static Collection computeAndMergeStatsIncremental( Part

Re: [PR] Core: Fix incremental compute of partition stats with COW deletes [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2111647311 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -336,16 +336,29 @@ private static PartitionMap computeStatsDiff( Sets.newHas

Re: [PR] Flink 1.19, 1.20: Remove the MiniClusterWithClientResource dependency [iceberg]

2025-05-28 Thread via GitHub
JeonDaehong commented on PR #13165: URL: https://github.com/apache/iceberg/pull/13165#issuecomment-2916040410 @nastra Hello, I’ve completed the backport. Please review and merge it when you get a chance! Also, this is a separate topic — After my previous PR was mer

Re: [I] Rest Catalog and Writing data to Minio Raises `OSError: When initiating multiple part upload` [iceberg-python]

2025-05-28 Thread via GitHub
hasanozciftcii commented on issue #974: URL: https://github.com/apache/iceberg-python/issues/974#issuecomment-2916096063 > As a workaround, you can define the S3 configurations (endpoint, secret access key, and access key ID) in the load_rest method. > > catalog = load_rest( >

Re: [PR] Core: Fix incremental compute of partition stats with COW deletes [iceberg]

2025-05-28 Thread via GitHub
pvary commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2111741154 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -336,16 +336,29 @@ private static PartitionMap computeStatsDiff( Sets.newHashSet(

[PR] Fix(catalog): Only remove metadata files if TableOp created a new one [iceberg]

2025-05-28 Thread via GitHub
fivetran-raulpersa opened a new pull request, #13169: URL: https://github.com/apache/iceberg/pull/13169 Fixes #13014 In multiple Table ops (Glue, Dynamo, BigQuery and Nessie) a failing operation will clean up the metadata file. However it should only do so if it was the one to cre

Re: [PR] Flink 1.19, 1.20: Remove the MiniClusterWithClientResource dependency [iceberg]

2025-05-28 Thread via GitHub
pvary merged PR #13165: URL: https://github.com/apache/iceberg/pull/13165 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[PR] feat: set snapshot summary property in append action [iceberg-rust]

2025-05-28 Thread via GitHub
dentiny opened a new pull request, #1391: URL: https://github.com/apache/iceberg-rust/pull/1391 ## Which issue does this PR close? - Closes https://github.com/apache/iceberg-rust/issues/1329 ## What changes are included in this PR? This PR tries to do the same thing as re

[PR] fix(table): fix race condition in scanner [iceberg-go]

2025-05-28 Thread via GitHub
zeroshade opened a new pull request, #445: URL: https://github.com/apache/iceberg-go/pull/445 fixes #443 Eliminates potential race condition in scanning caused by the metrics evaluator used for manifest pruning. Since the stats are only used for a given run of Eval, we don't ne

[PR] fix(catalog/glue): Fix nil map assignment issue in glue catalog [iceberg-go]

2025-05-28 Thread via GitHub
lliangyu-lin opened a new pull request, #446: URL: https://github.com/apache/iceberg-go/pull/446 ### Description When calling NewCatalog in glue catalog without any AWS properties, causing the catalog properties to be nil ``` props: iceberg.Properties(glueOps.awsProperties),

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

2025-05-28 Thread via GitHub
Fokko merged PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] feat: support decompress gzip metadata [iceberg-cpp]

2025-05-28 Thread via GitHub
Fokko commented on PR #108: URL: https://github.com/apache/iceberg-cpp/pull/108#issuecomment-2917339595 LGTM @dongxiao1198 , thanks for the review @wgtmac, @lidavidm and @yingcai-cy -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[I] Append-only option for Spark incremental append read that throws [iceberg]

2025-05-28 Thread via GitHub
smaheshwar-pltr opened a new issue, #13179: URL: https://github.com/apache/iceberg/issues/13179 ### Feature Request / Improvement Spark's [incremental append reading](https://iceberg.apache.org/docs/latest/spark-queries/#incremental-read) is useful for append-only incremental workflo

Re: [PR] Incremental Append Scan [iceberg-python]

2025-05-28 Thread via GitHub
smaheshwar-pltr commented on PR #2031: URL: https://github.com/apache/iceberg-python/pull/2031#issuecomment-2917350349 Put up https://github.com/apache/iceberg/issues/13179 re an append-only option -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [I] Append-only option for Spark incremental append read that throws [iceberg]

2025-05-28 Thread via GitHub
smaheshwar-pltr commented on issue #13179: URL: https://github.com/apache/iceberg/issues/13179#issuecomment-2917349624 Not sure who to ping here so CC-ing folks I see on previous Spark incremental read work 😄: @aokolnychyi @huaxingao @szehon-ho @RussellSpitzer, please let me know your thoug

[I] Build: Fix errorprone warnings [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat opened a new issue, #13178: URL: https://github.com/apache/iceberg/issues/13178 ### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 `./gradlew clean build -x test -x integrationTest --no-build-cache` Defaul

Re: [PR] feat: set snapshot summary property in append action [iceberg-rust]

2025-05-28 Thread via GitHub
dentiny commented on code in PR #1391: URL: https://github.com/apache/iceberg-rust/pull/1391#discussion_r2112436407 ## Cargo.lock: ## @@ -3712,7 +3712,7 @@ dependencies = [ "iceberg-catalog-rest", "iceberg-datafusion", "iceberg_test_utils", - "ordered-float 4.6.0", + "orde

Re: [PR] build(deps): bump the gomod_updates group with 3 updates [iceberg-go]

2025-05-28 Thread via GitHub
zeroshade merged PR #440: URL: https://github.com/apache/iceberg-go/pull/440 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [I] Feature request: expose `DataFile` builder as public [iceberg-rust]

2025-05-28 Thread via GitHub
dentiny closed issue #1392: Feature request: expose `DataFile` builder as public URL: https://github.com/apache/iceberg-rust/issues/1392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Feature request: expose `DataFile` builder as public [iceberg-rust]

2025-05-28 Thread via GitHub
dentiny commented on issue #1392: URL: https://github.com/apache/iceberg-rust/issues/1392#issuecomment-2917202154 Sorry I get fooled by gpt and deepseek, the default visibility is public Ref: https://docs.rs/derive_builder/latest/derive_builder/#setter-visibility -- This is an automated

Re: [I] Build: Fix errorprone warnings [iceberg]

2025-05-28 Thread via GitHub
TomasDarquier commented on issue #13178: URL: https://github.com/apache/iceberg/issues/13178#issuecomment-2917209543 - [x] I can contribute a fix for this bug independently Hi, I’d like to work on this issue — it seems like a great fit for my first contribution. -- This is an automate

Re: [I] Build: Fix errorprone warnings [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on issue #13178: URL: https://github.com/apache/iceberg/issues/13178#issuecomment-2917115167 Good first issue for new contributors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[I] Feature request: expose `DataFile` builder as public [iceberg-rust]

2025-05-28 Thread via GitHub
dentiny opened a new issue, #1392: URL: https://github.com/apache/iceberg-rust/issues/1392 ### Is your feature request related to a problem or challenge? My workflow looks like - Accumulate arrow batches in-memory, flush to local filesystem when it reaches certain threshold - Pe

Re: [PR] Flink 1.19, 1.20: Remove the MiniClusterWithClientResource dependency [iceberg]

2025-05-28 Thread via GitHub
pvary commented on PR #13165: URL: https://github.com/apache/iceberg/pull/13165#issuecomment-2916219167 Merged to main. Thanks @JeonDaehong for the backport and @nastra for the review. >After my previous PR was merged, I was expecting to see my name listed on the [contributors page

Re: [PR] feat: add retryable property for Error [iceberg-rust]

2025-05-28 Thread via GitHub
ZENOTME commented on code in PR #1383: URL: https://github.com/apache/iceberg-rust/pull/1383#discussion_r2111914945 ## crates/iceberg/src/error.rs: ## @@ -134,6 +134,8 @@ pub struct Error { source: Option, backtrace: Backtrace, + +retryable: bool, Review Comment

Re: [PR] Core: Fix incremental compute of partition stats with COW deletes [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on code in PR #13163: URL: https://github.com/apache/iceberg/pull/13163#discussion_r2111918206 ## core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java: ## @@ -336,16 +336,29 @@ private static PartitionMap computeStatsDiff( Sets.newHas

Re: [I] Support commit retrie [iceberg-rust]

2025-05-28 Thread via GitHub
ZENOTME closed issue #964: Support commit retrie URL: https://github.com/apache/iceberg-rust/issues/964 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Flink 1.19, 1.20: Remove the MiniClusterWithClientResource dependency [iceberg]

2025-05-28 Thread via GitHub
JeonDaehong commented on PR #13165: URL: https://github.com/apache/iceberg/pull/13165#issuecomment-2916382580 > Merged to main. Thanks @JeonDaehong for the backport and @nastra for the review. > > > After my previous PR was merged, I was expecting to see my name listed on the [contri

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-05-28 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2112113418 ## open-api/rest-catalog-open-api.yaml: ## @@ -3964,6 +4172,302 @@ components: metadata: $ref: '#/components/schemas/TableMetadata' +EventsR

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-05-28 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2112129175 ## open-api/rest-catalog-open-api.py: ## @@ -1389,6 +1558,101 @@ class CommitTableResponse(BaseModel): metadata: TableMetadata +class EventsResponse(BaseMode

Re: [PR] Flink: port range distribution to v2 iceberg sink [iceberg]

2025-05-28 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2112131384 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -645,72 +711,135 @@ private DataStream distributeDataStream(DataStream in

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-05-28 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2112110143 ## open-api/rest-catalog-open-api.yaml: ## @@ -3405,6 +3488,131 @@ components: allOf: - $ref: '#/components/schemas/ScanTasks' +Actor: + ty

Re: [I] INT96 timestamp is read as OffsetDateTime, not LocalDateTime [iceberg]

2025-05-28 Thread via GitHub
anesterenok commented on issue #12266: URL: https://github.com/apache/iceberg/issues/12266#issuecomment-2916672959 While I quite in favor of deprecating int96 storage, I must say that its only a projection of Iceberg types. If one uses Iceberg `timestamp` type, the value must be considere

Re: [PR] Flink: port range distribution to v2 iceberg sink [iceberg]

2025-05-28 Thread via GitHub
rodmeneses commented on code in PR #12071: URL: https://github.com/apache/iceberg/pull/12071#discussion_r2112131384 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java: ## @@ -645,72 +711,135 @@ private DataStream distributeDataStream(DataStream in

Re: [PR] Spark: Add 'skip_file_list' option to RewriteTablePathProcedure for optional file-list generation [iceberg]

2025-05-28 Thread via GitHub
manuzhang commented on code in PR #12844: URL: https://github.com/apache/iceberg/pull/12844#discussion_r2112148631 ## api/src/main/java/org/apache/iceberg/actions/RewriteTablePath.java: ## @@ -86,6 +86,16 @@ public interface RewriteTablePath extends Action

Re: [PR] Docs, Flink: Fix equality fields requirement in upsert mode [iceberg]

2025-05-28 Thread via GitHub
pvary merged PR #13127: URL: https://github.com/apache/iceberg/pull/13127 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Docs, Flink: Fix equality fields requirement in upsert mode [iceberg]

2025-05-28 Thread via GitHub
pvary commented on PR #13127: URL: https://github.com/apache/iceberg/pull/13127#issuecomment-2916483984 Merged to main. Thanks @manuzhang for the PR and @mxm for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] feat: add asan and ubsan support to cmake [iceberg-cpp]

2025-05-28 Thread via GitHub
raulcd commented on code in PR #107: URL: https://github.com/apache/iceberg-cpp/pull/107#discussion_r2112029921 ## .github/workflows/sanitizer_test.yml: ## @@ -0,0 +1,63 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

Re: [PR] Proposal: IRC Events endpoint [iceberg]

2025-05-28 Thread via GitHub
c-thiel commented on code in PR #12584: URL: https://github.com/apache/iceberg/pull/12584#discussion_r2112105575 ## open-api/rest-catalog-open-api.yaml: ## @@ -3041,6 +3096,34 @@ components: - $ref: '#/components/schemas/AddViewVersionUpdate' - $ref: '#/compone

Re: [I] Spark, Avro and Iceberg Timestamp, Timestamp NTZ definition need more clarifications [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on issue #12751: URL: https://github.com/apache/iceberg/issues/12751#issuecomment-2916840926 Is this an issue for Datafiles written with Avro Data instead of Parquet? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Infra: Set 1.9.1 to Latest Release [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer merged PR #13177: URL: https://github.com/apache/iceberg/pull/13177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core: Fix a cast that is too narrow [iceberg]

2025-05-28 Thread via GitHub
angelo-DNAStack commented on PR #12743: URL: https://github.com/apache/iceberg/pull/12743#issuecomment-2917511977 Checking in on this PR again, please let me know if there are any additional changes required. -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Docs: Adds 1.9.1 Versioned JavaDocs [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on PR #13173: URL: https://github.com/apache/iceberg/pull/13173#issuecomment-2917518749 Thanks @ajantha-bhat and @danielcweeks for reviewing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Docs: Adds 1.9.1 Versioned JavaDocs [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer merged PR #13173: URL: https://github.com/apache/iceberg/pull/13173 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Infra: Set 1.9.1 to Latest Release [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on PR #13177: URL: https://github.com/apache/iceberg/pull/13177#issuecomment-2917517509 Thanks @ajantha-bhat , @flyrain + @singhpk234 for reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Site: Updates for 1.9.1 Release [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer merged PR #13176: URL: https://github.com/apache/iceberg/pull/13176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Site: Updates for 1.9.1 Release [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on PR #13176: URL: https://github.com/apache/iceberg/pull/13176#issuecomment-2917516161 Thanks @ajantha-bhat + @danielcweeks for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Site: Add Versioned Docs for 1.9.1 [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer merged PR #13172: URL: https://github.com/apache/iceberg/pull/13172 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Site: Add Versioned Docs for 1.9.1 [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on PR #13172: URL: https://github.com/apache/iceberg/pull/13172#issuecomment-2917520514 Thanks @ajantha-bhat + @danielcweeks for reviewing! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2112716340 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -107,9 +115,12 @@ public abstract class TestRemoveOr

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2112725064 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -344,6 +374,8 @@ public void testWapFilesAreKept() {

Re: [PR] SPARK: Remove dependency on hadoop's filesystem class from remove orphan files [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer commented on code in PR #12254: URL: https://github.com/apache/iceberg/pull/12254#discussion_r2112725064 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestRemoveOrphanFilesAction.java: ## @@ -344,6 +374,8 @@ public void testWapFilesAreKept() {

Re: [PR] fix(Catalog): Handle NotFound exception for missing metadata file [iceberg]

2025-05-28 Thread via GitHub
nastra commented on code in PR #13143: URL: https://github.com/apache/iceberg/pull/13143#discussion_r2112280095 ## core/src/test/java/org/apache/iceberg/catalog/CatalogTests.java: ## @@ -959,6 +959,16 @@ public void testLoadMissingTable() { .hasMessageStartingWith("Tabl

[PR] Docs: Adds 1.9.1 Versioned JavaDocs [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer opened a new pull request, #13173: URL: https://github.com/apache/iceberg/pull/13173 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Spark 3.4: streaming-skip-overwrite-snapshots fix [iceberg]

2025-05-28 Thread via GitHub
huaxingao merged PR #13168: URL: https://github.com/apache/iceberg/pull/13168 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Spark 3.4: streaming-skip-overwrite-snapshots fix [iceberg]

2025-05-28 Thread via GitHub
huaxingao commented on PR #13168: URL: https://github.com/apache/iceberg/pull/13168#issuecomment-2916961579 Merged. Thanks @wypoon for the PR! Thanks @singhpk234 for the review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] Site: Add Versioned Docs for 1.9.1 [iceberg]

2025-05-28 Thread via GitHub
RussellSpitzer opened a new pull request, #13172: URL: https://github.com/apache/iceberg/pull/13172 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[I] ERROR: Procedure system.refresh_table not found [iceberg]

2025-05-28 Thread via GitHub
vishwanthp12 opened a new issue, #13171: URL: https://github.com/apache/iceberg/issues/13171 Command I'm using in Spark-sql: CALL iceberg_catalog.system.refresh_table('iceberg_catalog.$db.$table'); I'm using JDBC catalog configured on MariaDB (MYSQL) here is my spark-default.co

Re: [PR] Core: Add basic classes for writing table format-version 4 [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat commented on code in PR #13123: URL: https://github.com/apache/iceberg/pull/13123#discussion_r2112311313 ## api/src/test/java/org/apache/iceberg/TestHelpers.java: ## @@ -54,7 +54,7 @@ public class TestHelpers { private TestHelpers() {} - public static final

[PR] Spec: Mark version 3 as completed [iceberg]

2025-05-28 Thread via GitHub
ajantha-bhat opened a new pull request, #13175: URL: https://github.com/apache/iceberg/pull/13175 voted on mailing list: https://lists.apache.org/thread/9oncq0j0222nrm6scvm89xdq3gozg24p -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

  1   2   >