Re: [I] doc: Add how to verify a release section. [iceberg-rust]

2025-06-20 Thread via GitHub
kevinjqliu closed issue #1413: doc: Add how to verify a release section. URL: https://github.com/apache/iceberg-rust/issues/1413 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Update link to how to verify release. [iceberg-rust]

2025-06-20 Thread via GitHub
Xuanwo merged PR #1461: URL: https://github.com/apache/iceberg-rust/pull/1461 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

[PR] Update link to how to verify release. [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 opened a new pull request, #1461: URL: https://github.com/apache/iceberg-rust/pull/1461 ## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Update link to how to verify release. ## Are these changes tested?

Re: [PR] Add how to verify release doc [iceberg-rust]

2025-06-20 Thread via GitHub
kevinjqliu merged PR #1426: URL: https://github.com/apache/iceberg-rust/pull/1426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ice

Re: [PR] Add how to verify release doc [iceberg-rust]

2025-06-20 Thread via GitHub
kevinjqliu commented on PR #1426: URL: https://github.com/apache/iceberg-rust/pull/1426#issuecomment-2993316814 Thanks for adding this @liurenjie1024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Add docs for Table Maintenance in Flink [iceberg]

2025-06-20 Thread via GitHub
JeonDaehong commented on PR #13360: URL: https://github.com/apache/iceberg/pull/13360#issuecomment-2993314105 > > By the way, I noticed that commit history from a previous unrelated issue is still showing up in this PR. > > In this case, should I delete my forked repository and re-fork it

Re: [PR] feat: introduce Scalar and StructLike interfaces for Transform evaluation [iceberg-cpp]

2025-06-20 Thread via GitHub
gty404 closed pull request #77: feat: introduce Scalar and StructLike interfaces for Transform evaluation URL: https://github.com/apache/iceberg-cpp/pull/77 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Add docs for Table Maintenance in Flink [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on PR #13360: URL: https://github.com/apache/iceberg/pull/13360#issuecomment-2993271794 > By the way, I noticed that commit history from a previous unrelated issue is still showing up in this PR. > In this case, should I delete my forked repository and re-fork it f

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159824968 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SkipOnError.java: ## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159822838 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/AntiJoin.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Add how to verify release doc [iceberg-rust]

2025-06-20 Thread via GitHub
kevinjqliu commented on code in PR #1426: URL: https://github.com/apache/iceberg-rust/pull/1426#discussion_r2159822240 ## website/src/release.md: ## @@ -202,6 +202,9 @@ please cancel the release for the current `release_version`, _increase th RC counting_ and re-initiate a rele

Re: [PR] Add how to verify release doc [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on PR #1426: URL: https://github.com/apache/iceberg-rust/pull/1426#issuecomment-2993253570 cc @kevinjqliu Comments addressed, could you help to take a look again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Add how to verify release doc [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on code in PR #1426: URL: https://github.com/apache/iceberg-rust/pull/1426#discussion_r2159821562 ## Cargo.lock: ## Review Comment: No, auto updated by intellij. -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] Add how to verify release doc [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on code in PR #1426: URL: https://github.com/apache/iceberg-rust/pull/1426#discussion_r2159821517 ## website/src/release.md: ## @@ -202,6 +202,35 @@ please cancel the release for the current `release_version`, _increase th RC counting_ and re-initiate a

Re: [PR] Add how to verify release doc [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on code in PR #1426: URL: https://github.com/apache/iceberg-rust/pull/1426#discussion_r2159821423 ## website/src/release.md: ## @@ -202,6 +202,35 @@ please cancel the release for the current `release_version`, _increase th RC counting_ and re-initiate a

Re: [PR] doc: Add a readme for the SQL catalog [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on code in PR #1459: URL: https://github.com/apache/iceberg-rust/pull/1459#discussion_r2159819957 ## crates/catalog/sql/README.md: ## @@ -0,0 +1,39 @@ + + +# Apache Iceberg SQL Catalog Official Native Rust Implementation + +[![crates.io](https://img.shiel

[I] `iceberg-catalog-sql` is not published. [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 opened a new issue, #1460: URL: https://github.com/apache/iceberg-rust/issues/1460 ### Apache Iceberg Rust version 0.5.1 (latest version) ### Describe the bug iceberg-catalog-sql is not published see https://github.com/apache/iceberg-rust/blob/79711acc921b4

Re: [PR] feat(transaction): Remove current_table, updates, and requirements from Transaction [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on code in PR #1451: URL: https://github.com/apache/iceberg-rust/pull/1451#discussion_r2159817079 ## crates/iceberg/src/transaction/mod.rs: ## @@ -157,41 +114,45 @@ impl Transaction { /// Commit transaction. pub async fn commit(mut self, catal

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159817224 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/DeleteOrphanFiles.java: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software

Re: [PR] Add docs for Table Maintenance in Flink [iceberg]

2025-06-20 Thread via GitHub
JeonDaehong commented on code in PR #13360: URL: https://github.com/apache/iceberg/pull/13360#discussion_r2159816921 ## docs/docs/flink-table-maintenance.md: ## @@ -0,0 +1,268 @@ +--- +title: "Flink Table Maintenance " +--- + + +# Flink Table Maintenance + +## Overview +When us

Re: [PR] Add docs for Table Maintenance in Flink [iceberg]

2025-06-20 Thread via GitHub
JeonDaehong commented on PR #13360: URL: https://github.com/apache/iceberg/pull/13360#issuecomment-2993239258 @Guosmilesmile @pvary Thank you both so much for the feedback :D I've carefully reviewed and read through all of your comments. Based on your suggestions, I’ll make

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159815679 ## core/src/main/java/org/apache/iceberg/actions/FileURI.java: ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [I] Support intelligent commit retries [iceberg-python]

2025-06-20 Thread via GitHub
potatochipcoconut commented on issue #269: URL: https://github.com/apache/iceberg-python/issues/269#issuecomment-2993235897 @maxlucuta would you be able to share what that code looks like? Really interested to try that strategy out. -- This is an automated message from the Apache Git Serv

Re: [I] Python package size has ballooned dramatically [iceberg-rust]

2025-06-20 Thread via GitHub
kevinjqliu commented on issue #1458: URL: https://github.com/apache/iceberg-rust/issues/1458#issuecomment-2993228270 hey @gorilla-seb i believe this is the result of #1324 when we added the datafusion table provider to the python binding. > In increase by over 4000% between minor ver

Re: [PR] Add docs for Table Maintenance in Flink [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on code in PR #13360: URL: https://github.com/apache/iceberg/pull/13360#discussion_r2159805226 ## docs/docs/flink-table-maintenance.md: ## @@ -0,0 +1,268 @@ +--- +title: "Flink Table Maintenance " +--- + + +# Flink Table Maintenance + +## Overview +When

Re: [PR] Flink: Revise the display of the task name in TableMaintenance to show the specific task name. [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on PR #13024: URL: https://github.com/apache/iceberg/pull/13024#issuecomment-2993211150 I think we still need some time to discuss the next details. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Flink: Revise the display of the task name in TableMaintenance to show the specific task name. [iceberg]

2025-06-20 Thread via GitHub
Guosmilesmile commented on code in PR #13024: URL: https://github.com/apache/iceberg/pull/13024#discussion_r2159798371 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/ExpireSnapshots.java: ## @@ -47,6 +47,11 @@ public static class Builder extends Main

Re: [PR] Flink: Migrate Flink `TableSchema` to `Schema`/`ResolvedSchema` [iceberg]

2025-06-20 Thread via GitHub
liamzwbao commented on code in PR #13072: URL: https://github.com/apache/iceberg/pull/13072#discussion_r2159777330 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkV2DistributionMode.java: ## @@ -122,7 +122,7 @@ public void before() throws IOEx

Re: [PR] Add test for handling empty result set in loadTableMetadata [iceberg]

2025-06-20 Thread via GitHub
github-actions[bot] closed pull request #13055: Add test for handling empty result set in loadTableMetadata URL: https://github.com/apache/iceberg/pull/13055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Core: Remove deprecated MetricsConfig.fromProperties method [iceberg]

2025-06-20 Thread via GitHub
github-actions[bot] commented on PR #13056: URL: https://github.com/apache/iceberg/pull/13056#issuecomment-2993184040 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Add test for handling empty result set in loadTableMetadata [iceberg]

2025-06-20 Thread via GitHub
github-actions[bot] commented on PR #13055: URL: https://github.com/apache/iceberg/pull/13055#issuecomment-2993184024 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If

Re: [PR] Flink: Revise the display of the task name in TableMaintenance to show the specific task name. [iceberg]

2025-06-20 Thread via GitHub
github-actions[bot] commented on PR #13024: URL: https://github.com/apache/iceberg/pull/13024#issuecomment-2993184003 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] Decouple Committer from Kafka and Enable Custom Coordinator Election [iceberg]

2025-06-20 Thread via GitHub
github-actions[bot] commented on PR #12460: URL: https://github.com/apache/iceberg/pull/12460#issuecomment-2993183960 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [I] [Docs] Update Examples to Replace Hadoop Catalog with JDBC Catalog [iceberg]

2025-06-20 Thread via GitHub
github-actions[bot] commented on issue #11284: URL: https://github.com/apache/iceberg/issues/11284#issuecomment-2993183892 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

[PR] doc: Add a readme for the SQL catalog [iceberg-rust]

2025-06-20 Thread via GitHub
kyteware opened a new pull request, #1459: URL: https://github.com/apache/iceberg-rust/pull/1459 ## Which issue does this PR close? - ~Closes~ Amends problems from #1456 ## What changes are included in this PR? A `README.md` for the SQL catalog. Includes i

Re: [I] Sqlite sql catalogs work in unit tests, but not in prod because of config mistake [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 closed issue #1456: Sqlite sql catalogs work in unit tests, but not in prod because of config mistake URL: https://github.com/apache/iceberg-rust/issues/1456 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [I] Sqlite sql catalogs work in unit tests, but not in prod because of config mistake [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on issue #1456: URL: https://github.com/apache/iceberg-rust/issues/1456#issuecomment-2993167323 Close this issue as it's not a bug, see discussions in this pr: https://github.com/apache/iceberg-rust/pull/1457 -- This is an automated message from the Apache Git Serv

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-06-20 Thread via GitHub
ebyhr commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r2159730783 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return (Li

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-06-20 Thread via GitHub
ebyhr commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r2159730783 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return (Li

Re: [PR] fix: include sqlite-required features in sqlx dep [iceberg-rust]

2025-06-20 Thread via GitHub
kyteware commented on PR #1457: URL: https://github.com/apache/iceberg-rust/pull/1457#issuecomment-2993069781 @liurenjie1024 Wow, you're totally right! I didn't realize you could do that with cargo features. I'll make another PR to document this instead. Thanks for the insight. -- This i

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-06-20 Thread via GitHub
ebyhr commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r2159730783 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return (Li

Re: [PR] feat(transaction): Remove current_table, updates, and requirements from Transaction [iceberg-rust]

2025-06-20 Thread via GitHub
CTTY commented on code in PR #1451: URL: https://github.com/apache/iceberg-rust/pull/1451#discussion_r2159720577 ## crates/iceberg/src/transaction/mod.rs: ## @@ -45,60 +45,55 @@ use crate::{Catalog, TableCommit, TableRequirement, TableUpdate}; /// Table transaction. pub str

[PR] fix(catalog/rest): pass key-scope [iceberg-go]

2025-06-20 Thread via GitHub
laskoviymishka opened a new pull request, #464: URL: https://github.com/apache/iceberg-go/pull/464 closes: #451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [I] Duplicate File Remediation [iceberg-python]

2025-06-20 Thread via GitHub
ForeverAngry commented on issue #2130: URL: https://github.com/apache/iceberg-python/issues/2130#issuecomment-2993011282 Thanks for raising this @MrDerecho , this is something my team members have to deal with frequently, due to how we approach the use of `add_files`. Nice to know that it

Re: [PR] Added ExpireSnapshots Feature [iceberg-python]

2025-06-20 Thread via GitHub
Fokko commented on PR #1880: URL: https://github.com/apache/iceberg-python/pull/1880#issuecomment-2993003792 @ForeverAngry Appreciate that, thanks! 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[I] Add files support for parquet field_ids [iceberg-python]

2025-06-20 Thread via GitHub
MrDerecho opened a new issue, #2131: URL: https://github.com/apache/iceberg-python/issues/2131 ### Feature Request / Improvement Would it be possible to allow for parquet field_id support in the add_files method? Parquet field id's are a requirement for backwards compatibility with

[I] Duplicate File Remediation [iceberg-python]

2025-06-20 Thread via GitHub
MrDerecho opened a new issue, #2130: URL: https://github.com/apache/iceberg-python/issues/2130 ### Feature Request / Improvement I use pyiceberg add_files to perform enterprise-grade ETL loading and backfilling of iceberg tables- on occasion, there will be a duplicate file, I load so

Re: [PR] feat(transaction): Remove current_table, updates, and requirements from Transaction [iceberg-rust]

2025-06-20 Thread via GitHub
CTTY commented on code in PR #1451: URL: https://github.com/apache/iceberg-rust/pull/1451#discussion_r2159688238 ## crates/iceberg/src/transaction/mod.rs: ## @@ -45,60 +45,55 @@ use crate::{Catalog, TableCommit, TableRequirement, TableUpdate}; /// Table transaction. pub str

Re: [PR] feat(transaction): Remove current_table, updates, and requirements from Transaction [iceberg-rust]

2025-06-20 Thread via GitHub
CTTY commented on code in PR #1451: URL: https://github.com/apache/iceberg-rust/pull/1451#discussion_r2159684918 ## crates/iceberg/src/transaction/mod.rs: ## @@ -45,60 +45,55 @@ use crate::{Catalog, TableCommit, TableRequirement, TableUpdate}; /// Table transaction. pub str

[PR] Row lineage fields for v3 [iceberg-python]

2025-06-20 Thread via GitHub
rambleraptor opened a new pull request, #2129: URL: https://github.com/apache/iceberg-python/pull/2129 Closes #1821 # Rationale for this change This adds the proper row-lineage fields for v3. Row-lineage is enforced for v3, so all of this is done by default. # Ar

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
Fokko commented on PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125#issuecomment-2992914970 Thanks @kevinjqliu 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] CI: run FileIO integration tests (s3/adls/gcs) [iceberg-python]

2025-06-20 Thread via GitHub
Fokko closed issue #2124: CI: run FileIO integration tests (s3/adls/gcs) URL: https://github.com/apache/iceberg-python/issues/2124 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
Fokko merged PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] feat: adding schema update methods [iceberg-go]

2025-06-20 Thread via GitHub
zeroshade commented on code in PR #431: URL: https://github.com/apache/iceberg-go/pull/431#discussion_r2159656428 ## table/update_schema.go: ## @@ -0,0 +1,659 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the N

Re: [PR] Expose S3 retry strategy [iceberg-python]

2025-06-20 Thread via GitHub
Fokko merged PR #2110: URL: https://github.com/apache/iceberg-python/pull/2110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] feature: expire snapshots action [iceberg-rust]

2025-06-20 Thread via GitHub
cmcarthur commented on PR #1455: URL: https://github.com/apache/iceberg-rust/pull/1455#issuecomment-2992710247 In testing this on some larger tables, it's absolutely clear that some parallelism will be required in the initial load step in `collect_files_to_delete` -- calling `load_manifest_

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
kevinjqliu commented on PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125#issuecomment-2992612936 Made coverage fail percentage customizable. - 85% for unit tests, currently at 87% - 75% for integration tests, currently at 77% Also added this comment :) ```

Re: [I] Confusion around 'catalog' in the spark quickstart guide [iceberg]

2025-06-20 Thread via GitHub
andres-lowrie commented on issue #13312: URL: https://github.com/apache/iceberg/issues/13312#issuecomment-2992606329 here's the section in the doc website https://iceberg.apache.org/spark-quickstart/#adding-a-catalog the section is: > Iceberg has several catalog back-ends t

Re: [PR] feat: implement Primitive type Literal [iceberg-cpp]

2025-06-20 Thread via GitHub
mapleFU commented on code in PR #117: URL: https://github.com/apache/iceberg-cpp/pull/117#discussion_r2159167431 ## src/iceberg/expression/literal.cc: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreemen

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159361569 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/maintenance/api/TestDeleteOrphanFiles.java: ## @@ -0,0 +1,417 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Add docs for Table Maintenance in Flink [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13360: URL: https://github.com/apache/iceberg/pull/13360#discussion_r2159245823 ## docs/docs/flink-table-maintenance.md: ## @@ -0,0 +1,268 @@ +--- +title: "Flink Table Maintenance " +--- + + +# Flink Table Maintenance + +## Overview +When using **

Re: [PR] fix: correct type mismatch in avro zstd decompression. [iceberg-python]

2025-06-20 Thread via GitHub
Fokko merged PR #2128: URL: https://github.com/apache/iceberg-python/pull/2128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Core: Fix numeric overflow of timestamp nano literal [iceberg]

2025-06-20 Thread via GitHub
stevenzwu commented on code in PR #11775: URL: https://github.com/apache/iceberg/pull/11775#discussion_r2159503874 ## api/src/main/java/org/apache/iceberg/expressions/Literals.java: ## @@ -300,8 +300,7 @@ public Literal to(Type type) { case TIMESTAMP: return

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159288325 ## core/src/main/java/org/apache/iceberg/actions/FileURI.java: ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] fix(table): handle missing or nil stats + metadata field nil comparison [iceberg-go]

2025-06-20 Thread via GitHub
zeroshade commented on code in PR #460: URL: https://github.com/apache/iceberg-go/pull/460#discussion_r2159164308 ## table/metadata.go: ## @@ -877,8 +881,8 @@ func (c *commonMetadata) Equals(other *commonMetadata) bool { } return c.FormatVersion == other.Forma

Re: [PR] Improve upsert memory pressure [iceberg-python]

2025-06-20 Thread via GitHub
Fokko commented on PR #1995: URL: https://github.com/apache/iceberg-python/pull/1995#issuecomment-2992495411 Moving this forward, this looks great @koenvo 🙌 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Improve upsert memory pressure [iceberg-python]

2025-06-20 Thread via GitHub
Fokko merged PR #1995: URL: https://github.com/apache/iceberg-python/pull/1995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
Fokko commented on PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125#issuecomment-2992487465 @kevinjqliu This is the tricky part of the coverage. Looks like we have sufficient coverage for the unit tests, but not for the integration tests :( So maybe only report coverage for

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
Fokko commented on code in PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125#discussion_r2159496745 ## Makefile: ## @@ -14,106 +14,142 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # un

Re: [PR] Docs: Fix description of min-input-files option of Spark rewrite_data_files procedure [iceberg]

2025-06-20 Thread via GitHub
pvary commented on PR #13355: URL: https://github.com/apache/iceberg/pull/13355#issuecomment-2992099097 Merged to main. Thanks for the doc fix @manuzhang! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159307706 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/DeleteOrphanFiles.java: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159302508 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/DeleteOrphanFiles.java: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] feature: expire snapshots action [iceberg-rust]

2025-06-20 Thread via GitHub
cmcarthur commented on PR #1455: URL: https://github.com/apache/iceberg-rust/pull/1455#issuecomment-2992463463 ok, thanks again @CTTY for the review. I think this is ready for another look. changes: - Added tests to cover more realistic scenarios where new snapshots reference new and

Re: [PR] feat: `rustls-tls` and `rustls-native-roots` feature flags [iceberg-rust]

2025-06-20 Thread via GitHub
Xuanwo commented on PR #1361: URL: https://github.com/apache/iceberg-rust/pull/1361#issuecomment-2990298762 Hi, I'm going to close this PR now, feel free to reopen it if you have other ideas. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159282127 ## core/src/main/java/org/apache/iceberg/util/FileSystemWalker.java: ## @@ -0,0 +1,266 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] feature: expire snapshots action [iceberg-rust]

2025-06-20 Thread via GitHub
cmcarthur commented on code in PR #1455: URL: https://github.com/apache/iceberg-rust/pull/1455#discussion_r2159428085 ## crates/iceberg/src/maintenance/expire_snapshots.rs: ## @@ -0,0 +1,1144 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribut

Re: [PR] feat(transaction): Remove current_table, updates, and requirements from Transaction [iceberg-rust]

2025-06-20 Thread via GitHub
liurenjie1024 commented on code in PR #1451: URL: https://github.com/apache/iceberg-rust/pull/1451#discussion_r2158782394 ## crates/iceberg/src/transaction/mod.rs: ## @@ -45,60 +45,55 @@ use crate::{Catalog, TableCommit, TableRequirement, TableUpdate}; /// Table transaction.

Re: [PR] Optimise RowData evolution [iceberg]

2025-06-20 Thread via GitHub
aiborodin commented on code in PR #13340: URL: https://github.com/apache/iceberg/pull/13340#discussion_r2158284022 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordProcessor.java: ## @@ -31,10 +33,14 @@ import org.apache.iceberg.Schema; impo

Re: [PR] feat: implement PrimitiveLiteral [iceberg-cpp]

2025-06-20 Thread via GitHub
mapleFU commented on code in PR #117: URL: https://github.com/apache/iceberg-cpp/pull/117#discussion_r2158337676 ## src/iceberg/literal.h: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the

Re: [PR] Support ADLS with Pyarrow file IO [iceberg-python]

2025-06-20 Thread via GitHub
NikitaMatskevich commented on code in PR #2111: URL: https://github.com/apache/iceberg-python/pull/2111#discussion_r2158556942 ## pyiceberg/io/__init__.py: ## @@ -82,6 +82,10 @@ ADLS_CLIENT_ID = "adls.client-id" ADLS_CLIENT_SECRET = "adls.client-secret" ADLS_ACCOUNT_HOST = "a

Re: [I] Spark: Doing a Coalesce and foreachpartitions in spark directly on an iceberg table is leaking memory heavy iterators [iceberg]

2025-06-20 Thread via GitHub
jkolash commented on issue #13297: URL: https://github.com/apache/iceberg/issues/13297#issuecomment-2992083449 I filed https://issues.apache.org/jira/projects/SPARK/issues/SPARK-52516 and have a WIP branch that I need to test against the latest spark master https://github.com/apache/s

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159364395 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java: ## @@ -123,6 +118,7 @@ public class DeleteOrphanFilesSparkAction e

[I] Python package size has ballooned dramatically [iceberg-rust]

2025-06-20 Thread via GitHub
gorilla-seb opened a new issue, #1458: URL: https://github.com/apache/iceberg-rust/issues/1458 From 0.4.0 to 0.5.0 the size of `pyiceberg-core` Python packages has increased dramatically. [Python wheels for 0.4.0](https://pypi.org/project/pyiceberg-core/0.4.0/#files): between ~ 0.7 a

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159360815 ## flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/maintenance/api/MaintenanceTaskTestBase.java: ## @@ -41,7 +41,7 @@ void runAndWaitForSuccess( ManualSo

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159357935 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TableReader.java: ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] Optimise RowData evolution [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13340: URL: https://github.com/apache/iceberg/pull/13340#discussion_r2159220763 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableMetadataCache.java: ## @@ -220,37 +238,59 @@ SchemaInfo getSchemaInfo() { */ static

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159339418 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/FileUriConverter.java: ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159345579 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/SkipOnError.java: ## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159346783 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/TablePlanner.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
kevinjqliu commented on PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125#issuecomment-2992190858 ``` TOTAL 14886 189387% Coverage failure: total of 87 is less than fail-under=90 make: *** [Makefile:123: coverage-report] Erro

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159333119 ## core/src/main/java/org/apache/iceberg/actions/FileURI.java: ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contr

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159331046 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/AntiJoin.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159327236 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/AntiJoin.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] fix(table): handle missing or nil stats + metadata field nil comparison [iceberg-go]

2025-06-20 Thread via GitHub
zeroshade commented on PR #460: URL: https://github.com/apache/iceberg-go/pull/460#issuecomment-2992116664 Thanks for this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
kevinjqliu commented on code in PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125#discussion_r2159322794 ## .github/workflows/python-ci.yml: ## @@ -84,3 +84,21 @@ jobs: - name: Show debug logs if: ${{ failure() }} run: docker compose -f dev/doc

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159320739 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/AntiJoin.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159311893 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/DeleteOrphanFiles.java: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159313174 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/DeleteOrphanFiles.java: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Refactor `Makefile` and run s3/adls/gcs integration tests in CI [iceberg-python]

2025-06-20 Thread via GitHub
kevinjqliu commented on code in PR #2125: URL: https://github.com/apache/iceberg-python/pull/2125#discussion_r2159208827 ## .github/workflows/python-ci.yml: ## @@ -84,3 +84,21 @@ jobs: - name: Show debug logs if: ${{ failure() }} run: docker compose -f dev/doc

Re: [PR] Flink: Supports delete orphan files in TableMaintenance [iceberg]

2025-06-20 Thread via GitHub
pvary commented on code in PR #13302: URL: https://github.com/apache/iceberg/pull/13302#discussion_r2159303639 ## flink/v2.0/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/DeleteOrphanFiles.java: ## @@ -0,0 +1,358 @@ +/* + * Licensed to the Apache Software Foundati

  1   2   >