Re: [PR] Added test about CDC creation failure after expiring all the snapshots [iceberg]

flyrain commented on PR #9141: URL: https://github.com/apache/iceberg/pull/9141#issuecomment-1826229855 cc @aokolnychyi @RussellSpitzer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Added test about CDC creation failure after expiring all the snapshots [iceberg]

flyrain commented on PR #9141: URL: https://github.com/apache/iceberg/pull/9141#issuecomment-1826229811 Thanks a lot for the PR, @andreacfm! Will take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [I] BaseLockManager.scheduler doesn't shutdown when closing [iceberg]

gabrywu commented on issue #9142: URL: https://github.com/apache/iceberg/issues/9142#issuecomment-1826212384 @nastra please help to review this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] shutdown scheduler [iceberg]

gabrywu opened a new pull request, #9150: URL: https://github.com/apache/iceberg/pull/9150 closing #9142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[I] Consider publishing a BOM (bill of materials) [iceberg]

electrum opened a new issue, #7371: URL: https://github.com/apache/iceberg/issues/7371 ### Feature Request / Improvement This would make consuming the Iceberg dependencies easier, as one could simply import the BOM, rather than needing to ensure that all of the Iceberg dependencies h

Re: [I] feat: Add support for `UnboundSortOrder` [iceberg-rust]

liurenjie1024 commented on issue #99: URL: https://github.com/apache/iceberg-rust/issues/99#issuecomment-1826204833 > I will give this a try, started looking into it. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Fix compile failures [iceberg-rust]

liurenjie1024 commented on PR #105: URL: https://github.com/apache/iceberg-rust/pull/105#issuecomment-1826204512 It seems that ci settings ci didn't enforce rebase before merging. Not sure if this the limitation of apache infra. -- This is an automated message from the Apache Git Service.

[PR] Fix compile failures [iceberg-rust]

fqaiser94 opened a new pull request, #105: URL: https://github.com/apache/iceberg-rust/pull/105 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] feat: Add support for `UnboundSortOrder` [iceberg-rust]

fqaiser94 commented on issue #99: URL: https://github.com/apache/iceberg-rust/issues/99#issuecomment-1826201306 I will give this a try, started looking into it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Consider publishing a BOM (bill of materials) [iceberg]

github-actions[bot] closed issue #7371: Consider publishing a BOM (bill of materials) URL: https://github.com/apache/iceberg/issues/7371 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Consider publishing a BOM (bill of materials) [iceberg]

github-actions[bot] commented on issue #7371: URL: https://github.com/apache/iceberg/issues/7371#issuecomment-1826152454 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] The serialization problem caused by Flink shuffling design [iceberg]

github-actions[bot] closed issue #7393: The serialization problem caused by Flink shuffling design URL: https://github.com/apache/iceberg/issues/7393 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] The serialization problem caused by Flink shuffling design [iceberg]

github-actions[bot] commented on issue #7393: URL: https://github.com/apache/iceberg/issues/7393#issuecomment-1826152438 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Some question about zorder [iceberg]

github-actions[bot] closed issue #7405: Some question about zorder URL: https://github.com/apache/iceberg/issues/7405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [I] Some question about zorder [iceberg]

github-actions[bot] commented on issue #7405: URL: https://github.com/apache/iceberg/issues/7405#issuecomment-1826152417 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

[PR] Build: Bump moto from 4.2.9 to 4.2.10 [iceberg-python]

dependabot[bot] opened a new pull request, #164: URL: https://github.com/apache/iceberg-python/pull/164 Bumps [moto](https://github.com/getmoto/moto) from 4.2.9 to 4.2.10. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 4.2

[PR] Build: Bump mkdocs-material from 9.4.10 to 9.4.12 [iceberg-python]

dependabot[bot] opened a new pull request, #163: URL: https://github.com/apache/iceberg-python/pull/163 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.4.10 to 9.4.12. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mk

Re: [PR] Build: Bump mkdocs-material from 9.4.10 to 9.4.11 [iceberg-python]

dependabot[bot] commented on PR #162: URL: https://github.com/apache/iceberg-python/pull/162#issuecomment-1826117527 Superseded by #163. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Build: Bump mkdocs-material from 9.4.10 to 9.4.11 [iceberg-python]

dependabot[bot] closed pull request #162: Build: Bump mkdocs-material from 9.4.10 to 9.4.11 URL: https://github.com/apache/iceberg-python/pull/162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[I] test: Fix type error in `test_create_table` and `test_update_table` [iceberg-rust]

xiaozhiyan opened a new issue, #104: URL: https://github.com/apache/iceberg-rust/issues/104 Hi there, I've pulled `main` branch at https://github.com/apache/iceberg-rust/commit/a6d98c332385f5e89e8cb643df7ea15c5ec820e7 and found that tests `test_create_table` and `test_update_table` fail due

Re: [PR] Flink: Backport #8553 to v1.15, v1.16 [iceberg]

stevenzwu commented on PR #9139: URL: https://github.com/apache/iceberg/pull/9139#issuecomment-1826002055 @pvary I meant separate to multiple back port PRs. one PR for one purpose. it is easier to review too. -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] Flink missing backports [iceberg]

stevenzwu merged PR #9144: URL: https://github.com/apache/iceberg/pull/9144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] Nessie: reimplement namespace operations [iceberg]

adutra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1404580031 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -181,133 +186,206 @@ public IcebergTable table(TableIdentifier tableIdentifier) { }

Re: [PR] Flink: Proper backport for #8852 [iceberg]

pvary commented on PR #9146: URL: https://github.com/apache/iceberg/pull/9146#issuecomment-1825967639 Thanks @nastra for the review and the merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Core: Schema for a branch should return table schema [iceberg]

rdblue commented on code in PR #9131: URL: https://github.com/apache/iceberg/pull/9131#discussion_r1404544526 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java: ## @@ -171,7 +171,7 @@ public Table loadTable(Identifier ident, String version) throws NoS

Re: [PR] Flink: Proper backport for #8852 [iceberg]

nastra merged PR #9146: URL: https://github.com/apache/iceberg/pull/9146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Nessie: reimplement namespace operations [iceberg]

adutra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1404533616 ## nessie/src/test/java/org/apache/iceberg/nessie/TestMultipleClients.java: ## @@ -67,33 +71,84 @@ public void afterEach() throws Exception { } @Test - public v

Re: [PR] Aliyun: Switch iceberg-aliyun's tests to Junit5 [iceberg]

nastra commented on PR #9122: URL: https://github.com/apache/iceberg/pull/9122#issuecomment-1825872962 > > > Btw, should I squash the commits together or you will do that when merging the pr? > > > > > > commits will be squashed when the PR is getting merged, so it's up to you if

Re: [PR] Spec: Clarify partition equality [iceberg]

emkornfield commented on code in PR #9125: URL: https://github.com/apache/iceberg/pull/9125#discussion_r1404503860 ## format/spec.md: ## @@ -305,6 +305,10 @@ The source column, selected by id, must be a primitive type and cannot be contai Partition specs capture the transfor

Re: [PR] Spec: Clarify partition equality [iceberg]

emkornfield commented on code in PR #9125: URL: https://github.com/apache/iceberg/pull/9125#discussion_r1404501309 ## format/spec.md: ## @@ -607,6 +611,8 @@ Notes: 1. An alternative, *strict projection*, creates a partition predicate that will match a file if all of the rows

Re: [PR] Spec: Clarify partition equality [iceberg]

emkornfield commented on code in PR #9125: URL: https://github.com/apache/iceberg/pull/9125#discussion_r1404499619 ## format/spec.md: ## @@ -607,6 +611,8 @@ Notes: 1. An alternative, *strict projection*, creates a partition predicate that will match a file if all of the rows

Re: [I] Considering adjust the default row-group size of Parquet position delete file [iceberg]

zhongyujiang commented on issue #9149: URL: https://github.com/apache/iceberg/issues/9149#issuecomment-1825831844 Based on this idea I did some tuning and [benchmarking](https://github.com/apache/iceberg/commit/39b5939980f2d35ec822d8bf211f26489c1b#diff-9eb38b06454a156e92135fab05e90955ac4

[I] Considering adjust the default row-group size of Parquet position delete file [iceberg]

zhongyujiang opened a new issue, #9149: URL: https://github.com/apache/iceberg/issues/9149 ### Feature Request / Improvement When MoR mode is truned on, Spark-iceberg connector will write the position deletion of multiple data files into one pos delete file. This will reduce the per

[I] Unclosed input streams when writing concurrently [iceberg]

matthijseikelenboom opened a new issue, #9148: URL: https://github.com/apache/iceberg/issues/9148 ### Query engine Spark ### Question Hi, I have a question about the combined use of Apache Spark and Iceberg. I'm trying to concurrently write to Iceberg, but I get `Unclose

Re: [PR] Aliyun: Switch iceberg-aliyun's tests to Junit5 [iceberg]

lisirrx commented on PR #9122: URL: https://github.com/apache/iceberg/pull/9122#issuecomment-1825810278 > > Btw, should I squash the commits together or you will do that when merging the pr? > > commits will be squashed when the PR is getting merged, so it's up to you if you want to

Re: [PR] Nessie: reimplement namespace operations [iceberg]

adutra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1404449824 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -181,133 +186,206 @@ public IcebergTable table(TableIdentifier tableIdentifier) { }

Re: [PR] Nessie: reimplement namespace operations [iceberg]

nastra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1404431932 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -181,133 +186,206 @@ public IcebergTable table(TableIdentifier tableIdentifier) { }

Re: [PR] Nessie: reimplement namespace operations [iceberg]

nastra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1404430568 ## nessie/src/test/java/org/apache/iceberg/nessie/TestMultipleClients.java: ## @@ -78,10 +81,19 @@ public void testListNamespaces() { Assertions.assertThat(catalo

Re: [PR] Nessie: reimplement namespace operations [iceberg]

adutra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1404403854 ## nessie/src/test/java/org/apache/iceberg/nessie/TestMultipleClients.java: ## @@ -78,10 +81,19 @@ public void testListNamespaces() { Assertions.assertThat(catalo

Re: [I] Question about "Column ID" in Parquet Sepc [iceberg]

wgtmac commented on issue #9099: URL: https://github.com/apache/iceberg/issues/9099#issuecomment-1825733041 @bitsondatadev @Fokko It seems that we need to clear the confusion here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] Question about "Column ID" in Parquet Sepc [iceberg]

mapleFU commented on issue #9099: URL: https://github.com/apache/iceberg/issues/9099#issuecomment-1825718253 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Nessie: reimplement namespace operations [iceberg]

adutra commented on code in PR #8857: URL: https://github.com/apache/iceberg/pull/8857#discussion_r1404380359 ## nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java: ## @@ -181,133 +186,206 @@ public IcebergTable table(TableIdentifier tableIdentifier) { }

Re: [I] Core : Optimise manifest evaluation for tables with large number of columns [iceberg]

irshadcc commented on issue #9118: URL: https://github.com/apache/iceberg/issues/9118#issuecomment-1825680468 Pull request #9147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Core: Optimize manifest evaluation for super wide tables [iceberg]

irshadcc commented on PR #9147: URL: https://github.com/apache/iceberg/pull/9147#issuecomment-1825679606 Resolves issue #9118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Create JUnit5 version of FlinkTestBase [iceberg]

cgpoh commented on PR #9120: URL: https://github.com/apache/iceberg/pull/9120#issuecomment-1825619226 Thanks! Will create the PR later -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Create JUnit5-version of SparkCatalogTestBase [iceberg]

nastra closed issue #9076: Create JUnit5-version of SparkCatalogTestBase URL: https://github.com/apache/iceberg/issues/9076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [I] Create JUnit5-version of SparkTestBaseWithCatalog [iceberg]

nastra closed issue #9075: Create JUnit5-version of SparkTestBaseWithCatalog URL: https://github.com/apache/iceberg/issues/9075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Create JUnit5-version of SparkTestBase [iceberg]

nastra closed issue #9074: Create JUnit5-version of SparkTestBase URL: https://github.com/apache/iceberg/issues/9074 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] Create SparkCatalogTestBase class for the migration to JUnit5 [iceberg]

nastra merged PR #9129: URL: https://github.com/apache/iceberg/pull/9129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Hive: Refactor TestHiveCatalog tests to use the core CatalogTests [iceberg]

nk1506 commented on code in PR #8918: URL: https://github.com/apache/iceberg/pull/8918#discussion_r1404220048 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java: ## @@ -261,6 +261,12 @@ public void renameTable(TableIdentifier from, TableIdentifier original

Re: [PR] Hive: Refactor HiveTableOperations with common code for View. [iceberg]

nk1506 commented on PR #9011: URL: https://github.com/apache/iceberg/pull/9011#issuecomment-1825492456 Thanks @pvary for suggesting and reviewing this. I will start rebasing View [PR](https://github.com/apache/iceberg/pull/8907). -- This is an automated message from the Apache Git Servic

Re: [PR] Flink: Read parquet BINARY column as String for expected [iceberg]

fengjiajie commented on PR #8808: URL: https://github.com/apache/iceberg/pull/8808#issuecomment-1825487523 > I'm also a little nervous about this change, how are we guaranteed that the binary is parsable as UTF8 bytes? Seems like we should just be fixing the type annotations rather than cha

Re: [I] Create JUnit5-version of SparkTestBase [iceberg]

nastra commented on issue #9074: URL: https://github.com/apache/iceberg/issues/9074#issuecomment-1825425390 fixed by https://github.com/apache/iceberg/pull/9129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [I] Create JUnit5-version of SparkTestBaseWithCatalog [iceberg]

nastra commented on issue #9075: URL: https://github.com/apache/iceberg/issues/9075#issuecomment-1825425237 fixed by https://github.com/apache/iceberg/pull/9129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Create SparkCatalogTestBase class for the migration to JUnit5 [iceberg]

tomtongue commented on code in PR #9129: URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404151629 ## spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/SparkTestHelperBase.java: ## @@ -55,12 +55,15 @@ private Object[] toJava(Row row) { protected void ass

Re: [PR] Create SparkCatalogTestBase class for the migration to JUnit5 [iceberg]

tomtongue commented on PR #9129: URL: https://github.com/apache/iceberg/pull/9129#issuecomment-1825406804 Thanks for the review again. Change the tests that are commented here. I reviewed all my changes on my end again, but If there're any part that should be changed, please let me know @na

Re: [PR] Flink: Backport #8553 to v1.15, v1.16 [iceberg]

pvary commented on PR #9139: URL: https://github.com/apache/iceberg/pull/9139#issuecomment-1825349863 > @pvary can you separate this out as a separate PR? > > Fix backport differences for #8852 (#8877, #8880) TBH, I do not really understand your comment here 😞 To decrease

Re: [I] Implement rest catalog. [iceberg-rust]

Fokko closed issue #60: Implement rest catalog. URL: https://github.com/apache/iceberg-rust/issues/60 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: is

Re: [PR] feat: Implement create table and update table api for rest catalog. [iceberg-rust]

Fokko merged PR #97: URL: https://github.com/apache/iceberg-rust/pull/97 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] feat: Implement create table and update table api for rest catalog. [iceberg-rust]

Fokko commented on code in PR #97: URL: https://github.com/apache/iceberg-rust/pull/97#discussion_r1404061539 ## crates/catalog/rest/src/catalog.rs: ## @@ -1017,31 +1109,31 @@ mod tests { .with_summary(Summary { operation: Operation::Append,

Re: [PR] Create JUnit5 version of FlinkTestBase [iceberg]

pvary commented on PR #9120: URL: https://github.com/apache/iceberg/pull/9120#issuecomment-1825287747 Thanks for your work @cgpoh and @nastra for the review! @cgpoh: Please create the backport PR to v1.15, v1.16. It could be one PR, and please make sure that the resulting code is the