Re: [PR] Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.6 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] commented on PR #11357: URL: https://github.com/apache/iceberg/pull/11357#issuecomment-2439844286 Superseded by #11399. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] Build: Bump mkdocs-material from 9.5.39 to 9.5.41 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] closed pull request #11358: Build: Bump mkdocs-material from 9.5.39 to 9.5.41 URL: https://github.com/apache/iceberg/pull/11358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Build: Bump mkdocs-material from 9.5.39 to 9.5.41 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] commented on PR #11358: URL: https://github.com/apache/iceberg/pull/11358#issuecomment-2439844276 Superseded by #11398. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[PR] Build: Bump software.amazon.awssdk:bom from 2.28.26 to 2.29.1 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11400: URL: https://github.com/apache/iceberg/pull/11400 Bumps software.amazon.awssdk:bom from 2.28.26 to 2.29.1. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softw

[PR] Build: Bump mkdocs-material from 9.5.39 to 9.5.42 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11398: URL: https://github.com/apache/iceberg/pull/11398 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.39 to 9.5.42. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mkdoc

[PR] Build: Bump testcontainers from 1.20.2 to 1.20.3 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11404: URL: https://github.com/apache/iceberg/pull/11404 Bumps `testcontainers` from 1.20.2 to 1.20.3. Updates `org.testcontainers:testcontainers` from 1.20.2 to 1.20.3 Release notes Sourced from https://github.com/testcontainers/testco

[PR] Build: Bump junit from 5.11.1 to 5.11.3 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11401: URL: https://github.com/apache/iceberg/pull/11401 Bumps `junit` from 5.11.1 to 5.11.3. Updates `org.junit.jupiter:junit-jupiter` from 5.11.1 to 5.11.3 Release notes Sourced from https://github.com/junit-team/junit5/releases";>org

Re: [PR] Build: Bump junit from 5.10.1 to 5.11.2 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] closed pull request #11263: Build: Bump junit from 5.10.1 to 5.11.2 URL: https://github.com/apache/iceberg/pull/11263 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Build: Bump junit from 5.10.1 to 5.11.2 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] commented on PR #11263: URL: https://github.com/apache/iceberg/pull/11263#issuecomment-2439846057 Superseded by #11401. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[PR] Build: Bump net.snowflake:snowflake-jdbc from 3.19.0 to 3.19.1 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11406: URL: https://github.com/apache/iceberg/pull/11406 Bumps [net.snowflake:snowflake-jdbc](https://github.com/snowflakedb/snowflake-jdbc) from 3.19.0 to 3.19.1. Release notes Sourced from https://github.com/snowflakedb/snowflake-jdbc/

[PR] Build: Bump software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin from 2.2.0 to 2.3.0 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11405: URL: https://github.com/apache/iceberg/pull/11405 Bumps [software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin](https://github.com/aws/aws-s3-accessgrants-plugin-java-v2) from 2.2.0 to 2.3.0. Changelog Sourced from https

[PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.34.0 to 2.35.1 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11403: URL: https://github.com/apache/iceberg/pull/11403 Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.34.0 to 2.35.1. Release notes Sourced from https://github.com/google/error-prone

[PR] Build: Bump junit-platform from 1.11.2 to 1.11.3 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11402: URL: https://github.com/apache/iceberg/pull/11402 Bumps `junit-platform` from 1.11.2 to 1.11.3. Updates `org.junit.platform:junit-platform-suite-api` from 1.11.2 to 1.11.3 Commits See full diff in https://github.com/junit-tea

Re: [PR] Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.6 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] closed pull request #11357: Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.6 URL: https://github.com/apache/iceberg/pull/11357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[PR] Build: Bump org.xerial:sqlite-jdbc from 3.46.1.3 to 3.47.0.0 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11407: URL: https://github.com/apache/iceberg/pull/11407 Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.46.1.3 to 3.47.0.0. Release notes Sourced from https://github.com/xerial/sqlite-jdbc/releases";>org.xeri

[PR] Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.7 [iceberg]

2024-10-26 Thread via GitHub
dependabot[bot] opened a new pull request, #11399: URL: https://github.com/apache/iceberg/pull/11399 Bumps [mkdocs-macros-plugin](https://github.com/fralau/mkdocs_macros_plugin) from 1.2.0 to 1.3.7. Changelog Sourced from https://github.com/fralau/mkdocs-macros-plugin/blob/master/C

Re: [PR] Flink: Add RowConverter for Iceberg Source [iceberg]

2024-10-26 Thread via GitHub
stevenzwu commented on code in PR #11301: URL: https://github.com/apache/iceberg/pull/11301#discussion_r1817982191 ## flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/source/reader/RowConverter.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] Aliyun: Remove spring-boot dependency [iceberg]

2024-10-26 Thread via GitHub
jbonofre commented on PR #11291: URL: https://github.com/apache/iceberg/pull/11291#issuecomment-2439877711 @findepi conflict resolved. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] feat: implement IcebergTableProviderFactory for datafusion [iceberg-rust]

2024-10-26 Thread via GitHub
yukkit commented on code in PR #600: URL: https://github.com/apache/iceberg-rust/pull/600#discussion_r1817714442 ## crates/integrations/datafusion/src/table/table_provider_factory.rs: ## @@ -0,0 +1,312 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] Parquet: Use native getRowIndexOffset support instead of calculating it [iceberg]

2024-10-26 Thread via GitHub
github-actions[bot] commented on PR #10107: URL: https://github.com/apache/iceberg/pull/10107#issuecomment-2439070986 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pul

Re: [PR] abort the whole table transaction if any updates in the transaction has failed [iceberg-python]

2024-10-26 Thread via GitHub
HonahX commented on PR #1246: URL: https://github.com/apache/iceberg-python/pull/1246#issuecomment-2439780764 > This seems like a potential footgun. Perhaps we should get rid of _autocommit, its not used anywhere https://github.com/search?q=repo%3Aapache%2Ficeberg-python%20_autocommit&type=

Re: [PR] abort the whole table transaction if any updates in the transaction has failed [iceberg-python]

2024-10-26 Thread via GitHub
HonahX commented on code in PR #1246: URL: https://github.com/apache/iceberg-python/pull/1246#discussion_r1817956072 ## pyiceberg/table/__init__.py: ## @@ -231,9 +233,13 @@ def __enter__(self) -> Transaction: """Start a transaction to update the table.""" retur

Re: [PR] abort the whole table transaction if any updates in the transaction has failed [iceberg-python]

2024-10-26 Thread via GitHub
HonahX commented on code in PR #1246: URL: https://github.com/apache/iceberg-python/pull/1246#discussion_r1817956072 ## pyiceberg/table/__init__.py: ## @@ -231,9 +233,13 @@ def __enter__(self) -> Transaction: """Start a transaction to update the table.""" retur

[PR] WIP: Testing out using Polaris Docker image in Integration Test Suite [iceberg-python]

2024-10-26 Thread via GitHub
sungwy opened a new pull request, #1252: URL: https://github.com/apache/iceberg-python/pull/1252 Testing out using Polaris Docker image which has newer REST Catalog endpoints implemented. This will allow us to test more features in our integration tests -- This is an automated message fr

Re: [PR] abort the whole table transaction if any updates in the transaction has failed [iceberg-python]

2024-10-26 Thread via GitHub
HonahX commented on PR #1246: URL: https://github.com/apache/iceberg-python/pull/1246#issuecomment-2439772946 @stevie9868 @kevinjqliu Thanks for the great PR and discussions! I agree that there is some issue with the current Transaction mechanism: the `commit_transaction` can be incorre

Re: [I] Pushdown of aggregate functions in where [iceberg]

2024-10-26 Thread via GitHub
github-actions[bot] commented on issue #10109: URL: https://github.com/apache/iceberg/issues/10109#issuecomment-2439776797 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occur

Re: [PR] abort the whole table transaction if any updates in the transaction has failed [iceberg-python]

2024-10-26 Thread via GitHub
stevie9868 commented on PR #1246: URL: https://github.com/apache/iceberg-python/pull/1246#issuecomment-2439778620 @HonahX Thanks for providing a detailed example, and I agree that we should only call commit_transaction when there is no exception along the way. -- This is an autom

[I] Flaky test `TestFlinkIcebergSinkRangeDistributionBucketing > testBucketNumberHigherThanWriterParallelismNotDivisible()` [iceberg]

2024-10-26 Thread via GitHub
manuzhang opened a new issue, #11397: URL: https://github.com/apache/iceberg/issues/11397 ### Apache Iceberg version main (development) ### Query engine Flink ### Please describe the bug 🐞 https://github.com/apache/iceberg/actions/runs/11525609495/job/320883

Re: [PR] Spark 3.5: Fix NotSerializableException when migrating Spark tables [iceberg]

2024-10-26 Thread via GitHub
manuzhang commented on code in PR #11157: URL: https://github.com/apache/iceberg/pull/11157#discussion_r1817876302 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -92,6 +98,8 @@ import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan

Re: [PR] Spark 3.5: Fix NotSerializableException when migrating Spark tables [iceberg]

2024-10-26 Thread via GitHub
manuzhang commented on code in PR #11157: URL: https://github.com/apache/iceberg/pull/11157#discussion_r1817894453 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -711,7 +719,7 @@ public static void importSparkPartitions( spec,

Re: [PR] Spark 3.5: Fix NotSerializableException when migrating Spark tables [iceberg]

2024-10-26 Thread via GitHub
manuzhang commented on code in PR #11157: URL: https://github.com/apache/iceberg/pull/11157#discussion_r1817893959 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -711,7 +719,7 @@ public static void importSparkPartitions( spec,

Re: [PR] Flink: disable the flaky range distribution bucketing tests for now [iceberg]

2024-10-26 Thread via GitHub
manuzhang commented on PR #11347: URL: https://github.com/apache/iceberg/pull/11347#issuecomment-2439627068 It looks this still failed in recent builds. Details in https://github.com/apache/iceberg/issues/11397. -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] Core: Snapshot `summary` map must have `operation` key [iceberg]

2024-10-26 Thread via GitHub
amogh-jahagirdar commented on code in PR #11354: URL: https://github.com/apache/iceberg/pull/11354#discussion_r1817912195 ## core/src/main/java/org/apache/iceberg/SnapshotParser.java: ## @@ -129,13 +129,12 @@ static Snapshot fromJson(JsonNode node) { "Cannot parse sum

Re: [PR] Core: Snapshot `summary` map must have `operation` key [iceberg]

2024-10-26 Thread via GitHub
amogh-jahagirdar commented on code in PR #11354: URL: https://github.com/apache/iceberg/pull/11354#discussion_r1817912195 ## core/src/main/java/org/apache/iceberg/SnapshotParser.java: ## @@ -129,13 +129,12 @@ static Snapshot fromJson(JsonNode node) { "Cannot parse sum

Re: [PR] Core: Snapshot `summary` map must have `operation` key [iceberg]

2024-10-26 Thread via GitHub
amogh-jahagirdar commented on code in PR #11354: URL: https://github.com/apache/iceberg/pull/11354#discussion_r1817912195 ## core/src/main/java/org/apache/iceberg/SnapshotParser.java: ## @@ -129,13 +129,12 @@ static Snapshot fromJson(JsonNode node) { "Cannot parse sum

Re: [PR] Add list_view method for hive catalog [iceberg-python]

2024-10-26 Thread via GitHub
omkenge closed pull request #1250: Add list_view method for hive catalog URL: https://github.com/apache/iceberg-python/pull/1250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Add list_views for hive catalog [iceberg-python]

2024-10-26 Thread via GitHub
omkenge commented on PR #1251: URL: https://github.com/apache/iceberg-python/pull/1251#issuecomment-2439411543 Hello @kbendick As we discussed earlier, I created a PR for list_view for hive catalog Could you plz test it ? -- This is an automated message from the Apache Git Service.

[PR] Add list_views for hive catalog [iceberg-python]

2024-10-26 Thread via GitHub
omkenge opened a new pull request, #1251: URL: https://github.com/apache/iceberg-python/pull/1251 Added list_views and test_list_views for hive catalog -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Add list_views for hive catalog [iceberg-python]

2024-10-26 Thread via GitHub
omkenge commented on PR #1251: URL: https://github.com/apache/iceberg-python/pull/1251#issuecomment-2439411812 Hello @kevinjqliu As we discussed earlier, I created a PR for list_view for hive catalog Could you plz test it ? -- This is an automated message from the Apache Git Service

Re: [PR] Spark 3.5: Fix NotSerializableException when migrating Spark tables [iceberg]

2024-10-26 Thread via GitHub
manuzhang commented on code in PR #11157: URL: https://github.com/apache/iceberg/pull/11157#discussion_r1817893959 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java: ## @@ -711,7 +719,7 @@ public static void importSparkPartitions( spec,

Re: [PR] Core: Snapshot `summary` map must have `operation` key [iceberg]

2024-10-26 Thread via GitHub
rdblue commented on code in PR #11354: URL: https://github.com/apache/iceberg/pull/11354#discussion_r1817898860 ## core/src/main/java/org/apache/iceberg/SnapshotParser.java: ## @@ -129,13 +129,12 @@ static Snapshot fromJson(JsonNode node) { "Cannot parse summary from

Re: [PR] Core: Snapshot `summary` map must have `operation` key [iceberg]

2024-10-26 Thread via GitHub
amogh-jahagirdar commented on code in PR #11354: URL: https://github.com/apache/iceberg/pull/11354#discussion_r1817905519 ## core/src/main/java/org/apache/iceberg/SnapshotParser.java: ## @@ -129,13 +129,12 @@ static Snapshot fromJson(JsonNode node) { "Cannot parse sum

Re: [PR] Core: Snapshot `summary` map must have `operation` key [iceberg]

2024-10-26 Thread via GitHub
amogh-jahagirdar commented on code in PR #11354: URL: https://github.com/apache/iceberg/pull/11354#discussion_r1817905519 ## core/src/main/java/org/apache/iceberg/SnapshotParser.java: ## @@ -129,13 +129,12 @@ static Snapshot fromJson(JsonNode node) { "Cannot parse sum