[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #6335: Core: Avoid generating a large ManifestFile when committing

ConeyLiu commented on code in PR #6335: URL: https://github.com/apache/iceberg/pull/6335#discussion_r1063976959 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -49,8 +51,9 @@ class FastAppend extends SnapshotProducer implements AppendFiles { private final Lis

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6451: Hive: Lock hardening

szehon-ho commented on code in PR #6451: URL: https://github.com/apache/iceberg/pull/6451#discussion_r1063985982 ## docs/configuration.md: ## @@ -160,14 +160,20 @@ Here are the catalog properties related to locking. They are used by some catalo The following properties from

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6451: Hive: Lock hardening

szehon-ho commented on code in PR #6451: URL: https://github.com/apache/iceberg/pull/6451#discussion_r1063985982 ## docs/configuration.md: ## @@ -160,14 +160,20 @@ Here are the catalog properties related to locking. They are used by some catalo The following properties from

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6451: Hive: Lock hardening

szehon-ho commented on code in PR #6451: URL: https://github.com/apache/iceberg/pull/6451#discussion_r1063985982 ## docs/configuration.md: ## @@ -160,14 +160,20 @@ Here are the catalog properties related to locking. They are used by some catalo The following properties from

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #6451: Hive: Lock hardening

szehon-ho commented on code in PR #6451: URL: https://github.com/apache/iceberg/pull/6451#discussion_r1063985982 ## docs/configuration.md: ## @@ -160,14 +160,20 @@ Here are the catalog properties related to locking. They are used by some catalo The following properties from

[GitHub] [iceberg] marsupialtail opened a new issue, #6541: Does PyIceberg support DynamoDB catalog?

marsupialtail opened a new issue, #6541: URL: https://github.com/apache/iceberg/issues/6541 ### Query engine PyIceberg -- I just want to read the metadata ### Question Say I got a dynamodb table prodiceberg_metastore as my catalog. How I read it in PyIceberg? Can s

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6449: WIP: Delta: Adding support for Migrating Delta Lake Table to Iceberg Table

jackye1995 commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1064046246 ## delta-lake/src/integration/java/org/apache/iceberg/delta/TestMigrateDeltaLakeTable.java: ## @@ -0,0 +1,267 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [iceberg] danielcweeks merged pull request #6540: Python: Fix the mdformat issue

danielcweeks merged PR #6540: URL: https://github.com/apache/iceberg/pull/6540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

[GitHub] [iceberg] github-actions[bot] commented on issue #5251: Discrepancy between partition truncation transform in spec vs code (org.apache.iceberg.transforms.Transform)

github-actions[bot] commented on issue #5251: URL: https://github.com/apache/iceberg/issues/5251#issuecomment-1374663986 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[GitHub] [iceberg] github-actions[bot] commented on issue #5141: No way to rollback first commit in table

github-actions[bot] commented on issue #5141: URL: https://github.com/apache/iceberg/issues/5141#issuecomment-1374664011 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

[GitHub] [iceberg] github-actions[bot] closed issue #5141: No way to rollback first commit in table

github-actions[bot] closed issue #5141: No way to rollback first commit in table URL: https://github.com/apache/iceberg/issues/5141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [iceberg] github-actions[bot] closed issue #5139: Historical time travel imports

github-actions[bot] closed issue #5139: Historical time travel imports URL: https://github.com/apache/iceberg/issues/5139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [iceberg] github-actions[bot] commented on issue #5139: Historical time travel imports

github-actions[bot] commented on issue #5139: URL: https://github.com/apache/iceberg/issues/5139#issuecomment-1374664024 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064063897 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/DataStatistics.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064064168 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/DataStatisticsFactory.java: ## @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064064551 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/MapDataStatistics.java: ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064064723 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/MapDataStatistics.java: ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064064970 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/DataStatistics.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064064970 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/DataStatistics.java: ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064065085 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/MapDataStatistics.java: ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [iceberg] aokolnychyi commented on pull request #6534: Spark 3.3: Use regular planning for applicable row-level operations

aokolnychyi commented on PR #6534: URL: https://github.com/apache/iceberg/pull/6534#issuecomment-1374674560 I'll have to revisit the approach in this PR. It is actually not safe and may lead to data correctness bugs. I'll follow up with a fix next week. -- This is an automated message

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6542: Build: Bump spotless-plugin-gradle from 6.12.0 to 6.12.1

dependabot[bot] opened a new pull request, #6542: URL: https://github.com/apache/iceberg/pull/6542 Bumps [spotless-plugin-gradle](https://github.com/diffplug/spotless) from 6.12.0 to 6.12.1. Commits https://github.com/diffplug/spotless/commit/718a504c123de899e75300e7f2f6c55d7a4

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6543: Build: Bump moto from 4.0.12 to 4.0.13 in /python

dependabot[bot] opened a new pull request, #6543: URL: https://github.com/apache/iceberg/pull/6543 Bumps [moto](https://github.com/spulec/moto) from 4.0.12 to 4.0.13. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 4.0.13

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6544: Build: Bump rich from 13.0.0 to 13.0.1 in /python

dependabot[bot] opened a new pull request, #6544: URL: https://github.com/apache/iceberg/pull/6544 Bumps [rich](https://github.com/Textualize/rich) from 13.0.0 to 13.0.1. Release notes Sourced from https://github.com/Textualize/rich/releases";>rich's releases. Fix for splitti

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6545: Build: Bump coverage from 7.0.1 to 7.0.4 in /python

dependabot[bot] opened a new pull request, #6545: URL: https://github.com/apache/iceberg/pull/6545 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.0.1 to 7.0.4. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's changelog.

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064064551 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/MapDataStatistics.java: ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064080560 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064080676 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064080711 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064080833 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064081012 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r106408 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064083596 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064083656 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleRecordWrapper.java: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064083753 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleRecordWrapper.java: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064083871 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleRecordWrapper.java: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064083893 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/ShuffleRecordWrapper.java: ## @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064084354 ## flink/v1.16/flink/src/test/java/org/apache/iceberg/flink/sink/shuffle/TestShuffleOperator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] stevenzwu commented on a diff in pull request #6382: Implement ShuffleOperator to collect data statistics

stevenzwu commented on code in PR #6382: URL: https://github.com/apache/iceberg/pull/6382#discussion_r1064084432 ## flink/v1.16/flink/src/test/java/org/apache/iceberg/flink/sink/shuffle/TestShuffleOperator.java: ## @@ -0,0 +1,166 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6449: WIP: Delta: Adding support for Snapshot Delta Lake Table to Iceberg Table

jackye1995 commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1064097728 ## build.gradle: ## @@ -438,6 +442,70 @@ project(':iceberg-aws') { } } +project(':iceberg-delta-lake') { + configurations { +integrationImplementation.ext