[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098292696 ## spark/v3.3/spark-extensions/src/main/antlr/org.apache.spark.sql.catalyst.parser.extensions/IcebergSqlExtensions.g4: ## @@ -74,6 +74,7 @@ statement | AL

[GitHub] [iceberg] chenjunjiedada commented on pull request #6753: Flink: use projection when delete records in upsert mode

chenjunjiedada commented on PR #6753: URL: https://github.com/apache/iceberg/pull/6753#issuecomment-1420333684 @stevenzwu @hililiwei You guys may want to take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [iceberg] hililiwei commented on a diff in pull request #5029: Flink: Use Tag or Branch to scan data.

hililiwei commented on code in PR #5029: URL: https://github.com/apache/iceberg/pull/5029#discussion_r1098266653 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java: ## @@ -125,7 +137,24 @@ private void validate() { startSnapshotId ==

[GitHub] [iceberg] hililiwei commented on a diff in pull request #5029: Flink: Use Tag or Branch to scan data.

hililiwei commented on code in PR #5029: URL: https://github.com/apache/iceberg/pull/5029#discussion_r1098252250 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java: ## @@ -125,7 +137,24 @@ private void validate() { startSnapshotId ==

[GitHub] [iceberg] hililiwei commented on a diff in pull request #5029: Flink: Use Tag or Branch to scan data.

hililiwei commented on code in PR #5029: URL: https://github.com/apache/iceberg/pull/5029#discussion_r1098252250 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java: ## @@ -125,7 +137,24 @@ private void validate() { startSnapshotId ==

[GitHub] [iceberg] huaxingao commented on a diff in pull request #6622: push down min/max/count to iceberg

huaxingao commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1098240584 ## api/src/main/java/org/apache/iceberg/expressions/AggregateEvaluator.java: ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] huaxingao commented on a diff in pull request #6622: push down min/max/count to iceberg

huaxingao commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1098240039 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -158,6 +182,141 @@ public Filter[] pushedFilters() { return pushed

[GitHub] [iceberg] huaxingao commented on a diff in pull request #6622: push down min/max/count to iceberg

huaxingao commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1098239129 ## api/src/main/java/org/apache/iceberg/expressions/BoundAggregate.java: ## @@ -44,4 +57,85 @@ public Type type() { return term().type(); } } + + publ

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098229352 ## spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -210,7 +210,9 @@ cla

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098228124 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestDropBranch.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Softw

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098227272 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestDropBranch.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Softw

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098226674 ## spark/v3.3/spark-extensions/src/main/antlr/org.apache.spark.sql.catalyst.parser.extensions/IcebergSqlExtensions.g4: ## @@ -74,6 +74,7 @@ statement | AL

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098226674 ## spark/v3.3/spark-extensions/src/main/antlr/org.apache.spark.sql.catalyst.parser.extensions/IcebergSqlExtensions.g4: ## @@ -74,6 +74,7 @@ statement | AL

[GitHub] [iceberg] hililiwei commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

hililiwei commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098224917 ## spark/v3.3/spark-extensions/src/main/antlr/org.apache.spark.sql.catalyst.parser.extensions/IcebergSqlExtensions.g4: ## @@ -74,6 +74,7 @@ statement | ALTER TAB

[GitHub] [iceberg] hililiwei commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

hililiwei commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098223331 ## spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -210,7 +210,9 @@ class Iceb

[GitHub] [iceberg] hililiwei commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

hililiwei commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098223331 ## spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -210,7 +210,9 @@ class Iceb

[GitHub] [iceberg] hililiwei commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

hililiwei commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1098223331 ## spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala: ## @@ -210,7 +210,9 @@ class Iceb

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1097921035 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestDropBranch.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Softw

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

JonasJ-ap commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1098210476 ## delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java: ## @@ -0,0 +1,398 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

JonasJ-ap commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1098210476 ## delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java: ## @@ -0,0 +1,398 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [iceberg] bluzy commented on pull request #6761: Hive: add retry for metastore permission error

bluzy commented on PR #6761: URL: https://github.com/apache/iceberg/pull/6761#issuecomment-1420241011 > I think #6175 would be a better fix for your issue @pvary Oh I didn't know that. I think #6175 can solve my issue. Thank you. -- This is an automated message from the Apache Gi

[GitHub] [iceberg] bluzy closed pull request #6761: Hive: add retry for metastore permission error

bluzy closed pull request #6761: Hive: add retry for metastore permission error URL: https://github.com/apache/iceberg/pull/6761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [iceberg] pvary commented on pull request #6761: Hive: add retry for metastore permission error

pvary commented on PR #6761: URL: https://github.com/apache/iceberg/pull/6761#issuecomment-1420238961 I think #6175 would be a better fix for your issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [iceberg] pvary commented on issue #6750: Failed to get table info from metastore using impersonation

pvary commented on issue #6750: URL: https://github.com/apache/iceberg/issues/6750#issuecomment-1420235524 I think that adding the user to the cache key in the HMS client pool would be the best solution. See #6175 -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [iceberg] pvary commented on a diff in pull request #6614: Flink:fix flink streaming query problem [ Cannot get a client from a closed pool]

pvary commented on code in PR #6614: URL: https://github.com/apache/iceberg/pull/6614#discussion_r1098200427 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/TableLoader.java: ## @@ -34,12 +34,16 @@ * the cluster (for example, to get splits), not just on the client

[GitHub] [iceberg] pvary commented on a diff in pull request #6614: Flink:fix flink streaming query problem [ Cannot get a client from a closed pool]

pvary commented on code in PR #6614: URL: https://github.com/apache/iceberg/pull/6614#discussion_r1098199612 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/TableLoader.java: ## @@ -91,7 +106,6 @@ class CatalogTableLoader implements TableLoader { private final

[GitHub] [iceberg] bluzy opened a new pull request, #6761: Hive: add retry for metastore permission error

bluzy opened a new pull request, #6761: URL: https://github.com/apache/iceberg/pull/6761 FIx : https://github.com/apache/iceberg/issues/6750 ## Situation 1. A user create `RetryingMetaStoreClient` instance, use it, and release. 2. B user get `RetryingMetaStoreClient` created

[GitHub] [iceberg] slfan1989 commented on pull request #6735: replace-parquet-deprecated-method

slfan1989 commented on PR #6735: URL: https://github.com/apache/iceberg/pull/6735#issuecomment-1420194127 @Fokko @jackye1995 Can you help review this pr? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

jackye1995 commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1098151723 ## build.gradle: ## @@ -438,6 +442,77 @@ project(':iceberg-aws') { } } +project(':iceberg-delta-lake') { + // use integration test since we can take advantage

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

jackye1995 commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1098151208 ## delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java: ## @@ -0,0 +1,398 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [iceberg] ludlows opened a new pull request, #6760: spark: use a deterministic where condition to make rewrite_data_files…

ludlows opened a new pull request, #6760: URL: https://github.com/apache/iceberg/pull/6760 the behavior is requested in the issue #6759 here I implement the evaluation procedure to check if the where condition is a deterministic false. if so, the rewrite_data_files exits directly.

[GitHub] [iceberg] hililiwei commented on a diff in pull request #5029: Flink: Use Tag or Branch to scan data.

hililiwei commented on code in PR #5029: URL: https://github.com/apache/iceberg/pull/5029#discussion_r1098137262 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/source/StreamingMonitorFunction.java: ## @@ -124,11 +126,33 @@ public void initializeState(FunctionInitial

[GitHub] [iceberg] ludlows opened a new issue, #6759: Spark: use where condition to control the execution of rewriteDataFiles for procedure rewrite data files in spark sql

ludlows opened a new issue, #6759: URL: https://github.com/apache/iceberg/issues/6759 ### Feature Request / Improvement ### Improvement ### background we have known that the rewriteDataFiles is suggested to run periodically. in our production, we would like to run

[GitHub] [iceberg] eric666666 commented on pull request #6581: Spark 3.3: Add RemoveDanglingDeletes action

eric66 commented on PR #6581: URL: https://github.com/apache/iceberg/pull/6581#issuecomment-1420131220 > Is this action currently available? Is there any usage documentation. We have already applied the v2 table to production. It is too uncomfortable to delete the delete file when

[GitHub] [iceberg] eric666666 commented on pull request #6581: Spark 3.3: Add RemoveDanglingDeletes action

eric66 commented on PR #6581: URL: https://github.com/apache/iceberg/pull/6581#issuecomment-1420127931 Is this action currently available? Is there any usage documentation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [iceberg] namrathamyske commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

namrathamyske commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098126130 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStringM

[GitHub] [iceberg] namrathamyske commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

namrathamyske commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098126130 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStringM

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098097479 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098097479 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

JonasJ-ap commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1098097617 ## delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java: ## @@ -0,0 +1,398 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098097479 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098094220 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] zoucao commented on issue #6756: The out-of-order problem occurs around the process of recovery

zoucao commented on issue #6756: URL: https://github.com/apache/iceberg/issues/6756#issuecomment-1420050929 gentle ping @stevenzwu, what do you think about it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098094220 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

JonasJ-ap commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1098093883 ## delta-lake/src/main/java/org/apache/iceberg/delta/BaseSnapshotDeltaLakeTableAction.java: ## @@ -0,0 +1,405 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

JonasJ-ap commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1098093494 ## build.gradle: ## @@ -438,6 +442,77 @@ project(':iceberg-aws') { } } +project(':iceberg-delta-lake') { + // use integration test since we can take advantages

[GitHub] [iceberg] theoxu31 commented on a diff in pull request #6742: support registerTable in GlueCatalog

theoxu31 commented on code in PR #6742: URL: https://github.com/apache/iceberg/pull/6742#discussion_r1098092626 ## aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java: ## @@ -437,6 +439,44 @@ public void renameTable(TableIdentifier from, TableIdentifier to) { LO

[GitHub] [iceberg] 372242283 commented on issue #3466: "Manifest is missing" ValidationException when there have Concurrent applications to rewrite manifests

372242283 commented on issue #3466: URL: https://github.com/apache/iceberg/issues/3466#issuecomment-1420045743 > As I wrote above the issue os that the rewrite command becomes out of date while running so it fails. At least this is my hypothesis above This problem happened by accident

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098076045 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098075811 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098074207 ## core/src/test/java/org/apache/iceberg/TestSnapshotOperations.java: ## @@ -0,0 +1,209 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098073489 ## core/src/test/java/org/apache/iceberg/TestSnapshotOperations.java: ## @@ -0,0 +1,209 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098072879 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098072499 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098071002 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1071,6 +1022,11 @@ public Builder addSortOrder(SortOrder order) { return this; } +p

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098069109 ## core/src/main/java/org/apache/iceberg/TableMetadataParser.java: ## @@ -436,16 +436,14 @@ static TableMetadata fromJson(String metadataLocation, JsonNode node) {

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098068922 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098067983 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098067598 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098066977 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098067020 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098066474 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098065548 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098064310 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098063044 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098062631 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098061423 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6651: Spark 3.3 write to branch snapshot

amogh-jahagirdar commented on code in PR #6651: URL: https://github.com/apache/iceberg/pull/6651#discussion_r1098060983 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java: ## @@ -247,9 +247,6 @@ public ScanBuilder newScanBuilder(CaseInsensitiveStri

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098060389 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098059626 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098059479 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098058915 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] rdblue commented on a diff in pull request #6706: Refactor table metadata snapshot management

rdblue commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098057989 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more con

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098050794 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098050503 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098050188 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098049871 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] danielcweeks commented on a diff in pull request #6706: Refactor table metadata snapshot management

danielcweeks commented on code in PR #6706: URL: https://github.com/apache/iceberg/pull/6706#discussion_r1098049424 ## core/src/main/java/org/apache/iceberg/SnapshotOperations.java: ## @@ -0,0 +1,190 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

[GitHub] [iceberg] github-actions[bot] commented on issue #5418: Refactoring flink upsert ut test so the table is partitioned by date.

github-actions[bot] commented on issue #5418: URL: https://github.com/apache/iceberg/issues/5418#issuecomment-1419974962 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[GitHub] [iceberg] Fokko commented on a diff in pull request #6646: Implement Support for DynamoDB Catalog

Fokko commented on code in PR #6646: URL: https://github.com/apache/iceberg/pull/6646#discussion_r1097196623 ## python/pyiceberg/catalog/__init__.py: ## @@ -431,3 +440,114 @@ def namespace_from(identifier: Union[str, Identifier]) -> Identifier: Identifier: Namespac

[GitHub] [iceberg] Fokko commented on a diff in pull request #6646: Implement Support for DynamoDB Catalog

Fokko commented on code in PR #6646: URL: https://github.com/apache/iceberg/pull/6646#discussion_r1098000908 ## python/pyiceberg/catalog/dynamodb.py: ## @@ -0,0 +1,776 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [iceberg] Fokko commented on a diff in pull request #6646: Implement Support for DynamoDB Catalog

Fokko commented on code in PR #6646: URL: https://github.com/apache/iceberg/pull/6646#discussion_r1097181300 ## python/mkdocs/docs/configuration.md: ## @@ -85,3 +85,16 @@ catalog: default: type: glue ``` + +## DynamoDB Catalog + +If you want to use AWS DynamoDB as the c

[GitHub] [iceberg] aokolnychyi commented on issue #6739: Integrate CRT with Iceberg S3 client

aokolnychyi commented on issue #6739: URL: https://github.com/apache/iceberg/issues/6739#issuecomment-1419851420 Sounds really promising! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [iceberg] Fokko commented on issue #6708: Quick start docker-compose demo doesn't work

Fokko commented on issue #6708: URL: https://github.com/apache/iceberg/issues/6708#issuecomment-1419814726 Can you run a `docker-compose pull` to make sure that you're running the latest version? We recently did some updates to the stack. -- This is an automated message from the Apache Gi

[GitHub] [iceberg] Fokko commented on issue #6708: Quick start docker-compose demo doesn't work

Fokko commented on issue #6708: URL: https://github.com/apache/iceberg/issues/6708#issuecomment-1419813815 > py4j.protocol.Py4JNetworkError: Answer from Java side is empty Means that the Spark process has died. Do you see anything in the logs? -- This is an automated message from th

[GitHub] [iceberg] rdblue commented on a diff in pull request #6622: push down min/max/count to iceberg

rdblue commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1097939971 ## api/src/main/java/org/apache/iceberg/expressions/CountNonNull.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

[GitHub] [iceberg] rdblue commented on a diff in pull request #6622: push down min/max/count to iceberg

rdblue commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1097936851 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -158,6 +182,141 @@ public Filter[] pushedFilters() { return pushedFil

[GitHub] [iceberg] rdblue commented on a diff in pull request #6622: push down min/max/count to iceberg

rdblue commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1097930735 ## api/src/main/java/org/apache/iceberg/expressions/BoundAggregate.java: ## @@ -44,4 +57,85 @@ public Type type() { return term().type(); } } + + public

[GitHub] [iceberg] rdblue commented on a diff in pull request #6622: push down min/max/count to iceberg

rdblue commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1097921708 ## api/src/main/java/org/apache/iceberg/expressions/AggregateEvaluator.java: ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6752: Spark: DROP BRANCH SQL implementation

amogh-jahagirdar commented on code in PR #6752: URL: https://github.com/apache/iceberg/pull/6752#discussion_r1097921035 ## spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestDropBranch.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Softw

[GitHub] [iceberg] rdblue commented on a diff in pull request #6622: push down min/max/count to iceberg

rdblue commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1097921708 ## api/src/main/java/org/apache/iceberg/expressions/AggregateEvaluator.java: ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

[GitHub] [iceberg] rdblue commented on a diff in pull request #6622: push down min/max/count to iceberg

rdblue commented on code in PR #6622: URL: https://github.com/apache/iceberg/pull/6622#discussion_r1097920494 ## api/src/main/java/org/apache/iceberg/expressions/AggregateEvaluator.java: ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

  1   2   >