[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6365: Core: Add position deletes metadata table

2023-01-14 Thread GitBox
RussellSpitzer commented on code in PR #6365: URL: https://github.com/apache/iceberg/pull/6365#discussion_r1070285372 ## core/src/main/java/org/apache/iceberg/PositionDeletesTable.java: ## @@ -0,0 +1,396 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * o

[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6365: Core: Add position deletes metadata table

2023-01-14 Thread GitBox
RussellSpitzer commented on code in PR #6365: URL: https://github.com/apache/iceberg/pull/6365#discussion_r1070286324 ## core/src/main/java/org/apache/iceberg/SnapshotScan.java: ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

[GitHub] [iceberg] nazq commented on issue #6564: Python write support

2023-01-14 Thread GitBox
nazq commented on issue #6564: URL: https://github.com/apache/iceberg/issues/6564#issuecomment-1382840481 Keen to see this one progress, especially w pyarrow. Cc @asheeshgarg. Happy to help out -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [iceberg] dmgcodevil commented on issue #6587: Wrong class, java.lang.Long, for object: 19367

2023-01-14 Thread GitBox
dmgcodevil commented on issue #6587: URL: https://github.com/apache/iceberg/issues/6587#issuecomment-1382887043 @RussellSpitzer ``` java.lang.IllegalArgumentException: Wrong class, java.lang.Long, for object: 19371 at org.apache.iceberg.PartitionData.get(PartitionData.java

[GitHub] [iceberg] anuragmantri commented on a diff in pull request #6588: Spark 3.3: Add Default Parallelism Level for All Spark Driver Based Deletes

2023-01-14 Thread GitBox
anuragmantri commented on code in PR #6588: URL: https://github.com/apache/iceberg/pull/6588#discussion_r1070395384 ## spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkSQLProperties.java: ## @@ -47,4 +47,8 @@ private SparkSQLProperties() {} public static final Str

[GitHub] [iceberg] rdblue commented on pull request #6525: Python: Refactor loading manifests

2023-01-14 Thread GitBox
rdblue commented on PR #6525: URL: https://github.com/apache/iceberg/pull/6525#issuecomment-1382904373 Thanks for getting this in, @Fokko! Nice work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [iceberg] krvikash commented on pull request #6512: Core: Support registerTable with REST session catalog

2023-01-14 Thread GitBox
krvikash commented on PR #6512: URL: https://github.com/apache/iceberg/pull/6512#issuecomment-1382904647 > If we want to support register, then I think the protocol itself should support register Hi @rdblue, Thanks for the review. If I understood correctly, you wanted to introduce a

[GitHub] [iceberg] rdblue merged pull request #6574: Python: Raise exception on deletes

2023-01-14 Thread GitBox
rdblue merged PR #6574: URL: https://github.com/apache/iceberg/pull/6574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

[GitHub] [iceberg] rdblue commented on pull request #6574: Python: Raise exception on deletes

2023-01-14 Thread GitBox
rdblue commented on PR #6574: URL: https://github.com/apache/iceberg/pull/6574#issuecomment-1382904724 Thanks, @Fokko. Good to have this in just in case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [iceberg] krvikash opened a new pull request, #6589: Core: Fix BaseMetastoreCatalog#registerTable to return full table name

2023-01-14 Thread GitBox
krvikash opened a new pull request, #6589: URL: https://github.com/apache/iceberg/pull/6589 `registerTable` should return full table name, just like how `loadTable` and `create` API returns -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [iceberg] rdblue opened a new pull request, #6590: Python: Add sql command to the CLI

2023-01-14 Thread GitBox
rdblue opened a new pull request, #6590: URL: https://github.com/apache/iceberg/pull/6590 This adds an experimental `sql` command to the CLI. The table output is not quite finished, but it can use duckdb to run queries, like this: ``` [blue@work python]$ time poetry run pyiceberg -

[GitHub] [iceberg] krvikash opened a new pull request, #6591: Avoid creating new metadata file when `registerTable` API is used

2023-01-14 Thread GitBox
krvikash opened a new pull request, #6591: URL: https://github.com/apache/iceberg/pull/6591 `HiveTableOperations` and `NessieTableOperations` were already handling this case, but other table operations are missing this case. -- This is an automated message from the Apache Git Service. To

[GitHub] [iceberg] krvikash commented on pull request #6512: Core: Support registerTable with REST session catalog

2023-01-14 Thread GitBox
krvikash commented on PR #6512: URL: https://github.com/apache/iceberg/pull/6512#issuecomment-1382956003 CI Failure will be fixed by https://github.com/apache/iceberg/pull/6589 ``` TestJdbcCatalog > testRegisterTable() FAILED org.junit.ComparisonFailure: Name must match expe

[GitHub] [iceberg] github-actions[bot] commented on issue #5293: Bump Spark version to 3.1.3 and 3.2.2 for vulnerability

2023-01-14 Thread GitBox
github-actions[bot] commented on issue #5293: URL: https://github.com/apache/iceberg/issues/5293#issuecomment-1382975029 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6592: Build: Bump spotless-plugin-gradle from 6.12.1 to 6.13.0

2023-01-14 Thread GitBox
dependabot[bot] opened a new pull request, #6592: URL: https://github.com/apache/iceberg/pull/6592 Bumps [spotless-plugin-gradle](https://github.com/diffplug/spotless) from 6.12.1 to 6.13.0. Commits https://github.com/diffplug/spotless/commit/fdb58d79c7952e8fcf55ff049fefd84faeb

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6593: Build: Bump moto from 4.0.13 to 4.1.0 in /python

2023-01-14 Thread GitBox
dependabot[bot] opened a new pull request, #6593: URL: https://github.com/apache/iceberg/pull/6593 Bumps [moto](https://github.com/getmoto/moto) from 4.0.13 to 4.1.0. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 4.1.0

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6594: Build: Bump rich from 13.0.1 to 13.1.0 in /python

2023-01-14 Thread GitBox
dependabot[bot] opened a new pull request, #6594: URL: https://github.com/apache/iceberg/pull/6594 Bumps [rich](https://github.com/Textualize/rich) from 13.0.1 to 13.1.0. Release notes Sourced from https://github.com/Textualize/rich/releases";>rich's releases. Jupyter traceba

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6595: Build: Bump pytest from 7.2.0 to 7.2.1 in /python

2023-01-14 Thread GitBox
dependabot[bot] opened a new pull request, #6595: URL: https://github.com/apache/iceberg/pull/6595 Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.2.0 to 7.2.1. Release notes Sourced from https://github.com/pytest-dev/pytest/releases";>pytest's releases. 7.2.1

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6596: Build: Bump coverage from 7.0.4 to 7.0.5 in /python

2023-01-14 Thread GitBox
dependabot[bot] opened a new pull request, #6596: URL: https://github.com/apache/iceberg/pull/6596 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.0.4 to 7.0.5. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's changelog.

[GitHub] [iceberg] dependabot[bot] opened a new pull request, #6597: Build: Bump requests from 2.28.1 to 2.28.2 in /python

2023-01-14 Thread GitBox
dependabot[bot] opened a new pull request, #6597: URL: https://github.com/apache/iceberg/pull/6597 Bumps [requests](https://github.com/psf/requests) from 2.28.1 to 2.28.2. Release notes Sourced from https://github.com/psf/requests/releases";>requests's releases. v2.28.2 2

[GitHub] [iceberg] dongjoon-hyun commented on pull request #6585: Update ORC to 1.8.2

2023-01-14 Thread GitBox
dongjoon-hyun commented on PR #6585: URL: https://github.com/apache/iceberg/pull/6585#issuecomment-1383071525 Than kyou, @Fokko and @nastra . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [iceberg] pvary merged pull request #6583: Flink: Refactor sink tests to use HadoopCatalogResource

2023-01-15 Thread GitBox
pvary merged PR #6583: URL: https://github.com/apache/iceberg/pull/6583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] pvary commented on pull request #6583: Flink: Refactor sink tests to use HadoopCatalogResource

2023-01-15 Thread GitBox
pvary commented on PR #6583: URL: https://github.com/apache/iceberg/pull/6583#issuecomment-1383105218 Merged to master. Thanks for the review @stevenzwu! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [iceberg] amogh-jahagirdar opened a new pull request, #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar opened a new pull request, #6598: URL: https://github.com/apache/iceberg/pull/6598 View representation core implementation Co-authored-by: John Zhuge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070698774 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070698774 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070698774 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
jackye1995 commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070708186 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default catal

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
jackye1995 commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070708150 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default catal

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
jackye1995 commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070708365 ## api/src/main/java/org/apache/iceberg/view/ViewRepresentation.java: ## @@ -19,14 +19,21 @@ package org.apache.iceberg.view; import java.util.Locale; +import or

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070719355 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] rdblue opened a new pull request, #6599: Core: Add information_schema.namespaces table

2023-01-15 Thread GitBox
rdblue opened a new pull request, #6599: URL: https://github.com/apache/iceberg/pull/6599 This is based on #6357 and will be rebased once that PR is merged. This adds `information_schema.namespaces` and base classes for creating new `information_schema` tables. Namespace filtering is

[GitHub] [iceberg] github-actions[bot] commented on issue #5163: Support catalog method to set table metadata

2023-01-15 Thread GitBox
github-actions[bot] commented on issue #5163: URL: https://github.com/apache/iceberg/issues/5163#issuecomment-1383295084 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[GitHub] [iceberg] github-actions[bot] commented on issue #5099: Spark3.2 + Iceberg 0.13.1: Produced a lot of small files

2023-01-15 Thread GitBox
github-actions[bot] commented on issue #5099: URL: https://github.com/apache/iceberg/issues/5099#issuecomment-1383295096 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

[GitHub] [iceberg] github-actions[bot] closed issue #5099: Spark3.2 + Iceberg 0.13.1: Produced a lot of small files

2023-01-15 Thread GitBox
github-actions[bot] closed issue #5099: Spark3.2 + Iceberg 0.13.1: Produced a lot of small files URL: https://github.com/apache/iceberg/issues/5099 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070722634 ## api/src/main/java/org/apache/iceberg/view/ViewRepresentation.java: ## @@ -19,14 +19,21 @@ package org.apache.iceberg.view; import java.util.Locale; +imp

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070722634 ## api/src/main/java/org/apache/iceberg/view/ViewRepresentation.java: ## @@ -19,14 +19,21 @@ package org.apache.iceberg.view; import java.util.Locale; +imp

[GitHub] [iceberg] JonasJ-ap opened a new pull request, #6600: Doc: Add descriptions and examples for Snapshot Delta Lake Table to Iceberg Table

2023-01-15 Thread GitBox
JonasJ-ap opened a new pull request, #6600: URL: https://github.com/apache/iceberg/pull/6600 This PR adds documentation for the `iceberg-delta-lake` module created in #6449 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070738205 ## core/src/main/java/org/apache/iceberg/util/JsonUtil.java: ## @@ -98,6 +103,14 @@ public static T parse(String json, FromJson parser) { } } + pub

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070738898 ## core/src/test/java/org/apache/iceberg/view/TestViewRepresentationParser.java: ## @@ -0,0 +1,177 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070738898 ## core/src/test/java/org/apache/iceberg/view/TestViewRepresentationParser.java: ## @@ -0,0 +1,177 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [iceberg] hililiwei commented on pull request #6222: Flink: Support inspecting table

2023-01-15 Thread GitBox
hililiwei commented on PR #6222: URL: https://github.com/apache/iceberg/pull/6222#issuecomment-1383338686 thanks @stevenzwu @pvary @szehon-ho -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [iceberg] hililiwei commented on pull request #6401: Flink: Change to oldestAncestorAfter

2023-01-15 Thread GitBox
hililiwei commented on PR #6401: URL: https://github.com/apache/iceberg/pull/6401#issuecomment-1383339946 thanks @stevenzwu @rdblue @PraveenNanda124 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [iceberg] youngxinler commented on pull request #6554: Parquet: Improve Test Coverage of RowGroupFilter Code with Nans #6518

2023-01-15 Thread GitBox
youngxinler commented on PR #6554: URL: https://github.com/apache/iceberg/pull/6554#issuecomment-1383363142 @RussellSpitzer can you help with the review? About improve Test Coverage of RowGroupFilter Code with Nans. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070754570 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070758730 ## core/src/main/java/org/apache/iceberg/view/ViewRepresentationParser.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070762329 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070754570 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] ajantha-bhat commented on pull request #6575: Spark 3.3: support version travel by reference name

2023-01-15 Thread GitBox
ajantha-bhat commented on PR #6575: URL: https://github.com/apache/iceberg/pull/6575#issuecomment-1383390015 we also need to update this SQL syntax in https://github.com/apache/iceberg/blob/master/docs/spark-queries.md#sql -- This is an automated message from the Apache Git Service. To

[GitHub] [iceberg] ajantha-bhat opened a new pull request, #6601: Docs: Fix minor typo in time travel dataframe section

2023-01-15 Thread GitBox
ajantha-bhat opened a new pull request, #6601: URL: https://github.com/apache/iceberg/pull/6601 one of my comments was not addressed in #6573. Hence, a follow-up PR. #6573 adds two more spark read options in the data frame time travel syntax. -- This is an automated message from the Apa

[GitHub] [iceberg] ajantha-bhat commented on pull request #6601: Docs: Fix minor typo in time travel dataframe section

2023-01-15 Thread GitBox
ajantha-bhat commented on PR #6601: URL: https://github.com/apache/iceberg/pull/6601#issuecomment-1383394932 cc: @Fokko, @jackye1995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [iceberg] ajantha-bhat commented on pull request #6090: Core: Handle statistics file clean up from expireSnapshots

2023-01-15 Thread GitBox
ajantha-bhat commented on PR #6090: URL: https://github.com/apache/iceberg/pull/6090#issuecomment-1383414321 Just another reminder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [iceberg] jackye1995 commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
jackye1995 commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070842799 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default catal

[GitHub] [iceberg] pvary opened a new pull request, #6602: Flink: Backport: Flink: Refactor sink tests to use HadoopCatalogResource

2023-01-15 Thread GitBox
pvary opened a new pull request, #6602: URL: https://github.com/apache/iceberg/pull/6602 Backport of #6583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[GitHub] [iceberg] pvary commented on pull request #6602: Flink: Backport: Flink: Refactor sink tests to use HadoopCatalogResource

2023-01-15 Thread GitBox
pvary commented on PR #6602: URL: https://github.com/apache/iceberg/pull/6602#issuecomment-1383593672 - 1.15 is a clean backport - 1.14 the `tablePath` is a class variable, so it is not clean, but easy to follow the changes -- This is an automated message from the Apache Git Service. T

[GitHub] [iceberg] Fokko merged pull request #6601: Docs: Fix minor typo in time travel dataframe section

2023-01-15 Thread GitBox
Fokko merged PR #6601: URL: https://github.com/apache/iceberg/pull/6601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] Fokko merged pull request #6597: Build: Bump requests from 2.28.1 to 2.28.2 in /python

2023-01-15 Thread GitBox
Fokko merged PR #6597: URL: https://github.com/apache/iceberg/pull/6597 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] nastra commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-15 Thread GitBox
nastra commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070917805 ## core/src/main/java/org/apache/iceberg/view/SQLViewRepresentationParser.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] nastra commented on pull request #6357: Core: Remove TableOperations from metadata tables.

2023-01-15 Thread GitBox
nastra commented on PR #6357: URL: https://github.com/apache/iceberg/pull/6357#issuecomment-1383614825 @rdblue looks like this needs some RevAPI changes because the APIs changed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [iceberg] Fokko merged pull request #6596: Build: Bump coverage from 7.0.4 to 7.0.5 in /python

2023-01-16 Thread GitBox
Fokko merged PR #6596: URL: https://github.com/apache/iceberg/pull/6596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] Fokko merged pull request #6592: Build: Bump spotless-plugin-gradle from 6.12.1 to 6.13.0

2023-01-16 Thread GitBox
Fokko merged PR #6592: URL: https://github.com/apache/iceberg/pull/6592 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] Fokko merged pull request #6595: Build: Bump pytest from 7.2.0 to 7.2.1 in /python

2023-01-16 Thread GitBox
Fokko merged PR #6595: URL: https://github.com/apache/iceberg/pull/6595 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] nastra commented on a diff in pull request #6449: Delta: Support Snapshot Delta Lake Table to Iceberg Table

2023-01-16 Thread GitBox
nastra commented on code in PR #6449: URL: https://github.com/apache/iceberg/pull/6449#discussion_r1071029734 ## delta-lake/src/integration/java/org/apache/iceberg/delta/SparkDeltaLakeSnapshotTestBase.java: ## @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [iceberg] pvary commented on pull request #6602: Flink: Backport: Flink: Refactor sink tests to use HadoopCatalogResource

2023-01-16 Thread GitBox
pvary commented on PR #6602: URL: https://github.com/apache/iceberg/pull/6602#issuecomment-1383814310 The root cause of the difference of the backport is that #4699 is missing from the v1.14 code path. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [iceberg] Fokko merged pull request #6594: Build: Bump rich from 13.0.1 to 13.1.0 in /python

2023-01-16 Thread GitBox
Fokko merged PR #6594: URL: https://github.com/apache/iceberg/pull/6594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[GitHub] [iceberg] pvary opened a new pull request, #6603: Flink: Backport: Improve unit tests for sink

2023-01-16 Thread GitBox
pvary opened a new pull request, #6603: URL: https://github.com/apache/iceberg/pull/6603 Backport #4699 for v1.14 Flink code, so we can have clean backport for the testing infra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [iceberg] pvary commented on pull request #6602: Flink: Backport: Flink: Refactor sink tests to use HadoopCatalogResource

2023-01-16 Thread GitBox
pvary commented on PR #6602: URL: https://github.com/apache/iceberg/pull/6602#issuecomment-1383844429 Created #6603 for backporting #4699 to v1.14 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [iceberg] pvary commented on a diff in pull request #5544: Encryption integration and test

2023-01-16 Thread GitBox
pvary commented on code in PR #5544: URL: https://github.com/apache/iceberg/pull/5544#discussion_r1071094167 ## core/src/main/java/org/apache/iceberg/encryption/DefaultEncryptionManagerFactory.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [iceberg] pvary commented on a diff in pull request #5544: Encryption integration and test

2023-01-16 Thread GitBox
pvary commented on code in PR #5544: URL: https://github.com/apache/iceberg/pull/5544#discussion_r1071095748 ## core/src/main/java/org/apache/iceberg/encryption/DefaultEncryptionManagerFactory.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [iceberg] rbalamohan opened a new issue, #6604: Consider introducing repartitioning in MergeIntoIcebergTable

2023-01-16 Thread GitBox
rbalamohan opened a new issue, #6604: URL: https://github.com/apache/iceberg/issues/6604 ### Apache Iceberg version 0.14.0 ### Query engine Spark ### Please describe the bug 🐞 Without repartitioning, there is a chance of creating way too many files in merge

[GitHub] [iceberg] pvary commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
pvary commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071105915 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -139,6 +153,13 @@ private WriteBuilder(OutputFile file) { } else { this.conf = new

[GitHub] [iceberg] pvary commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
pvary commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071105915 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -139,6 +153,13 @@ private WriteBuilder(OutputFile file) { } else { this.conf = new

[GitHub] [iceberg] pvary commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
pvary commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071108598 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -231,6 +252,39 @@ private WriteBuilder createContextFunc( return this; } +privat

[GitHub] [iceberg] pvary commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
pvary commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071109266 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -231,6 +252,39 @@ private WriteBuilder createContextFunc( return this; } +privat

[GitHub] [iceberg] pvary commented on a diff in pull request #6584: Flink: support reading as Avro GenericRecord for FLIP-27 IcebergSource

2023-01-16 Thread GitBox
pvary commented on code in PR #6584: URL: https://github.com/apache/iceberg/pull/6584#discussion_r1071172080 ## flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/source/reader/ListDataIteratorBatcher.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [iceberg] ggershinsky commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
ggershinsky commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071228113 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -231,6 +252,39 @@ private WriteBuilder createContextFunc( return this; } +

[GitHub] [iceberg] RussellSpitzer commented on issue #6604: Consider introducing repartitioning in MergeIntoIcebergTable

2023-01-16 Thread GitBox
RussellSpitzer commented on issue #6604: URL: https://github.com/apache/iceberg/issues/6604#issuecomment-1384067709 This should be already covered with the write distribution mode code for Spark. We did mention in the last sync that we should default to a distribution rather than require us

[GitHub] [iceberg] nastra commented on issue #6579: Compaction: Abort the job smartly when partial commit starts to fail

2023-01-16 Thread GitBox
nastra commented on issue #6579: URL: https://github.com/apache/iceberg/issues/6579#issuecomment-1384084128 @qinghui-xu it seems reasonable to have different commit failure handling modes . Would you like to open a PR for that? -- This is an automated message from the Apache Git Servi

[GitHub] [iceberg] RussellSpitzer commented on issue #6579: Compaction: Abort the job smartly when partial commit starts to fail

2023-01-16 Thread GitBox
RussellSpitzer commented on issue #6579: URL: https://github.com/apache/iceberg/issues/6579#issuecomment-1384128689 Seems good to me, I would probably just stick to something like "number of allowed failures, or smart" as the two failure modes, although if we can figure out "smart" let's ju

[GitHub] [iceberg] rbalamohan commented on issue #6604: Consider introducing repartitioning in MergeIntoIcebergTable

2023-01-16 Thread GitBox
rbalamohan commented on issue #6604: URL: https://github.com/apache/iceberg/issues/6604#issuecomment-1384150789 >> This should be already covered with the write distribution mode code for Spark. Thanks @RussellSpitzer . Mind pointing to the name of the file/codebase for this? --

[GitHub] [iceberg] RussellSpitzer commented on issue #6604: Consider introducing repartitioning in MergeIntoIcebergTable

2023-01-16 Thread GitBox
RussellSpitzer commented on issue #6604: URL: https://github.com/apache/iceberg/issues/6604#issuecomment-1384165741 Example tests https://github.com/apache/iceberg/blob/dbb8a404f6632a55acb36e949f0e7b84b643cede/spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/TestSparkDistributionAn

[GitHub] [iceberg] kingeasternsun commented on pull request #3987: :bug: fix Flink Read support for parquet int96 timestamps

2023-01-16 Thread GitBox
kingeasternsun commented on PR #3987: URL: https://github.com/apache/iceberg/pull/3987#issuecomment-1384179727 Hi, @rdblue @openinx , Thanks for your advice, the conflict has been resolved. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [iceberg] pvary commented on pull request #6603: Flink: Backport: Improve unit tests for sink

2023-01-16 Thread GitBox
pvary commented on PR #6603: URL: https://github.com/apache/iceberg/pull/6603#issuecomment-1384186002 > Isn't this missing the updates for `TestFlinkIcebergSink` / `TestTaskWriters`? Good catch @nastra! I have missed the `TestTaskWriters`. The `TestFlinkIcebergSink` is inclu

[GitHub] [iceberg] findepi commented on pull request #2891: Fix -NaN ordering in spec

2023-01-16 Thread GitBox
findepi commented on PR #2891: URL: https://github.com/apache/iceberg/pull/2891#issuecomment-1384250410 > I think they should all just be grouped together yes, that's practical > regardless of if we strictly decide how they are ordered within that group. in Java it's har

[GitHub] [iceberg] rdblue commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
rdblue commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071414198 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -1000,6 +1064,19 @@ public ReadBuilder withNameMapping(NameMapping newNameMapping) { retu

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-16 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1071414863 ## core/src/main/java/org/apache/iceberg/view/SQLViewRepresentationParser.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [iceberg] deniskuzZ commented on a diff in pull request #6570: Hive: Use EnvironmentContext instead of Hive Locks to provide transactional commits after HIVE-26882

2023-01-16 Thread GitBox
deniskuzZ commented on code in PR #6570: URL: https://github.com/apache/iceberg/pull/6570#discussion_r1071409985 ## hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java: ## @@ -809,168 +603,38 @@ private static boolean hiveEngineEnabled(TableMetadata me

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-16 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1071424411 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-16 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070762329 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-16 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070762329 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] amogh-jahagirdar commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-16 Thread GitBox
amogh-jahagirdar commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1070698774 ## api/src/main/java/org/apache/iceberg/view/SQLViewRepresentation.java: ## @@ -36,17 +38,21 @@ default Type type() { String dialect(); /** The default

[GitHub] [iceberg] nastra commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-16 Thread GitBox
nastra commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1071446145 ## core/src/main/java/org/apache/iceberg/view/SQLViewRepresentationParser.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [iceberg] ggershinsky commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
ggershinsky commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071448461 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -1000,6 +1064,19 @@ public ReadBuilder withNameMapping(NameMapping newNameMapping) {

[GitHub] [iceberg] nastra commented on a diff in pull request #6598: Core: View representation core implementation

2023-01-16 Thread GitBox
nastra commented on code in PR #6598: URL: https://github.com/apache/iceberg/pull/6598#discussion_r1071448625 ## core/src/test/java/org/apache/iceberg/view/TestViewRepresentationParser.java: ## @@ -0,0 +1,163 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [iceberg] danielcweeks merged pull request #6563: Core: Align commit metric name

2023-01-16 Thread GitBox
danielcweeks merged PR #6563: URL: https://github.com/apache/iceberg/pull/6563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceber

[GitHub] [iceberg] rdblue commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
rdblue commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071456336 ## core/src/test/java/org/apache/iceberg/encryption/EncryptionTestUtils.java: ## @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

[GitHub] [iceberg] rdblue commented on a diff in pull request #2639: Parquet: Support parquet modular encryption

2023-01-16 Thread GitBox
rdblue commented on code in PR #2639: URL: https://github.com/apache/iceberg/pull/2639#discussion_r1071456708 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -1000,6 +1064,19 @@ public ReadBuilder withNameMapping(NameMapping newNameMapping) { retu

[GitHub] [iceberg] rdblue commented on a diff in pull request #6485: API: New KMS Client Interface

2023-01-16 Thread GitBox
rdblue commented on code in PR #6485: URL: https://github.com/apache/iceberg/pull/6485#discussion_r1071460077 ## api/src/main/java/org/apache/iceberg/encryption/KeyManagementClient.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

[GitHub] [iceberg] rdblue commented on a diff in pull request #6485: API: New KMS Client Interface

2023-01-16 Thread GitBox
rdblue commented on code in PR #6485: URL: https://github.com/apache/iceberg/pull/6485#discussion_r1071460718 ## api/src/main/java/org/apache/iceberg/encryption/KeyManagementClient.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

[GitHub] [iceberg] rdblue commented on a diff in pull request #6566: Python: Add visitor to DNF expr into Dask format

2023-01-16 Thread GitBox
rdblue commented on code in PR #6566: URL: https://github.com/apache/iceberg/pull/6566#discussion_r1071462286 ## python/pyiceberg/expressions/visitors.py: ## @@ -881,3 +881,82 @@ def rewrite_to_dnf(expr: BooleanExpression) -> Tuple[BooleanExpression, ...]: # (A AND NOT(B)

<    1   2   3   4   5   6   7   8   9   10   >