[GitHub] [iceberg] aokolnychyi closed pull request #6055: Spark 3.3: Use separate scan during file filtering in copy-on-write operations

aokolnychyi closed pull request #6055: Spark 3.3: Use separate scan during file filtering in copy-on-write operations URL: https://github.com/apache/iceberg/pull/6055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [iceberg] flyrain commented on a diff in pull request #5783: Build: Update Spark to 3.3.1

flyrain commented on code in PR #5783: URL: https://github.com/apache/iceberg/pull/5783#discussion_r1008594453 ## spark/v3.3/build.gradle: ## @@ -27,6 +27,16 @@ def sparkProjects = [ project(":iceberg-spark:iceberg-spark-runtime-${sparkMajorVersion}_${scalaVersion}"), ]

[GitHub] [iceberg] rbalamohan commented on issue #6044: Column pruning/projection is not happening in correlated queries (e.g Q94, Q16)

rbalamohan commented on issue #6044: URL: https://github.com/apache/iceberg/issues/6044#issuecomment-1295648072 It was Spark 3.2.x Prashant. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [iceberg] singhpk234 commented on issue #6044: Column pruning/projection is not happening in correlated queries (e.g Q94, Q16)

singhpk234 commented on issue #6044: URL: https://github.com/apache/iceberg/issues/6044#issuecomment-1295643951 Can you please also add spark version you are observing this ? I tried this with a sample UT below with spark 3.3 I could see schema pruning happening : UT :

[GitHub] [iceberg] aokolnychyi commented on pull request #5783: Build: Update Spark to 3.3.1

aokolnychyi commented on PR #5783: URL: https://github.com/apache/iceberg/pull/5783#issuecomment-1295614949 Thanks, @wangyum! Thanks for reviewing, @singhpk234! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [iceberg] aokolnychyi merged pull request #5783: Build: Update Spark to 3.3.1

aokolnychyi merged PR #5783: URL: https://github.com/apache/iceberg/pull/5783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #5632: Core: Avoid reading ManifestFile when create ManifestReader

szehon-ho commented on code in PR #5632: URL: https://github.com/apache/iceberg/pull/5632#discussion_r1008565020 ## core/src/main/java/org/apache/iceberg/ManifestReader.java: ## @@ -101,20 +100,32 @@ private String fileClass() { protected ManifestReader( InputFile fi

[GitHub] [iceberg] szehon-ho commented on a diff in pull request #4577: Fixes read metadata table failed due to illegal character

szehon-ho commented on code in PR #4577: URL: https://github.com/apache/iceberg/pull/4577#discussion_r1008532668 ## core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java: ## @@ -978,6 +1091,32 @@ private Set expectedManifestListPaths(Iterable snapshots, Long

[GitHub] [iceberg] jzhuge commented on pull request #4925: API: Add view interfaces

jzhuge commented on PR #4925: URL: https://github.com/apache/iceberg/pull/4925#issuecomment-1295507260 @rdblue @wmoustafa Could you take look at the PR again? Especially `ViewBuilder`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [iceberg] huaxingao commented on pull request #6065: Fix TestAggregateBinding

huaxingao commented on PR #6065: URL: https://github.com/apache/iceberg/pull/6065#issuecomment-1295322595 @rdblue Could you please take a look when you have a moment? Thanks a lot! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [iceberg] flyrain commented on pull request #6063: Spark 3.2: #6041 follow-up/cleanup

flyrain commented on PR #6063: URL: https://github.com/apache/iceberg/pull/6063#issuecomment-1295225620 Merged. Thanks @wypoon! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [iceberg] flyrain merged pull request #6063: Spark 3.2: #6041 follow-up/cleanup

flyrain merged PR #6063: URL: https://github.com/apache/iceberg/pull/6063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[GitHub] [iceberg] gaborkaszab commented on pull request #6074: API,Core: SnapshotManager to be created through Transaction

gaborkaszab commented on PR #6074: URL: https://github.com/apache/iceberg/pull/6074#issuecomment-1295175523 This fixes #5882 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [iceberg] gaborkaszab opened a new pull request, #6074: API,Core: SnapshotManager to be created through Transaction

gaborkaszab opened a new pull request, #6074: URL: https://github.com/apache/iceberg/pull/6074 Currently, SnapshotManager encapsulates its own BaseTransaction object and there is no way to expose it opposed to for example ExpireSnapshots that can be created through the Transaction API. As a

[GitHub] [iceberg] nastra commented on a diff in pull request #6073: Core: Pass purgeRequested flag to REST server

nastra commented on code in PR #6073: URL: https://github.com/apache/iceberg/pull/6073#discussion_r1008121297 ## core/src/test/java/org/apache/iceberg/rest/TestHTTPClient.java: ## @@ -219,7 +219,7 @@ private static Item doExecuteRequest( restClient.head(path, headers, o

[GitHub] [iceberg] nastra commented on a diff in pull request #6073: Core: Pass purgeRequested flag to REST server

nastra commented on code in PR #6073: URL: https://github.com/apache/iceberg/pull/6073#discussion_r1008118659 ## core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java: ## @@ -223,8 +223,20 @@ public boolean dropTable(SessionContext context, TableIdentifier identifi

[GitHub] [iceberg] nastra commented on a diff in pull request #6073: Core: Pass purgeRequested flag to REST server

nastra commented on code in PR #6073: URL: https://github.com/apache/iceberg/pull/6073#discussion_r1008117319 ## core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java: ## @@ -320,7 +321,10 @@ public T handleRequest( case DROP_TABLE: { - C

[GitHub] [iceberg] nastra commented on a diff in pull request #6073: Core: Pass purgeRequested flag to REST server

nastra commented on code in PR #6073: URL: https://github.com/apache/iceberg/pull/6073#discussion_r1008114969 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -222,8 +222,8 @@ public static LoadTableResponse createTable( throw new IllegalStateExcept

[GitHub] [iceberg] nastra commented on a diff in pull request #6073: Core: Pass purgeRequested flag to REST server

nastra commented on code in PR #6073: URL: https://github.com/apache/iceberg/pull/6073#discussion_r1008114969 ## core/src/main/java/org/apache/iceberg/rest/CatalogHandlers.java: ## @@ -222,8 +222,8 @@ public static LoadTableResponse createTable( throw new IllegalStateExcept

[GitHub] [iceberg] nastra opened a new pull request, #6073: Core: Pass purgeRequested flag to REST server

nastra opened a new pull request, #6073: URL: https://github.com/apache/iceberg/pull/6073 The motivation behind this change is that the REST server should know whether a `purge` was requested or not, rather than having the REST client throw an error. -- This is an automated message from

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #5632: Core: Avoid reading ManifestFile when create ManifestReader

ConeyLiu commented on code in PR #5632: URL: https://github.com/apache/iceberg/pull/5632#discussion_r1007807328 ## core/src/main/java/org/apache/iceberg/ManifestReader.java: ## @@ -126,15 +138,12 @@ protected ManifestReader( specId = Integer.parseInt(specProperty); }

[GitHub] [iceberg] nastra commented on a diff in pull request #6068: Docs: Fix link in the Java Custom Catalog page

nastra commented on code in PR #6068: URL: https://github.com/apache/iceberg/pull/6068#discussion_r1008039068 ## docs/java-custom-catalog.md: ## @@ -30,7 +30,7 @@ menu: It's possible to read an iceberg table either from an hdfs path or from a hive table. It's also possible to

[GitHub] [iceberg] Jonathan-Rosenberg commented on a diff in pull request #6068: Docs: Fix link in the Java Custom Catalog page

Jonathan-Rosenberg commented on code in PR #6068: URL: https://github.com/apache/iceberg/pull/6068#discussion_r1007977385 ## docs/java-custom-catalog.md: ## @@ -30,7 +30,7 @@ menu: It's possible to read an iceberg table either from an hdfs path or from a hive table. It's also

[GitHub] [iceberg] Jonathan-Rosenberg commented on a diff in pull request #6068: Docs: Fix link in the Java Custom Catalog page

Jonathan-Rosenberg commented on code in PR #6068: URL: https://github.com/apache/iceberg/pull/6068#discussion_r1007977385 ## docs/java-custom-catalog.md: ## @@ -30,7 +30,7 @@ menu: It's possible to read an iceberg table either from an hdfs path or from a hive table. It's also

[GitHub] [iceberg] nastra commented on a diff in pull request #6068: Docs: Fix link in the Java Custom Catalog page

nastra commented on code in PR #6068: URL: https://github.com/apache/iceberg/pull/6068#discussion_r1007974447 ## docs/java-custom-catalog.md: ## @@ -30,7 +30,7 @@ menu: It's possible to read an iceberg table either from an hdfs path or from a hive table. It's also possible to

[GitHub] [iceberg] Jonathan-Rosenberg commented on a diff in pull request #6068: Docs: Fix link in the Java Custom Catalog page

Jonathan-Rosenberg commented on code in PR #6068: URL: https://github.com/apache/iceberg/pull/6068#discussion_r1007972402 ## docs/java-custom-catalog.md: ## @@ -30,7 +30,7 @@ menu: It's possible to read an iceberg table either from an hdfs path or from a hive table. It's also

[GitHub] [iceberg] nastra commented on issue #6071: Should ClientPool consider UGI when reusing a connection?

nastra commented on issue #6071: URL: https://github.com/apache/iceberg/issues/6071#issuecomment-1294818143 @pvary is this something that you could potentially help answering? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [iceberg] ConeyLiu commented on issue #4576: Read metadata table failed due to illegal character

ConeyLiu commented on issue #4576: URL: https://github.com/apache/iceberg/issues/4576#issuecomment-1294801011 @rdblue @nastra Would you mind taking a look at this issue? This is true should be a bug. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #5659: Core: Support set system level properties with environmental variables

ConeyLiu commented on code in PR #5659: URL: https://github.com/apache/iceberg/pull/5659#discussion_r1007825452 ## core/src/main/java/org/apache/iceberg/SystemProperties.java: ## @@ -30,14 +32,23 @@ private SystemProperties() {} */ public static final String WORKER_THREAD

[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #5632: Core: Avoid reading ManifestFile when create ManifestReader

ConeyLiu commented on code in PR #5632: URL: https://github.com/apache/iceberg/pull/5632#discussion_r1007807328 ## core/src/main/java/org/apache/iceberg/ManifestReader.java: ## @@ -126,15 +138,12 @@ protected ManifestReader( specId = Integer.parseInt(specProperty); }