Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-14 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1800532477 ## hive-metastore/src/test/java/org/apache/iceberg/hive/HiveTableTest.java: ## @@ -213,7 +213,7 @@ public void testDropTable() throws IOException { table.newAppen

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-14 Thread via GitHub
aokolnychyi commented on PR #11158: URL: https://github.com/apache/iceberg/pull/11158#issuecomment-2411784644 Thanks, @nastra! Thanks for reviewing, @amogh-jahagirdar! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-14 Thread via GitHub
aokolnychyi merged PR #11158: URL: https://github.com/apache/iceberg/pull/11158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@icebe

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-14 Thread via GitHub
aokolnychyi commented on PR #11158: URL: https://github.com/apache/iceberg/pull/11158#issuecomment-2411782938 Actually, the comments are minor and I verified that changes in tests are cosmetic locally. I'll go ahead and merge this one to unblock the subsequent PR. -- This is an automated

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-14 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1799824634 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -71,9 +70,9 @@ public String partition() { private final PartitionSet deleteFilePart

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on PR #11158: URL: https://github.com/apache/iceberg/pull/11158#issuecomment-2402473805 > Discussed with @nastra offline, overall feel like the change is good but I think it's worth running some benchmarks as a sanity check. There really shouldn't be much of a change afterw

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793489743 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -215,7 +213,7 @@ private List writeNewManifests() throws IOException { } if (newManifests

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793462877 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -533,4 +531,51 @@ private Pair metricsEvaluator return metricsEvaluators.get(partitio

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793418362 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -71,6 +72,7 @@ public String partition() { private final PartitionSet deleteFilePartition

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793359976 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -372,8 +367,14 @@ private boolean manifestHasDeletedFiles( for (ManifestEntry entry :

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793354776 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -230,27 +232,20 @@ SnapshotSummary.Builder buildSummary(Iterable manifests) { * * @

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793063470 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -71,6 +72,7 @@ public String partition() { private final PartitionSet deleteFilePartition

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793063470 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -71,6 +72,7 @@ public String partition() { private final PartitionSet deleteFilePartition

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793058000 ## core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java: ## @@ -81,8 +83,8 @@ abstract class MergingSnapshotProducer extends SnapshotProducer { //

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793049678 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -71,6 +72,7 @@ public String partition() { private final PartitionSet deleteFilePartition

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-09 Thread via GitHub
nastra commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1793045220 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -372,8 +367,14 @@ private boolean manifestHasDeletedFiles( for (ManifestEntry entry :

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792837752 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -421,7 +421,7 @@ private ManifestFile filterManifestWithDeletedFiles(

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792838808 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -230,27 +232,20 @@ SnapshotSummary.Builder buildSummary(Iterable manifests) { *

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792837752 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -421,7 +421,7 @@ private ManifestFile filterManifestWithDeletedFiles(

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792836843 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -372,8 +367,14 @@ private boolean manifestHasDeletedFiles( for (ManifestEntry en

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792836843 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -372,8 +367,14 @@ private boolean manifestHasDeletedFiles( for (ManifestEntry en

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792834646 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -71,6 +72,7 @@ public String partition() { private final PartitionSet deleteFilePart

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792834057 ## core/src/main/java/org/apache/iceberg/ManifestFilterManager.java: ## @@ -71,6 +72,7 @@ public String partition() { private final PartitionSet deleteFilePart

Re: [PR] Core: Switch usage to DataFileSet / DeleteFileSet [iceberg]

2024-10-08 Thread via GitHub
aokolnychyi commented on code in PR #11158: URL: https://github.com/apache/iceberg/pull/11158#discussion_r1792639080 ## core/src/main/java/org/apache/iceberg/FastAppend.java: ## @@ -215,7 +213,7 @@ private List writeNewManifests() throws IOException { } if (newManif