szehon-ho commented on code in PR #6680:
URL: https://github.com/apache/iceberg/pull/6680#discussion_r1089385138
##########
core/src/main/java/org/apache/iceberg/ManifestFilterManager.java:
##########
@@ -510,16 +510,19 @@ private Pair<InclusiveMetricsEvaluator,
StrictMetricsEvaluator> metricsEvaluator
// in other words, ResidualEvaluator returns a part of the expression
that needs to be
// evaluated
// for rows in the given partition using metrics
- return metricsEvaluators.computeIfAbsent(
- file.partition(),
- partition -> {
- Expression residual = residualEvaluator.residualFor(partition);
- InclusiveMetricsEvaluator inclusive =
- new InclusiveMetricsEvaluator(tableSchema, residual,
caseSensitive);
- StrictMetricsEvaluator strict =
- new StrictMetricsEvaluator(tableSchema, residual,
caseSensitive);
- return Pair.of(inclusive, strict);
- });
+ PartitionData partition = (PartitionData) file.partition();
+ if (!metricsEvaluators.containsKey(partition)) {
Review Comment:
Will the fix still work if we just keep computeIfAbsent, but pass
file.partition().copy()?
##########
core/src/test/java/org/apache/iceberg/TestDeleteFiles.java:
##########
@@ -349,6 +352,54 @@ public void testDeleteFilesOnIndependentBranches() {
statuses(Status.EXISTING, Status.DELETED, Status.DELETED));
}
+ @Test
+ public void testDeleteWithCollision() {
+ Schema schema = new Schema(Types.NestedField.of(0, false, "x",
Types.StringType.get()));
+ PartitionSpec spec =
PartitionSpec.builderFor(schema).identity("x").build();
+ Table collisionTable =
+ TestTables.create(tableDir, "hashcollision", schema, spec,
formatVersion);
+
+ PartitionData partitionOne = new PartitionData(spec.partitionType());
+ partitionOne.set(0, "Aa");
+ PartitionData partitionTwo = new PartitionData(spec.partitionType());
+ partitionTwo.set(0, "BB");
+
+ Assert.assertEquals(
+
StructLikeWrapper.forType(spec.partitionType()).set(partitionOne).hashCode(),
+
StructLikeWrapper.forType(spec.partitionType()).set(partitionTwo).hashCode());
+
+ Metrics metrics = new Metrics(1L, null, null, null, null);
+
+ DataFile testFileOne =
+ DataFiles.builder(spec)
+ .withPartition(partitionOne)
+ .withPath("/g1.parquet")
+ .withFileSizeInBytes(100)
+ .withMetrics(metrics)
Review Comment:
Is metrics strictly required here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
