randreucetti commented on code in PR #18434:
URL: https://github.com/apache/iceberg/pull/18434#discussion_r4245720520
##########
arrow/src/test/java/org/apache/iceberg/arrow/vectorized/TestVectorizedReaderBuilder.java:
##########
@@ -70,6 +72,41 @@ public void testVariantSkippedWhenNotInProjection() {
() -> TypeWithSchemaVisitor.visit(icebergSchema.asStruct(),
parquetSchema, builder));
}
+ @Test
+ public void testShreddedVariantSkippedWhenNotInProjection() {
+ Schema icebergSchema = new Schema(NestedField.required(1, "id",
IntegerType.get()));
+
+ MessageType parquetSchema = parquetSchemaWithShreddedVariant();
+
+ VectorizedReaderBuilder builder =
+ new VectorizedReaderBuilder(
+ icebergSchema, parquetSchema, false, ImmutableMap.of(), readers ->
null);
+
+ assertThatNoException()
+ .describedAs("Shredded variant not in projection should not throw")
+ .isThrownBy(
+ () -> TypeWithSchemaVisitor.visit(icebergSchema.asStruct(),
parquetSchema, builder));
+ }
+
+ @Test
+ public void testShreddedVariantInProjectionThrows() {
Review Comment:
Done. Renamed to `projectingShreddedVariantUnsupported`, and dropped the
`test` prefix on the other new tests too
(`shreddedVariantSkippedWhenNotInProjection`, and
`readShreddedWithoutProjectingVariant` in the Spark test).
##########
arrow/src/test/java/org/apache/iceberg/arrow/vectorized/TestVectorizedReaderBuilder.java:
##########
@@ -70,6 +72,41 @@ public void testVariantSkippedWhenNotInProjection() {
() -> TypeWithSchemaVisitor.visit(icebergSchema.asStruct(),
parquetSchema, builder));
}
+ @Test
+ public void testShreddedVariantSkippedWhenNotInProjection() {
Review Comment:
Added. `shreddedVariantSkippedWhenNotInProjection` is now parameterized over
top-level, struct field, list element, map key and map value. All five fail on
`main` without the fix and pass with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
