amogh-jahagirdar commented on code in PR #14153:
URL: https://github.com/apache/iceberg/pull/14153#discussion_r2372694582
##########
parquet/src/test/java/org/apache/iceberg/parquet/TestParquetDataWriter.java:
##########
@@ -113,13 +121,17 @@ public void testDataWriter() throws IOException {
List<Record> writtenRecords;
try (CloseableIterable<Record> reader =
Parquet.read(file.toInputFile())
- .project(SCHEMA)
- .createReaderFunc(fileSchema ->
GenericParquetReaders.buildReader(SCHEMA, fileSchema))
+ .project(schema)
+ .createReaderFunc(fileSchema ->
GenericParquetReaders.buildReader(schema, fileSchema))
.build()) {
writtenRecords = Lists.newArrayList(reader);
}
- assertThat(writtenRecords).as("Written records should
match").isEqualTo(records);
+ assertThat(writtenRecords).hasSameSizeAs(records);
+
+ for (int i = 0; i < records.size(); i++) {
+ InternalTestHelpers.assertEquals(schema.asStruct(), records.get(i),
writtenRecords.get(i));
Review Comment:
This change in the assertion looks good to me, it's the same approach for
how we validate variants are what we expect in `TestVariantWriters`
##########
parquet/src/test/java/org/apache/iceberg/parquet/TestParquetDataWriter.java:
##########
@@ -266,4 +278,40 @@ public void testInvalidUpperBoundBinary() throws Exception
{
assertThat(dataFile.lowerBounds()).as("Should have a valid lower
bound").containsKey(3);
assertThat(dataFile.upperBounds()).as("Should have a null upper
bound").doesNotContainKey(3);
}
+
+ @Test
+ public void testDataWriterWithVariantShredding() throws IOException {
+ Schema variantSchema =
+ new Schema(
+ ImmutableList.<Types.NestedField>builder()
+ .addAll(SCHEMA.columns())
+ .add(Types.NestedField.optional(4, "variant",
Types.VariantType.get()))
+ .build());
+
+ ByteBuffer testMetadataBuffer =
+ VariantTestUtil.createMetadata(ImmutableList.of("a", "b"), true);
+
+ ByteBuffer testObjectBuffer =
+ VariantTestUtil.createObject(
+ testMetadataBuffer,
+ ImmutableMap.of(
+ "a", Variants.of(123456789),
+ "b", Variants.of("string")));
+
+ Variant testVariant =
+ Variant.of(
+ Variants.metadata(testMetadataBuffer),
+ Variants.value(Variants.metadata(testMetadataBuffer),
testObjectBuffer));
Review Comment:
Minor: I don't think the `test` prefix in these variable names is
particularly useful since they already exist in the context of a test. I'd also
inline `testMetadataBuffer` instead of having a separate variable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
