Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on PR #12139: URL: https://github.com/apache/iceberg/pull/12139#issuecomment-2667157868 Thanks for reviewing, @aihuaxu, @danielcweeks, @RussellSpitzer, and @amogh-jahagirdar! -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue merged PR #12139: URL: https://github.com/apache/iceberg/pull/12139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960633603 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,424 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960605176 ## core/src/test/java/org/apache/iceberg/variants/VariantTestUtil.java: ## @@ -27,10 +29,55 @@ import java.util.stream.Collectors; import java.util.stream.Stream; i

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960603787 ## core/src/test/java/org/apache/iceberg/variants/VariantTestUtil.java: ## @@ -27,10 +29,55 @@ import java.util.stream.Collectors; import java.util.stream.Stream; i

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960602946 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantVisitor.java: ## @@ -0,0 +1,285 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960242711 ## parquet/src/test/java/org/apache/iceberg/parquet/TestVariantReaders.java: ## @@ -0,0 +1,1107 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one +

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1960241060 ## parquet/src/main/java/org/apache/iceberg/parquet/TypeWithSchemaVisitor.java: ## @@ -201,6 +211,17 @@ private static List visitFields( return results; } +

Re: [PR] Parquet: Implement Variant readers [iceberg]

aihuaxu commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1958863116 ## core/src/test/java/org/apache/iceberg/variants/VariantTestUtil.java: ## @@ -27,10 +29,55 @@ import java.util.stream.Collectors; import java.util.stream.Stream;

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943818949 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943809738 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943800768 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943546072 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943546072 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943539725 ## core/src/main/java/org/apache/iceberg/variants/Variants.java: ## @@ -209,59 +213,59 @@ public static VariantPrimitive ofNull() { return new PrimitiveWrapper<>(

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943536772 ## parquet/src/main/java/org/apache/iceberg/parquet/PruneColumns.java: ## @@ -162,9 +168,11 @@ private Integer getId(Type type) { return type.getId() == null ? nu

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1943535834 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941890941 ## api/src/main/java/org/apache/iceberg/types/GetProjectedIds.java: ## @@ -47,7 +47,9 @@ public Set struct(Types.StructType struct, List> fieldResu @Override

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941578309 ## parquet/src/main/java/org/apache/iceberg/parquet/TypeToMessageType.java: ## @@ -86,6 +86,9 @@ public Type field(NestedField field) { if (field.type().isP

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941577496 ## parquet/src/main/java/org/apache/iceberg/parquet/PruneColumns.java: ## @@ -162,9 +168,11 @@ private Integer getId(Type type) { return type.getId() == nul

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941571976 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941558670 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941507423 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941495914 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941495914 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941495914 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantReaders.java: ## @@ -0,0 +1,409 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] Parquet: Implement Variant readers [iceberg]

danielcweeks commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1941486767 ## api/src/main/java/org/apache/iceberg/types/GetProjectedIds.java: ## @@ -47,7 +47,9 @@ public Set struct(Types.StructType struct, List> fieldResu @Overri

Re: [PR] Parquet: Implement Variant readers [iceberg]

RussellSpitzer commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1938261512 ## core/src/main/java/org/apache/iceberg/variants/Variants.java: ## @@ -209,59 +213,59 @@ public static VariantPrimitive ofNull() { return new PrimitiveWr

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936493915 ## parquet/src/main/java/org/apache/iceberg/parquet/TypeWithSchemaVisitor.java: ## @@ -54,101 +57,108 @@ public static T visit( } else { // if not a primi

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936493090 ## parquet/src/main/java/org/apache/iceberg/parquet/PruneColumns.java: ## @@ -162,9 +168,11 @@ private Integer getId(Type type) { return type.getId() == null ? nu

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936492645 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetVariantVisitor.java: ## @@ -0,0 +1,287 @@ +/* + * + * * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936494392 ## parquet/src/main/java/org/apache/iceberg/parquet/TypeWithSchemaVisitor.java: ## @@ -217,11 +238,19 @@ public T map(Types.MapType iMap, GroupType map, T key, T valu

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936493604 ## parquet/src/main/java/org/apache/iceberg/parquet/TypeToMessageType.java: ## @@ -117,6 +120,14 @@ public GroupType map(MapType map, Type.Repetition repetition, int

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936491857 ## parquet/src/main/java/org/apache/iceberg/parquet/ParquetSchemaUtil.java: ## @@ -75,6 +76,32 @@ private static Schema convertInternal( converter.getAliases(

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936491220 ## api/src/main/java/org/apache/iceberg/types/GetProjectedIds.java: ## @@ -47,7 +47,7 @@ public Set struct(Types.StructType struct, List> fieldResu @Override

Re: [PR] Parquet: Implement Variant readers [iceberg]

rdblue commented on code in PR #12139: URL: https://github.com/apache/iceberg/pull/12139#discussion_r1936491473 ## core/src/main/java/org/apache/iceberg/variants/Variants.java: ## @@ -209,59 +213,59 @@ public static VariantPrimitive ofNull() { return new PrimitiveWrapper<>(