Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1955144135 ## core/src/main/java/org/apache/iceberg/V2Metadata.java: ## @@ -278,37 +277,41 @@ static Types.StructType fileType(Types.StructType partitionType) { DataFil

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue merged PR #12060: URL: https://github.com/apache/iceberg/pull/12060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on PR #12060: URL: https://github.com/apache/iceberg/pull/12060#issuecomment-2657544978 Merging this now that the 1.8.0 vote has passed. Thanks for reviewing, everyone! -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953603432 ## core/src/main/java/org/apache/iceberg/V1Metadata.java: ## @@ -236,34 +235,38 @@ static Types.StructType dataFileSchema(Types.StructType partitionType) { }

Re: [PR] Core: Add InternalData read and write builders [iceberg]

pvary commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953323496 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953255779 ## core/src/main/java/org/apache/iceberg/V1Metadata.java: ## @@ -236,34 +235,38 @@ static Types.StructType dataFileSchema(Types.StructType partitionType) {

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953266565 ## core/src/main/java/org/apache/iceberg/V2Metadata.java: ## @@ -278,37 +277,41 @@ static Types.StructType fileType(Types.StructType partitionType) {

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953260076 ## core/src/main/java/org/apache/iceberg/V1Metadata.java: ## @@ -236,34 +235,38 @@ static Types.StructType dataFileSchema(Types.StructType partitionType) {

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953236460 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953228465 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1953002033 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,169 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Core: Add InternalData read and write builders [iceberg]

ajantha-bhat commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1952861075 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add InternalData read and write builders [iceberg]

pvary commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1952507195 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,169 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1938350796 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1938350909 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Core: Add InternalData read and write builders [iceberg]

pvary commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1934192589 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] Core: Add InternalData read and write builders [iceberg]

pvary commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1933980314 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contribu

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on PR #12060: URL: https://github.com/apache/iceberg/pull/12060#issuecomment-2610781637 > From a release prospective, should we merge this post 1.8? Just thinking we probably want it in the build for a bit before we ship it? I know that partition stats is downstream of this

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927537854 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927534205 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927532569 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -125,6 +126,18 @@ public class Parquet { private Parquet() {} + public static void regi

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927527955 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

Re: [PR] Core: Add InternalData read and write builders [iceberg]

danielcweeks commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927497489 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add InternalData read and write builders [iceberg]

danielcweeks commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927511341 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add InternalData read and write builders [iceberg]

danielcweeks commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927507516 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on PR #12060: URL: https://github.com/apache/iceberg/pull/12060#issuecomment-2610712280 From a release prospective, should we merge this post 1.8? Just thinking we probably want it in the build for a bit before we ship it? I know that partition stats is downstream o

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927476699 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927443891 ## core/src/main/java/org/apache/iceberg/V1Metadata.java: ## @@ -330,32 +333,44 @@ public ManifestEntry copyWithoutStats() { } } - static class Inde

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927434627 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927422910 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927427344 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927419737 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927414041 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

RussellSpitzer commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927397698 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] Core: Add InternalData read and write builders [iceberg]

danielcweeks commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927393190 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -125,6 +126,18 @@ public class Parquet { private Parquet() {} + public static voi

Re: [PR] Core: Add InternalData read and write builders [iceberg]

danielcweeks commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1927383162 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more c

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926214561 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -125,6 +126,18 @@ public class Parquet { private Parquet() {} + public void register()

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926214945 ## parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java: ## @@ -1171,6 +1188,16 @@ public ReadBuilder withNameMapping(NameMapping newNameMapping) { re

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926214256 ## core/src/main/java/org/apache/iceberg/avro/SupportsCustomRecords.java: ## @@ -20,7 +20,7 @@ import java.util.Map; -/** An interface for Avro DatumReaders to su

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926212266 ## core/src/main/java/org/apache/iceberg/avro/InternalReader.java: ## @@ -76,6 +76,15 @@ public void setSchema(Schema schema) { initReader(); } + @Override

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926211300 ## core/src/main/java/org/apache/iceberg/avro/Avro.java: ## @@ -90,14 +103,18 @@ private enum Codec { } public static WriteBuilder write(OutputFile file) { +

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926210150 ## core/src/main/java/org/apache/iceberg/V1Metadata.java: ## @@ -330,32 +333,44 @@ public ManifestEntry copyWithoutStats() { } } - static class IndexedDataF

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926209609 ## core/src/main/java/org/apache/iceberg/V1Metadata.java: ## @@ -64,17 +59,21 @@ public ManifestFile wrap(ManifestFile file) { } @Override -public org.

Re: [PR] Core: Add InternalData read and write builders [iceberg]

rdblue commented on code in PR #12060: URL: https://github.com/apache/iceberg/pull/12060#discussion_r1926206956 ## core/src/main/java/org/apache/iceberg/InternalData.java: ## @@ -0,0 +1,159 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contrib

[PR] Core: Add InternalData read and write builders [iceberg]

rdblue opened a new pull request, #12060: URL: https://github.com/apache/iceberg/pull/12060 This adds `InternalData` with read and write builder interfaces that can be used with Avro and Parquet by passing a `FileFormat`. Formats are registered by calling `InternalData.register` with callba