szehon-ho commented on code in PR #5376:
URL: https://github.com/apache/iceberg/pull/5376#discussion_r1015603424
##########
api/src/main/java/org/apache/iceberg/DataFile.java:
##########
@@ -99,10 +99,24 @@ public interface DataFile extends ContentFile<DataFile> {
optional(140, "sort_order_id", IntegerType.get(), "Sort order ID");
Types.NestedField SPEC_ID = optional(141, "spec_id", IntegerType.get(),
"Partition spec ID");
+ Types.StructType READABLE_METRICS_VALUE_TYPE =
+ Types.StructType.of(
+ optional(142, "column_size", Types.LongType.get(), "Total size on
disk"),
+ optional(143, "value_count", Types.LongType.get(), "Total count,
including null and NaN"),
+ optional(144, "null_value_count", Types.LongType.get(), "Null value
count"),
+ optional(145, "nan_value_count", Types.LongType.get(), "NaN value
count"),
+ optional(146, "lower_bound", Types.StringType.get(), "Lower bound in
string form"),
Review Comment:
I spent some time prototyping keeping the original struct (which I think you
also bringing up). That is a ton of work and complexity, in writing schema
utils to clone the schema for each struct. In the end I felt all that
complexity is not really worth it, a map is much simpler.
And it is more in line with what trino has (everything there is converted
to human readable string)
##########
api/src/main/java/org/apache/iceberg/DataFile.java:
##########
@@ -99,10 +99,24 @@ public interface DataFile extends ContentFile<DataFile> {
optional(140, "sort_order_id", IntegerType.get(), "Sort order ID");
Types.NestedField SPEC_ID = optional(141, "spec_id", IntegerType.get(),
"Partition spec ID");
+ Types.StructType READABLE_METRICS_VALUE_TYPE =
+ Types.StructType.of(
+ optional(142, "column_size", Types.LongType.get(), "Total size on
disk"),
+ optional(143, "value_count", Types.LongType.get(), "Total count,
including null and NaN"),
+ optional(144, "null_value_count", Types.LongType.get(), "Null value
count"),
+ optional(145, "nan_value_count", Types.LongType.get(), "NaN value
count"),
+ optional(146, "lower_bound", Types.StringType.get(), "Lower bound in
string form"),
Review Comment:
I spent some time prototyping keeping the original struct (which I think you
also bringing up). That is a ton of work and complexity, in writing schema
utils to clone the schema for each struct. In the end I felt all that
complexity is not really worth it, a map is much simpler.
And it is more in line with what trino has (everything there is converted
to readable string type)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
