rdblue commented on code in PR #17159:
URL: https://github.com/apache/iceberg/pull/17159#discussion_r3581671409
##########
core/src/main/java/org/apache/iceberg/FieldStats.java:
##########
@@ -46,17 +44,20 @@ interface FieldStats<T> extends StructLike {
boolean tightBounds();
/** The total value count, including null and NaN */
- Long valueCount();
+ long valueCount();
/** The total null value count */
- Long nullValueCount();
+ long nullValueCount();
/** The total NaN value count */
- Long nanValueCount();
+ long nanValueCount();
Review Comment:
I think these accessors should reflect the API that we want for building on
top of stats. So the decision should be made when we introduce classes that
consume these.
I think that primitives are more locked down. They will throw a
`NullPointerException` if called when there is no value, which is good for
catching calls that should not be made. For example, `IS NaN` and `IS NOT NaN`
predicates cannot be bound to fields that are not floating points (throws
`ValidationException`) so accessing the NaN count should not happen if the
field is missing from the stats struct.
On the other hand, these values may just be missing for a specific data file
because they weren't collected. In this case, we may want to signal that the
value is missing by returning the underlying null. This is why I left the
`avg_value_size_in_bytes` column as an `Integer`.
If there is a case where we know we always have a value, then it makes sense
to use the primitive. But it may also be better to have a method to check
whether the value is known and still use a primitive when we may not have a
value. I'm undecided at the moment and I'm waiting for actual uses. I'm also
currently writing the evaluator and, while it isn't done, I think we may want
to add more methods and keep the types more restrictive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
