zhongyujiang opened a new pull request, #6431: URL: https://github.com/apache/iceberg/pull/6431
This PR fixs ParquetDictionaryRowGroupFilter evaluating `notNaN`. Because Parquet dictionaries cannot contain null values, ParquetDictionaryRowGroupFilter should check if there is null values in the column chunk when evaluting `notNaN` and return `true` if yes. This also improves looking up `NaN` value in the dictionary set, in Java, both Double.NaN and Float.NaN are considered to be equal to themselves, so: - the dict must contain values that are not `NaN` when its' size is greater than 1, can directly return `true` in this case; - can use `Set#contains` to look up `NaN` directly instead of comparing elements in the collection one by one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
