mattfaltyn opened a new pull request, #2597:
URL: https://github.com/apache/iceberg-go/pull/2597

   Fixes #2596.
   
   ## Summary
   
   - Read null count and min/max presence from the Parquet footer. Arrow's 
typed statistics can represent an absent null count as zero and a single bound 
as a min/max pair.
   - Omit incomplete Iceberg file aggregates independently. Keep valid bounds 
across provably all-null row groups and continue recording complete column 
sizes and value counts.
   - Use the same footer presence checks for row-group pruning and shredded 
Variant bounds. When suppressing partial bounds removes the basis for partition 
inference, require an explicit partition value.
   
   ## Validation
   
   The regression imports a two-row-group Parquet file with an omitted null 
count, omitted min/max, or a one-sided bound. It checks persisted metrics and 
filtered scans, and verifies that a copy-on-write delete preserves a null row. 
Complete footer and all-null row-group cases cover valid pruning. Additional 
cases cover shredded Variant bounds and explicit partition values.
   
   When footer statistics are incomplete, observed bounds can still reject a 
contradictory supplied partition value. They are omitted from the Iceberg 
manifest and cannot be used to infer a partition. Existing manifest entries 
with incorrect metrics are outside this change.
   
   - `make test` passed (41 package suites).
   - `make lint` passed with zero findings.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to