[I] Column statistics are not collected when the columns name is escaped [iceberg]

[via GitHub]

DimaRabkin opened a new issue, #11950:
URL: https://github.com/apache/iceberg/issues/11950

   ### Apache Iceberg version
   
   1.7.1 (latest release)
   
   ### Query engine
   
   None
   
   ### Please describe the bug 🐞
   
   When an Iceberg has many columns, it calculates statistics only up to 100 
columns (can be changed using config 
`write.metadata.metrics.max-inferred-column-defaults`).
   
   When a column name is escaped, there is a bug that it tries to use the 
escaped column name when calculating the statistics while the metrics mode was 
saved using the original column name.
   
   For example, I have a column `$event_time` which is escaped to 
`_x24event_time`.
   When building the metrics mode, it uses the original name in 
[MetricsConfig.java:138](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetricsConfig.java#L138):
   ![image 
(5)](https://github.com/user-attachments/assets/f731fd1b-7efd-4a6a-ac05-be1a2427c545)
   
   while when closing the parquet writer, it tries to get the metrics mode for 
`_x24event_time` in 
[MetricsUtil.java:118](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/MetricsUtil.java#L118):
   
   ![image 
(4)](https://github.com/user-attachments/assets/01d1fa48-e984-4d29-986b-8a35e1f0efcb)
   
   when it doesn't find the metrics mode, it uses the default which is `none` 
so the stats are not collected.
   
   ### Willingness to contribute
   
   - [ ] I can contribute a fix for this bug independently
   - [ ] I would be willing to contribute a fix for this bug with guidance from 
the Iceberg community
   - [ ] I cannot contribute a fix for this bug at this time


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org