mengna-lin opened a new issue, #18235: URL: https://github.com/apache/iceberg/issues/18235
### Background Several table properties are keyed by **column name**, e.g.: - `write.parquet.compression-codec.column.<name>` / `write.parquet.compression-level.column.<name>` (added in #16094) - `write.parquet.bloom-filter-enabled.column.<name>`, `...bloom-filter-fpp.column.<name>`, `...bloom-filter-ndv.column.<name>` - `write.parquet.stats-enabled.column.<name>` - `write.parquet.dict-encoding-enabled.column.<name>` - `write.metadata.metrics.column.<name>` ### Problem Column names are not a stable identity; field IDs are. A `DROP COLUMN foo` followed by `ADD COLUMN foo` (or a rename) produces a new column that reuses the name but has a different field ID and possibly a different type, so a name-keyed hint can silently bind to the wrong column. This is hard for users to detect — the stale setting is invisible after the schema change, and the person who set the property may not be the one running the `ALTER`. This is not a read-correctness issue (Parquet codecs are stored per column chunk in the file); it affects write-time hint binding. ### Proposal Store these per-column properties **internally keyed by field ID** (the stable identity) while keeping the SQL / property interface accepting column names. Iceberg performs the name → field-ID translation on write and resolves by field ID thereafter. This should be tackled consistently across the whole `*.column.<name>` property family rather than for compression alone, to avoid one member of the family behaving differently from the rest. ### Context Follow-up from the discussion in #16094 (per-column Parquet compression), where the name-vs-field-ID keying tradeoff was raised. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
