NoahKusaba opened a new issue, #3297: URL: https://github.com/apache/iceberg-rust/issues/3297
### Apache Iceberg Rust version main (8cb2adedd), 0.10.1 ### Describe the bug `iceberg::arrow::strip_metadata_from_schema` returns an error for any Arrow schema that contains a list or map column: ``` Unexpected => Field stack underflow in list ``` `MetadataStripVisitor` remembers each field's name and nullability by pushing onto `field_stack` in `before_field`, and pops it when rebuilding the field in `list`, `map`, `struct` and `primitive`. But `visit_list` and the map arm of `visit_type` call `before_list_element`, `before_map_key` and `before_map_value` for the inner fields, not `before_field`. The visitor doesn't override those hooks, so rebuilding a list element or a map key or value pops from an empty stack. The only test of this function is the doctest, which uses a flat schema, so this wasn't caught. Downstream, [datafusion-iceberg](https://github.com/apache/datafusion-iceberg) calls this function to validate INSERTs into partitioned tables, so an INSERT into a partitioned table with a list or map column fails (found in apache/datafusion-iceberg#19). ### To Reproduce ```rust use std::sync::Arc; use arrow_schema::{DataType, Field, Schema}; use iceberg::arrow::strip_metadata_from_schema; let schema = Schema::new(vec![Field::new( "l", DataType::List(Arc::new(Field::new("element", DataType::Int32, true))), true, )]); strip_metadata_from_schema(&schema).unwrap(); // Field stack underflow in list ``` ### Expected behavior It returns the same schema with metadata removed at every level, including list elements and map keys and values. ### Willingness to contribute I can contribute a fix for this bug independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
