ZENOTME commented on PR #731: URL: https://github.com/apache/iceberg-rust/pull/731#issuecomment-2545020315
> Thanks @ZENOTME 's effort. I saw that both java/python have schema with partner visitor: > > 1. [SchemaWithPartnerVisitor](https://github.com/apache/iceberg/blob/c07f2aabc0a1d02f068ecf1514d2479c0fbdd3b0/core/src/main/java/org/apache/iceberg/schema/SchemaWithPartnerVisitor.java#L27) > 2. https://github.com/apache/iceberg-python/blob/bfc0d9a62176803094da0867ee793808f105d352/pyiceberg/io/pyarrow.py#L1726 > > I'm thinking that we should adopt similar design? I tried to adopt the SchemaWithPartner design, but I find the interface is not so general. https://github.com/apache/iceberg-python/blob/bfc0d9a62176803094da0867ee793808f105d352/pyiceberg/io/pyarrow.py#L1726 is not for array convert I think. To make it suitable for arrow array convert, I make some change to it: 1. [Introduce ListIterator and MapIterator](https://github.com/apache/iceberg-rust/pull/731/files#diff-919de395f497e9e93101d26e0618400083c15c4eb7ac3679e3662107f0c4b0eeR1225), the control flow of list and map is not so suitable for arrow array convert. 2. array in arrow rust can be NullArray, so we need some preorder access before diving into the subcolumn. So I introduce `visit_type_before` to solve it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
