kevinjqliu commented on issue #601: URL: https://github.com/apache/iceberg-python/issues/601#issuecomment-2058220348
Looks like your schema is nested, which makes things more complicated. It's pretty difficult to deal with merging nested schemas. I'm not sure if there's an out-of-the-box solution for this. One possible solution could be to use pandas (or another engine) to merge the data once it is read into memory. Then use the union schema as the Iceberg table's schema. Another solution can be to read into memory, flatten the schemas, and then write to Iceberg. That said, most of the difficulties here are not related to Iceberg. One thing I wonder is if PyIceberg can handle schema evolution of nested structs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
