guptaakashdeep commented on issue #1948: URL: https://github.com/apache/iceberg-python/issues/1948#issuecomment-2831943321
@scottjarman you can see all the metadata changes that has happened on a table using Iceberg metadata table called `metadata_log_entries`. In PyIceberg, you can see this table using `table.inspect.metadata_log_entries()` --> `latest_schema_id` shows the latest schema that will be active at the time of particular snapshot. On the other hand, are there any steps to replicate it? I tried replicating it locally but I am unable to, the only way i was able to do it is via reading from PyIceberg at one point and at the same time updating the schema from another engine like Apache Spark. But also, in this case, the metadata locations are different which actually makes sense. One thing you can cross check is that in the mentioned metadata.json file, what is the `current-schema-id` and does that `schema-id` has the schema change id -> id2. Looking at [glue.py](https://github.com/apache/iceberg-python/blob/be528ae2a936e9c4432b48ff3370a682cd9029f1/pyiceberg/catalog/glue.py#L347), implementation seems as expected and straightforward, where it's reading the file mentioned in the `metadata_location` returned from Glue API `get_table` call. If you can run a debug session with break point inside this method, we might figure out the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
