kevinjqliu commented on PR #2173:
URL: https://github.com/apache/iceberg-python/pull/2173#issuecomment-3042459341

   > Avro schema does not have a notion of field-id and element-id. I could add 
a helper function that would add these. I know this is not the core 
responsibility of this library. I was using this to create iceberg tables from 
Kafka topics, where the schema is stored in the schema registry. I think this 
is a viable use case and hence these helpers would add value.
   
   @matthias-Q im curious about the specific usecase. I think the field-id and 
element-id are already part of the avro schema. 
   
   According to the iceberg spec, https://iceberg.apache.org/spec/#avro under 
`Field IDs`
   ```
   Iceberg struct, list, and map types identify nested types by ID. When 
writing data to Avro files, these IDs must be stored in the Avro schema to 
support ID-based column pruning.
   ```
   
   also see 
https://github.com/apache/iceberg-python/blob/ecc52183f8160984c9a7cd446a3390405fd29647/tests/avro/test_reader.py#L262


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org

Reply via email to