wgtmac opened a new issue, #332: URL: https://github.com/apache/iceberg-cpp/issues/332
The current Avro writer and reader implementations are not that efficient because we need to use `avro::GenericDatum` as the intermediate layer: - [Avro writer](https://github.com/apache/iceberg-cpp/blob/main/src/iceberg/avro/avro_writer.cc#L104): `ArrowArray` -> `avro::GenericDatum` -> `avro::Encoder` - [Avro reader](https://github.com/apache/iceberg-cpp/blob/main/src/iceberg/avro/avro_reader.cc#L134): `avro::Decoder` -> `avro::GenericDatum` -> `ArrowArray` It would be more efficient if we can get rid of conversion from/to `avro::GenericDatum` by directly using the encoder/decoder in the Avro writer/reader. It can greatly improve the performance of manifest file I/O. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
