yogevyuval commented on PR #2004: URL: https://github.com/apache/iceberg-python/pull/2004#issuecomment-3342035232
For documentation - I tested out 0.7rc and in a manifest with 50K entries the new rust version in python took more than 2X than the original version. I'm not sure if that's because a different parsing strategy (laziness) as @Fokko suggested, or something regarding objects transefrring from rust to python, but I think we should investigate before merging, as this code path is already one of the slowest in pyiceberg -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
