jhump commented on issue #386: URL: https://github.com/apache/iceberg-go/issues/386#issuecomment-2848117728
> If the above is compelling to continue with exposing this, it would be ideal if the properties could be extracted prior to reading the entries, mainly to avoid some I/O processing when it is determined that the file's schema ID means the processor can ignore all of its entries. I wrote this previously, but I actually need to take it back. This would mean to examine the schema ID _and also_ read the entries would incur two read operations to an object store API. An API that made it possible to read both with a single `io.Reader` would be grand (like maybe it returns the metadata like schema ID and some other object that can be used to read the entries). But if that seems inelegant or awkward to implement, then a way to get the schema ID at the same time as the entries is fine, too. Slight aside: I would also really love to see a top-level function that accepts an `io.Reader` for this, parallel to the existing `ReadManifestList`. I can file that as a separate issue if preferred. It feels missing, from a consistency point of view, but it would also be of value to me. (In the application I work on, we have a separate abstraction for accessing the object store -- with lots of bells and whistles for such as custom-configured HTTP client, metrics. and retries/hedging -- and it's much easier to use that and provide `io.Reader` values to the Iceberg library.) FWIW, I could also put up a PR for this if this is something you choose not to prioritize. (Thanks for the conversation and consideration!) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
