Sl1mb0 commented on issue #778: URL: https://github.com/apache/iceberg-rust/issues/778#issuecomment-2539813154
> If I understand correctly if you could provide your own implementation of FileIO, would you be able to make it work? This would avoid the copy. Hmm - this may work, but it's hard to say given it's not clear what the `FileIO` trait would look like at the moment. > Iceberg works with absolute paths that are immutable after being written. I understand that, but in my opinion that is the responsibility of a 'higher-level' layer above building and serialization. Ideally you would have `*Builder`s for each type of metadata: `ManifestFile`, `ManifestList`, etc. You could then implement [`ToBytes`](https://shadow.github.io/docs/rust/num_traits/ops/bytes/trait.ToBytes.html) for each type, and _that_ would handle the serialization of those types. This would also allow users to either use `FileIO` out of the box _or_ provide their own implementations for "where the bytes go". I think you would need to be **_very_** clear for example, about the invariant that where a `ManifestFile` is written to **match** the [`manifest_path`](https://docs.rs/iceberg/latest/iceberg/spec/struct.ManifestFile.html#structfield.manifest_path) for that `ManifestFile`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
