Xuanwo commented on code in PR #176:
URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1470626858
##########
crates/iceberg/src/io.rs:
##########
@@ -278,7 +288,7 @@ impl OutputFile {
}
/// Creates output file for writing.
- pub async fn writer(&self) -> Result<impl FileWrite> {
+ pub async fn writer(&self) -> Result<Writer> {
Ok(self.op.writer(&self.path[self.relative_path_pos..]).await?)
Review Comment:
It's better to add a `buffer` (for example, 8MiB) for this writer to avoid
`EntiryTooSmall` error from s3.
And in the future, we can add a config for this to allow users to control
them.
---
There is another thing we can do is adding `concurrent` here so we can
upload parts concurrently. But we need to make a design on how users control
the concurrency. Maybe we can discuss them later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
