CTTY commented on code in PR #1657:
URL: https://github.com/apache/iceberg-rust/pull/1657#discussion_r2334450044
##########
crates/iceberg/src/writer/mod.rs:
##########
@@ -229,20 +229,30 @@ pub mod file_writer;
use arrow_array::RecordBatch;
use crate::Result;
-use crate::spec::DataFile;
+use crate::io::OutputFile;
+use crate::spec::{DataFile, PartitionKey};
type DefaultInput = RecordBatch;
type DefaultOutput = Vec<DataFile>;
+/// The partitioning writer used to write data to multiple partitions.
+pub trait PartitioningWriter {
+ /// Write a record batch, all rows from this record batch should come from
one partition
+ fn write(&mut self, partition_key: PartitionKey, batch: RecordBatch) ->
Result<()>;
+
+ /// Close all writers and return the data files.
+ fn close(&mut self) -> Result<Vec<DataFile>>;
+}
+
/// The builder for iceberg writer.
#[async_trait::async_trait]
pub trait IcebergWriterBuilder<I = DefaultInput, O = DefaultOutput>:
Review Comment:
I believe we will also need to change the DefaultOutput for `IcebergWriter`
from `Vec<DataFile>` to `Vec<DataFileBuilder>` since `IcebergWriter` is no
longer the outermost writer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
