Dysprosium0626 commented on issue #341:
URL: https://github.com/apache/iceberg-rust/issues/341#issuecomment-2094225277

   Hi I nearly complete adding `EqualityDeleteWriter` but I encounter some 
problem.
   My impl is here: 
https://github.com/Dysprosium0626/iceberg-rust/blob/add_equality_delete_writer/crates/iceberg/src/writer/base_writer/equality_delete_writer.rs
   
   Basically, in my test case, I write some schema to build up a 
`ParquetWriterBuilder` and pass it into `EqualityDeleteFileWriterBuilder`. 
   ```rust
           // prepare writer
           let pb = ParquetWriterBuilder::new(
               WriterProperties::builder().build(),
               to_write.schema(),
               file_io.clone(),
               location_gen,
               file_name_gen,
           );
           let equality_ids = vec![1, 3];
           let mut equality_delete_writer = 
EqualityDeleteFileWriterBuilder::new(pb)
               .build(EqualityDeleteWriterConfig::new(
                   equality_ids,
                   schema.clone(),
                   PARQUET_FIELD_ID_META_KEY,
               ))
               .await?;
   ```
   The `FieldProjector` will filter columns in schema by the equality_ids and I 
tried to generate a delete_schema with fields after projection.
   ```rust
       async fn build(self, config: Self::C) -> Result<Self::R> {
           let (projector, fields) = FieldProjector::new(
               config.schema.fields(),
               &config.equality_ids,
               &config.column_id_meta_key,
           )?;
           let delete_schema = Arc::new(arrow_schema::Schema::new(fields));
           Ok(EqualityDeleteFileWriter {
               inner_writer: Some(self.inner.clone().build().await?),
               projector,
               delete_schema,
               equality_ids: config.equality_ids,
           })
       }
   
   ```
   **The problem is I cannot pass the delete_schema to 
`FileWriterBuilder`(`ParquetWriterBuilder` in this case), and the schema for 
inner writer is the old version(without projection), so the inner writer canno 
write file with properly.**
   Do you have any ideas? @ZENOTME 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org

Reply via email to