CTTY commented on code in PR #2868:
URL: https://github.com/apache/iceberg-rust/pull/2868#discussion_r3847397518
##########
crates/iceberg/src/scan/task.rs:
##########
@@ -239,6 +242,27 @@ pub struct FileScanTaskDeleteFile {
#[builder(default)]
pub equality_ids: Option<Vec<i32>>,
+ /// For a deletion vector, the location of the data file whose rows it
deletes. Required for
+ /// deletion vectors, and may also be set on a position delete file scoped
to one data file.
+ #[serde(default)]
+ #[serde(skip_serializing_if = "Option::is_none")]
+ #[builder(default)]
+ pub referenced_data_file: Option<String>,
+
+ /// For a deletion vector, the offset of the blob within its Puffin file.
Set only for
+ /// deletion vectors, where it locates the blob for direct access.
+ #[serde(default)]
+ #[serde(skip_serializing_if = "Option::is_none")]
+ #[builder(default)]
+ pub content_offset: Option<i64>,
+
+ /// For a deletion vector, the length in bytes of the blob within its
Puffin file. Set
+ /// whenever `content_offset` is.
+ #[serde(default)]
+ #[serde(skip_serializing_if = "Option::is_none")]
+ #[builder(default)]
+ pub content_size_in_bytes: Option<i64>,
+
Review Comment:
In #3035 I noticed we are also trying to add record_count for cardinality
check. I'm leaning toward adding it here
##########
crates/iceberg/src/scan/task.rs:
##########
@@ -239,6 +242,27 @@ pub struct FileScanTaskDeleteFile {
#[builder(default)]
pub equality_ids: Option<Vec<i32>>,
+ /// For a deletion vector, the location of the data file whose rows it
deletes. Required for
+ /// deletion vectors, and may also be set on a position delete file scoped
to one data file.
+ #[serde(default)]
+ #[serde(skip_serializing_if = "Option::is_none")]
+ #[builder(default)]
+ pub referenced_data_file: Option<String>,
+
+ /// For a deletion vector, the offset of the blob within its Puffin file.
Set only for
+ /// deletion vectors, where it locates the blob for direct access.
+ #[serde(default)]
+ #[serde(skip_serializing_if = "Option::is_none")]
+ #[builder(default)]
+ pub content_offset: Option<i64>,
+
+ /// For a deletion vector, the length in bytes of the blob within its
Puffin file. Set
Review Comment:
this is a bit unclear, I'd recommend
```suggestion
/// For a deletion vector, the length in bytes of the blob within its
Puffin file.
/// Required together with `content_offset`; both are absent for non-DV
delete files.
```
also I noticed in #3035 , we are checking if a file is DV by checking `if
content_offset.is_some()`. I believe instead of that, we should check `isDv =
(if type == PosDel && format == puffin)`. and we should throw error `if (isDV
&& (content_offset.is_none() || content_size_in_bytes.is_none()))` because both
of them are required for DVs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
