viirya commented on code in PR #2710:
URL: https://github.com/apache/iceberg-rust/pull/2710#discussion_r4419104878
##########
crates/iceberg/src/arrow/schema.rs:
##########
@@ -890,7 +890,8 @@ impl SchemaVisitor for ToArrowSchemaConverter {
.map(DataType::FixedSizeBinary)
.unwrap_or(DataType::LargeBinary),
)),
- PrimitiveType::Binary | PrimitiveType::Geometry(_) |
PrimitiveType::Geography(_) => {
+ PrimitiveType::Binary =>
Ok(ArrowSchemaOrFieldOrType::Type(DataType::Binary)),
Review Comment:
Good catch, confirmed with a test: before the fix, a batch with a
`LargeBinaryArray` equality column failed with `expected Binary but found
LargeBinary`. Fixed in 203bd7058: `RecordBatchProjector::project_batch` now
casts `LargeBinary` columns to `Binary` when the projected field is `Binary`
(other mismatches still error, and the cast errors rather than truncating if
the data exceeds 32-bit offsets). Added
`test_equality_delete_accepts_large_binary_input`.
I also checked the other paths that see the new `Binary` schema:
`ParquetWriter` still accepts `LargeBinary` input for a `Binary` column, and
the partition splitter/value calculator only use `project_column`, so they see
the same arrays as before.
##########
crates/iceberg/src/writer/base_writer/equality_delete_writer.rs:
##########
@@ -763,10 +763,10 @@ mod test {
)
.unwrap(),
) as ArrayRef;
- let col13 = Arc::new(arrow_array::LargeBinaryArray::from_opt_vec(vec![
- Some(b"one"),
- Some(b""),
- Some(b"zzzz"),
+ let col13 = Arc::new(arrow_array::BinaryArray::from_opt_vec(vec![
+ Some(b"one".as_slice()),
+ Some(b"".as_slice()),
+ Some(b"zzzz".as_slice()),
Review Comment:
Dropped the `.as_slice()` calls in both test fixtures in 203bd7058.
`from_opt_vec` infers `&[u8]` without them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
