shangxinli commented on code in PR #942:
URL: https://github.com/apache/iceberg-cpp/pull/942#discussion_r4304041454
##########
src/iceberg/schema_util.cc:
##########
@@ -104,15 +104,22 @@ Result<FieldProjection> ProjectField(const SchemaField&
expected_field,
expected_field.field_id());
}
if (source_field.type()->type_id() == TypeId::kUnknown &&
- expected_field.type()->is_nested()) {
+ (expected_field.type()->is_nested() || mode == ProjectionMode::kWrite)) {
+ // The write path must materialize nulls with the expected type rather
than copy
+ // the source's unknown array into a differently typed column.
projection.kind = FieldProjection::Kind::kNull;
return projection;
}
if (expected_field.type()->is_nested()) {
ICEBERG_ASSIGN_OR_RAISE(
projection,
- ProjectNested(*expected_field.type(), *source_field.type(),
prune_source));
+ ProjectNested(*expected_field.type(), *source_field.type(),
prune_source, mode));
+ } else if (mode == ProjectionMode::kWrite) {
Review Comment:
The kWrite branch checks types but never compares source_field.optional()
against expected_field.optional(), so an optional input column projects onto a
required write column and its nulls get written straight into it. Java rejects
that at CheckCompatibility.java:184, "should be required, but is optional",
reached from TypeUtil.validateWriteSchema at TypeUtil.java:498.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
