shangxinli commented on code in PR #1043:
URL: https://github.com/apache/iceberg-cpp/pull/1043#discussion_r4328471692
##########
src/iceberg/arrow_c_data_util.cc:
##########
@@ -341,8 +341,14 @@ Result<ArrowArray> ProjectBatchNanoarrow(
internal::ArrowArrayViewGuard input_view_guard(&input_view);
ICEBERG_NANOARROW_RETURN_UNEXPECTED_WITH_ERROR(
ArrowArrayViewSetArray(&input_view, &input_batch, &error), error);
+ // FULL validation is required here: the append path below uses
+ // ArrowArrayViewGet{String,Bytes}Unsafe and raw list/map offset arithmetic,
+ // which nanoarrow only documents as memory-safe after FULL validation
+ // (DEFAULT does not check that every offsets-buffer entry is non-decreasing
+ // and in-bounds). This matches the FULL validation already performed on the
+ // equivalent untrusted-batch boundary in manifest_reader.cc.
ICEBERG_NANOARROW_RETURN_UNEXPECTED_WITH_ERROR(
- ArrowArrayViewValidate(&input_view, NANOARROW_VALIDATION_LEVEL_DEFAULT,
&error),
+ ArrowArrayViewValidate(&input_view, NANOARROW_VALIDATION_LEVEL_FULL,
&error),
Review Comment:
ProjectBatch runs per batch on the scan path via
file_scan_task_reader.cc:118, so FULL validation now walks every offsets buffer
on every batch. Unlike the manifest_reader case these come from our own readers
- is that cost intended?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
