Gabriel39 commented on code in PR #67906:
URL: https://github.com/apache/doris/pull/67906#discussion_r3999245736
##########
be/src/exec/rowid_fetcher.cpp:
##########
@@ -609,6 +635,36 @@ Status RowIdStorageReader::submit_external_scan_tasks(
return scan_status.ok() ? Status::OK() : scan_status.status();
}
+TFileScanRangeParams RowIdStorageReader::build_external_scan_params(
+ const TFileScanRangeParams& source_params, const TFileRangeDesc& range,
+ const std::vector<SlotDescriptor>& scan_slots,
+ const std::vector<uint32_t>& scan_column_idxs) {
+ DORIS_CHECK(scan_slots.size() == scan_column_idxs.size());
+ auto params = source_params;
+ params.required_slots.clear();
+ params.column_idxs.clear();
+ params.slot_name_to_schema_pos.clear();
+ const std::set partition_names(range.columns_from_path_keys.begin(),
+ range.columns_from_path_keys.end());
+ for (size_t slot_idx = 0; slot_idx < scan_slots.size(); ++slot_idx) {
+ const auto& slot = scan_slots[slot_idx];
+ const auto column_idx = scan_column_idxs[slot_idx];
+ TFileScanSlotInfo slot_info;
+ slot_info.__set_slot_id(slot.id());
Review Comment:
Fixed in 32d20cb0d5. FE now retains the pinned full schema's non-regular
column categories in the scan parameters, including lazy columns absent from
phase one's tuple. BE restores those categories when rebuilding the fetch
projection and includes only REGULAR/GENERATED slots in column_idxs. An
explicitly empty category map preserves ordinary physical columns with
metadata-like names.
The fetch also preserves original Iceberg file paths and row-lineage
metadata when removing delete files, using a local range copy so shared
FileMapping entries are not modified.
Both new BE regressions failed before their respective fixes. Validation:
283 focused ASAN tests and 16 targeted FE tests passed; the FE run required a
temporary exclusion of an unrelated pre-existing IVM test compilation error.
Added eager-versus-lazy _file/_pos comparisons for Parquet and ORC, with a
materialization-plan assertion. External SQL validation is pending the newly
requested CI run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
