Gabriel39 opened a new pull request, #65437:
URL: https://github.com/apache/doris/pull/65437
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
FileScannerV2 selection previously depended on
`SplitSourceConnector::all_scan_ranges_match`. Remote batch split sources
cannot enumerate all ranges up front, so the default predicate result forced
those queries back to FileScanner even when `enable_file_scanner_v2` was
enabled.
This change:
- Selects FileScannerV2 solely when `enable_file_scanner_v2` is set and the
scan is not a load.
- Validates every range after it is fetched, including ranges returned by
remote batch split sources, and returns `NotSupported` instead of falling back
when `FileScannerV2::is_supported` is false.
- Enables `enable_external_table_batch_mode` by default and marks the
session variable as fuzzy.
- Replaces the eager split-source matching test with unsupported-range error
coverage and adds FE coverage for the new session-variable defaults.
### Release note
External table batch mode is enabled by default. Queries with
`enable_file_scanner_v2` enabled now use FileScannerV2 for all split-source
types and fail when a range is unsupported instead of falling back to
FileScanner.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- Added BE coverage for unsupported FileScannerV2 ranges.
- Added FE coverage for the batch-mode default and fuzzy attribute.
- Test execution was attempted but blocked before compilation
because this worktree's `thirdparty/installed` lacks `protoc` and Snappy.
- [x] Manual test
- `PATH=/usr/local/opt/llvm@16/bin:$PATH
build-support/check-format.sh`
- [ ] No need to test or manual test.
- Behavior changed:
- [ ] No.
- [x] Yes. External table batch mode now defaults to enabled, and
enabled FileScannerV2 queries no longer fall back based on split-source
behavior.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
