sollhui opened a new pull request, #69716:
URL: https://github.com/apache/doris/pull/69716
### What problem does this PR solve?
Issue Number: None
Related PR: #69333
Problem Summary:
After CSV ingestion switched to FileScannerV2, `DelimitedTextReader` opened
remote files directly through `FileFactory`, bypassing the sequential prefetch
used by the original CSV reader. S3 reads therefore block the scanner instead
of overlapping remote I/O with CSV parsing.
Restore `DelegateReader::create_file_reader(SEQUENTIAL)` in the shared
CSV/Text reader, passing the shared IOContext and the adjusted split range.
Unknown-size first splits use an end sentinel that the prefetch reader clamps
to the physical file size. Keep tracing outside the prefetch layer and preserve
the existing stream-load pipe path.
### Release note
Restore asynchronous S3 prefetch for FileScannerV2 CSV and text reads.
### Check List (For Author)
- Test
- [x] Unit Test added:
`CsvV2ReaderTest.S3PrefetchPreservesSplitRowsAndReadStatistics` covers
whole-file, mid-row split and unknown-size reads; checks row values, IO
statistics and background-thread S3 reads.
- Local runtime execution is **blocked**, not passed: `./run-be-ut.sh
--run --filter='CsvV2ReaderTest.*' -j 8` stops during JDK initialization with
`Failed setting boot class path`. Reproduced outside the sandbox. Runtime
validation is pending CI.
- Passed repository clang-format 16 formatting/check scripts,
`build-support/check-build-hygiene.sh`, and `git diff --check`.
- No post-fix throughput or same-host benchmark result is claimed.
- Behavior changed:
- [x] Yes. Eligible S3 CSV/Text readers use the existing sequential
prefetch implementation. No row parsing, schema or stream-pipe semantics change.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
