This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 04ff5069bf7 [fix](filecache) fix benchmark varification problem
(#58845) (#61195)
04ff5069bf7 is described below
commit 04ff5069bf769c02bc2b7e9a4e902f7043014829
Author: zhengyu <[email protected]>
AuthorDate: Tue Mar 24 18:08:42 2026 +0800
[fix](filecache) fix benchmark varification problem (#58845) (#61195)
Cause of the the problem:
The generator reused one 1 MB buffer, but the writer batched up to 5 MB
before calling appendv, so every pending Slice pointed at the same
buffer that kept being overwritten; the uploaded data no longer matched
the key/offset tags, making DataVerifier fail.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---
be/src/io/tools/file_cache_microbench.cpp | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/be/src/io/tools/file_cache_microbench.cpp
b/be/src/io/tools/file_cache_microbench.cpp
index e924fb077e0..6d05755a335 100644
--- a/be/src/io/tools/file_cache_microbench.cpp
+++ b/be/src/io/tools/file_cache_microbench.cpp
@@ -1103,13 +1103,17 @@ private:
std::vector<doris::Slice> slices;
slices.reserve(4);
+ std::vector<std::string> slice_buffers;
+ slice_buffers.reserve(4);
size_t accumulated_size = 0;
// Stream data writing
while (data_generator.has_more()) {
doris::Slice chunk = data_generator.next_chunk(key);
- slices.push_back(chunk);
- accumulated_size += chunk.size;
+ slice_buffers.emplace_back(chunk.data, chunk.size);
+ const std::string& stored_chunk = slice_buffers.back();
+ slices.emplace_back(stored_chunk.data(),
stored_chunk.size());
+ accumulated_size += stored_chunk.size();
if (accumulated_size >= config.write_batch_size ||
!data_generator.has_more()) {
@@ -1120,6 +1124,7 @@ private:
status.to_string());
}
slices.clear();
+ slice_buffers.clear();
accumulated_size = 0;
}
}
@@ -1333,15 +1338,12 @@ private:
file_size = exist_job_perfile_size;
}
-// TODO(dengxin): fix verify
-#if 0
- // Verify read data
+ // Verify read data
if (!DataVerifier::verify_data(key, file_size,
read_offset, read_buffer,
read_length)) {
throw std::runtime_error("Data verification
failed for key: " +
key);
}
-#endif
LOG(INFO)
<< "read_offset=" << read_offset
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
