qweek commented on PR #58:
URL:
https://github.com/apache/maven-build-cache-extension/pull/58#issuecomment-1503243437
I have a couple more comments to your benchmark.
1. Throughput benchmark does not show you real picture, because it depends
on IO performance and total load on the disk. This means that results will most
likely contain outliers, and averaged throughput will show the wrong value. It
is better in this case to use Mode.SampleTime with percentiles.
2. Benchmark results also depends on file size, disk cache, etc. Your
benchmark includes have small test suite of 149 java files with total size less
than 1 MB.
I ran similar benchmark on Linux with random generated files 128B .. 128KB,
16 files for each size (total size ~4 MB).
```
Benchmark Mode Cnt Score Error Units
PerfTest.XX sample 3232 2.783 ± 0.016 ms/op
PerfTest.XX:XX·p0.00 sample 2.49 ms/op
PerfTest.XX:XX·p0.50 sample 2.793 ms/op
PerfTest.XX:XX·p0.90 sample 2.99 ms/op
PerfTest.XX:XX·p0.95 sample 3.127 ms/op
PerfTest.XX:XX·p0.99 sample 3.502 ms/op
PerfTest.XX:XX·p0.999 sample 5.278 ms/op
PerfTest.XX:XX·p0.9999 sample 5.833 ms/op
PerfTest.XX:XX·p1.00 sample 5.833 ms/op
Benchmark Mode Cnt Score Error Units
PerfTest.XXMM sample 2701 3.332 ± 0.014 ms/op
PerfTest.XXMM:XXMM·p0.00 sample 3.211 ms/op
PerfTest.XXMM:XXMM·p0.50 sample 3.297 ms/op
PerfTest.XXMM:XXMM·p0.90 sample 3.457 ms/op
PerfTest.XXMM:XXMM·p0.95 sample 3.502 ms/op
PerfTest.XXMM:XXMM·p0.99 sample 3.715 ms/op
PerfTest.XXMM:XXMM·p0.999 sample 6.555 ms/op
PerfTest.XXMM:XXMM·p0.9999 sample 11.682 ms/op
PerfTest.XXMM:XXMM·p1.00 sample 11.682 ms/op
```
In this benchmark XX with Memory Mapped file was **6 - 29% slower**
With random generated files 128KB .. 128MB, 16 files for each size (total
size ~4 GB, which is more realistic suite in our case)
```
Benchmark Mode Cnt Score Error Units
PerfTest.XX sample 3 3836.39 ± 309.251 ms/op
PerfTest.XX:XX·p0.00 sample 3821.011 ms/op
PerfTest.XX:XX·p0.50 sample 3833.594 ms/op
PerfTest.XX:XX·p0.90 sample 3854.565 ms/op
PerfTest.XX:XX·p0.95 sample 3854.565 ms/op
PerfTest.XX:XX·p0.99 sample 3854.565 ms/op
PerfTest.XX:XX·p0.999 sample 3854.565 ms/op
PerfTest.XX:XX·p0.9999 sample 3854.565 ms/op
PerfTest.XX:XX·p1.00 sample 3854.565 ms/op
Benchmark Mode Cnt Score Error Units
PerfTest.XXMM sample 7 1523.132 ± 233.943 ms/op
PerfTest.XXMM:XXMM·p0.00 sample 1413.48 ms/op
PerfTest.XXMM:XXMM·p0.50 sample 1533.018 ms/op
PerfTest.XXMM:XXMM·p0.90 sample 1663.042 ms/op
PerfTest.XXMM:XXMM·p0.95 sample 1663.042 ms/op
PerfTest.XXMM:XXMM·p0.99 sample 1663.042 ms/op
PerfTest.XXMM:XXMM·p0.999 sample 1663.042 ms/op
PerfTest.XXMM:XXMM·p0.9999 sample 1663.042 ms/op
PerfTest.XXMM:XXMM·p1.00 sample 1663.042 ms/op
```
In this benchmark XX with Memory Mapped file was **132 - 170% faster**
You can adjust benchmark according to your load, but main problem is that
test run takes a lot of time
```
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.SecureRandom;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
public static class HashState {
private static final String USER_DIRECTORY =
System.getProperty("user.dir");
private static List<Path> paths;
@Setup(Level.Iteration)
public void setup() {
paths = new ArrayList<>();
startFileSize = Integer.parseInt(System.getProperty("from", "128"));
endFileSize = Integer.parseInt(System.getProperty("to", "131072"));
for (int fileSize = startFileSize; fileSize <= endFileSize; fileSize
*= 2) {
for (int i = 0; i < 16; i++) {
String fileName = fileSize + "B_" + UUID.randomUUID() +
".bin";
Path filePath = Paths.get(USER_DIRECTORY, "target",
fileName);
generateRandomBinaryFile(filePath, fileSize);
paths.add(filePath);
}
}
}
private static void generateRandomBinaryFile(Path path, int fileSize) {
SecureRandom random = new SecureRandom();
byte[] buffer = new byte[fileSize];
random.nextBytes(buffer);
try {
Files.createDirectories(path.getParent());
Files.write(path, buffer);
System.out.println("Random binary file generated successfully: "
+ path);
} catch (IOException e) {
System.err.println("Error while generating random binary file: "
+ e.getMessage());
}
}
@TearDown(Level.Iteration)
public void deleteAllPaths() {
for (Path path : paths) {
try {
Files.delete(path);
System.out.println("Deleted file: " + path);
} catch (IOException e) {
System.err.println("Error while deleting file: " + path + ",
" + e.getMessage());
}
}
}
}
```
code generated by GPT-4 ©
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
