uschindler commented on pull request #177:
URL: https://github.com/apache/lucene/pull/177#issuecomment-860641916
Hi,
I executed luceneutil bench using jdk-17-ea+b25:
```
TaskQPS baseline StdDevQPS my_modified_version
StdDev Pct diff p-value
BrowseMonthTaxoFacets 4136.71 (3.6%) 2684.53 (4.2%)
-35.1% ( -41% - -28%) 0.000
BrowseDateTaxoFacets 3842.73 (4.5%) 2579.73 (1.8%)
-32.9% ( -37% - -27%) 0.000
BrowseDayOfYearTaxoFacets 3269.90 (4.4%) 2296.66 (3.1%)
-29.8% ( -35% - -23%) 0.000
PKLookup 340.50 (3.6%) 305.61 (1.9%)
-10.2% ( -15% - -4%) 0.000
MedSloppyPhrase 750.49 (9.8%) 684.83 (8.5%)
-8.7% ( -24% - 10%) 0.003
HighSpanNear 758.20 (8.0%) 703.82 (11.1%)
-7.2% ( -24% - 12%) 0.019
HighIntervalsOrdered 932.88 (7.4%) 872.75 (9.9%)
-6.4% ( -22% - 11%) 0.019
Fuzzy1 453.74 (13.8%) 431.35 (11.2%)
-4.9% ( -26% - 23%) 0.214
HighSloppyPhrase 825.27 (11.2%) 784.94 (7.9%)
-4.9% ( -21% - 16%) 0.111
AndHighHigh 1069.65 (11.0%) 1028.11 (10.5%)
-3.9% ( -22% - 19%) 0.254
Respell 365.08 (10.8%) 352.91 (9.1%)
-3.3% ( -20% - 18%) 0.291
AndHighLow 5517.43 (6.7%) 5340.85 (7.9%)
-3.2% ( -16% - 12%) 0.168
MedSpanNear 1480.21 (9.8%) 1434.36 (6.9%)
-3.1% ( -18% - 15%) 0.249
LowPhrase 1073.64 (5.9%) 1042.16 (6.9%)
-2.9% ( -14% - 10%) 0.150
HighTermDayOfYearSort 4737.54 (5.7%) 4619.94 (3.7%)
-2.5% ( -11% - 7%) 0.102
LowSloppyPhrase 1402.06 (5.7%) 1367.43 (5.6%)
-2.5% ( -13% - 9%) 0.166
OrHighMed 894.25 (10.3%) 873.07 (9.5%)
-2.4% ( -20% - 19%) 0.450
Wildcard 576.83 (8.6%) 563.68 (6.4%)
-2.3% ( -15% - 13%) 0.343
Fuzzy2 59.21 (16.1%) 58.06 (18.1%)
-1.9% ( -31% - 38%) 0.719
HighPhrase 647.47 (8.7%) 635.63 (9.3%)
-1.8% ( -18% - 17%) 0.521
MedTerm 7892.79 (4.3%) 7751.00 (4.6%)
-1.8% ( -10% - 7%) 0.206
HighTerm 6971.95 (4.4%) 6867.18 (6.0%)
-1.5% ( -11% - 9%) 0.365
LowSpanNear 1689.37 (8.7%) 1664.10 (7.8%)
-1.5% ( -16% - 16%) 0.565
HighTermMonthSort 5044.03 (4.2%) 4970.03 (6.6%)
-1.5% ( -11% - 9%) 0.399
IntNRQ 1565.30 (5.4%) 1549.65 (2.4%)
-1.0% ( -8% - 7%) 0.451
MedPhrase 890.99 (9.5%) 884.98 (10.3%)
-0.7% ( -18% - 21%) 0.830
Prefix3 585.76 (16.3%) 588.03 (18.8%)
0.4% ( -29% - 42%) 0.944
AndHighMed 1946.50 (8.1%) 1954.42 (8.3%)
0.4% ( -14% - 18%) 0.875
BrowseDayOfYearSSDVFacets 2349.67 (2.7%) 2360.57 (2.7%)
0.5% ( -4% - 5%) 0.583
OrHighHigh 402.33 (16.4%) 404.38 (13.1%)
0.5% ( -24% - 35%) 0.913
OrHighLow 1760.39 (11.1%) 1770.64 (8.4%)
0.6% ( -17% - 22%) 0.852
LowTerm 8072.83 (4.7%) 8140.49 (5.0%)
0.8% ( -8% - 11%) 0.586
BrowseMonthSSDVFacets 2949.63 (6.6%) 3015.76 (3.2%)
2.2% ( -7% - 12%) 0.170
```
For those interested:
```
JAVA:
WARNING: Using incubator modules: jdk.incubator.foreign
openjdk version "17-ea" 2021-09-14
OpenJDK Runtime Environment (build 17-ea+25-2252)
OpenJDK 64-Bit Server VM (build 17-ea+25-2252, mixed mode, sharing)
OS:
Linux serv1.sd-datasolutions.de 5.8.0-50-generic #56~20.04.1-Ubuntu SMP Mon
Apr 12 21:46:35 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
LOGS:
/home/thetaphi/benchmark/logs
```
My localconstants.py, so that's how I added the module command line params.
@mikemccand It would be better to have the tool respect `RUNTIME_JAVA_HOME`
like gradlew does.
```python
BASE_DIR = '/home/thetaphi/benchmark'
BENCH_BASE_DIR = '/home/thetaphi/benchmark/util'
JAVA_HOME = '/home/jenkins/tools/java/64bit/latest-jdk17'
JAVA_EXE = '%s/bin/java' %JAVA_HOME
JAVAC_EXE = '%s/bin/javac' %JAVA_HOME
JAVA_COMMAND = '%s --add-modules jdk.incubator.foreign -server -Xms2g -Xmx2g
-XX:-TieredCompilation -XX:+HeapDumpOnOutOfMemoryError
```
Executed this with all defaults in `localrun.py`:
```
$ python3 src/python/localrun.py -source wikimedium10k
```
I have no idea why the facetting stuff at the beginning of the bench output
is so badly behaving with MMapDirectory#v2 on top of project panama, I'll
ignore this for now. Maybe @mikemccand has an idea! The rest looks perfectly
fine to me from the performance (19 runs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
