uschindler edited a comment on pull request #177:
URL: https://github.com/apache/lucene/pull/177#issuecomment-860641916
Hi,
I executed luceneutil bench using jdk-17-ea+b25:
```
TaskQPS baseline StdDevQPS my_modified_version
StdDev Pct diff p-value
BrowseMonthTaxoFacets 1.00 (4.8%) 0.71 (6.7%)
-29.1% ( -38% - -18%) 0.000
BrowseDayOfYearTaxoFacets 0.98 (5.9%) 0.70 (7.3%)
-28.1% ( -38% - -15%) 0.000
BrowseDateTaxoFacets 0.98 (5.9%) 0.70 (7.2%)
-28.1% ( -38% - -15%) 0.000
AndHighLow 326.58 (6.1%) 288.18 (3.6%)
-11.8% ( -20% - -2%) 0.000
OrHighLow 203.59 (6.2%) 182.35 (3.3%)
-10.4% ( -18% - 0%) 0.000
AndHighMed 51.56 (6.6%) 46.25 (4.8%)
-10.3% ( -20% - 1%) 0.000
Respell 35.89 (1.5%) 32.70 (1.8%)
-8.9% ( -12% - -5%) 0.000
LowSloppyPhrase 13.87 (3.4%) 12.77 (3.1%)
-7.9% ( -13% - -1%) 0.000
PKLookup 189.60 (2.6%) 174.64 (2.7%)
-7.9% ( -12% - -2%) 0.000
LowSpanNear 20.18 (3.1%) 18.76 (2.2%)
-7.0% ( -11% - -1%) 0.000
Fuzzy1 59.60 (6.5%) 55.88 (6.2%)
-6.2% ( -17% - 6%) 0.002
Fuzzy2 49.20 (9.0%) 46.35 (7.5%)
-5.8% ( -20% - 11%) 0.026
AndHighHigh 28.28 (4.6%) 26.69 (3.6%)
-5.6% ( -13% - 2%) 0.000
MedSpanNear 14.89 (3.4%) 14.16 (3.4%)
-4.9% ( -11% - 1%) 0.000
MedSloppyPhrase 52.15 (4.9%) 49.89 (4.2%)
-4.3% ( -12% - 5%) 0.003
OrHighMed 40.28 (2.9%) 38.64 (2.7%)
-4.1% ( -9% - 1%) 0.000
MedPhrase 247.62 (3.3%) 238.13 (3.0%)
-3.8% ( -9% - 2%) 0.000
HighTermDayOfYearSort 21.04 (16.5%) 20.33 (13.5%)
-3.4% ( -28% - 31%) 0.475
OrNotHighLow 457.51 (4.1%) 442.74 (2.9%)
-3.2% ( -9% - 3%) 0.004
OrHighHigh 7.00 (3.0%) 6.78 (3.0%)
-3.0% ( -8% - 2%) 0.001
HighSpanNear 1.77 (3.4%) 1.72 (3.2%)
-2.8% ( -9% - 3%) 0.006
Wildcard 55.88 (3.4%) 54.34 (3.2%)
-2.8% ( -9% - 4%) 0.009
BrowseMonthSSDVFacets 4.18 (5.4%) 4.08 (5.2%)
-2.5% ( -12% - 8%) 0.140
HighSloppyPhrase 6.16 (7.9%) 6.01 (6.0%)
-2.4% ( -15% - 12%) 0.278
TermDTSort 56.84 (19.7%) 55.64 (15.8%)
-2.1% ( -31% - 41%) 0.707
BrowseDayOfYearSSDVFacets 4.06 (3.8%) 3.97 (4.0%)
-2.1% ( -9% - 5%) 0.096
LowPhrase 38.74 (3.7%) 37.95 (2.7%)
-2.1% ( -8% - 4%) 0.047
Prefix3 59.38 (5.9%) 58.16 (6.0%)
-2.0% ( -13% - 10%) 0.273
HighIntervalsOrdered 5.89 (6.2%) 5.78 (5.1%)
-2.0% ( -12% - 9%) 0.274
HighTermTitleBDVSort 34.99 (13.7%) 34.60 (13.1%)
-1.1% ( -24% - 29%) 0.795
HighTermMonthSort 56.96 (16.7%) 57.28 (14.7%)
0.6% ( -26% - 38%) 0.910
OrHighNotLow 550.54 (5.1%) 558.86 (5.0%)
1.5% ( -8% - 12%) 0.347
IntNRQ 33.75 (30.5%) 34.28 (29.5%)
1.6% ( -44% - 88%) 0.870
HighTerm 1141.71 (6.2%) 1163.18 (4.6%)
1.9% ( -8% - 13%) 0.273
HighPhrase 12.90 (3.9%) 13.15 (4.3%)
1.9% ( -6% - 10%) 0.136
OrHighNotHigh 640.96 (6.4%) 665.45 (3.8%)
3.8% ( -5% - 14%) 0.022
OrNotHighMed 692.81 (4.4%) 726.70 (4.6%)
4.9% ( -3% - 14%) 0.001
OrNotHighHigh 698.56 (4.5%) 739.13 (3.7%)
5.8% ( -2% - 14%) 0.000
MedTerm 1594.10 (5.5%) 1700.62 (4.6%)
6.7% ( -3% - 17%) 0.000
LowTerm 1531.60 (5.2%) 1647.26 (4.6%)
7.6% ( -2% - 18%) 0.000
OrHighNotMed 911.74 (5.3%) 990.26 (4.4%)
8.6% ( -1% - 19%) 0.000
```
For those interested:
```
JAVA:
WARNING: Using incubator modules: jdk.incubator.foreign
openjdk version "17-ea" 2021-09-14
OpenJDK Runtime Environment (build 17-ea+25-2252)
OpenJDK 64-Bit Server VM (build 17-ea+25-2252, mixed mode, sharing)
OS:
Linux serv1.sd-datasolutions.de 5.8.0-50-generic #56~20.04.1-Ubuntu SMP Mon
Apr 12 21:46:35 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
LOGS:
/home/thetaphi/benchmark/logs
```
My localconstants.py, so that's how I added the module command line params.
@mikemccand It would be better to have the tool respect `RUNTIME_JAVA_HOME`
like gradlew does.
```python
BASE_DIR = '/home/thetaphi/benchmark'
BENCH_BASE_DIR = '/home/thetaphi/benchmark/util'
JAVA_HOME = '/home/jenkins/tools/java/64bit/latest-jdk17'
JAVA_EXE = '%s/bin/java' %JAVA_HOME
JAVAC_EXE = '%s/bin/javac' %JAVA_HOME
JAVA_COMMAND = '%s --add-modules jdk.incubator.foreign -server -Xms2g -Xmx2g
-XX:-TieredCompilation -XX:+HeapDumpOnOutOfMemoryError
```
Executed this with all defaults in `localrun.py`:
```
$ python3 src/python/localrun.py -source wikimediumall
```
I have no idea why the facetting stuff at the beginning of the bench output
is so badly behaving with MMapDirectory#v2 on top of project panama, I'll
ignore this for now. Maybe @mikemccand has an idea! The rest looks perfectly
fine to me from the performance (19 runs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
