slow-J commented on issue #11076: URL: https://github.com/apache/lucene/issues/11076#issuecomment-1824516300
Hi @gsmiller, I found a drill sideways tasks file in luceneutil ([ds.tasks](https://github.com/mikemccand/luceneutil/blob/5fae60800edd84c70492ac8765824ca2e4a6a991/tasks/ds.tasks)) I tried locally creating a wikimediumds (`WIKI_MEDIUM_DS = Data('wikimediumds', constants.WIKI_MEDIUM_DOCS_LINE_FILE, constants.WIKI_MEDIUM_DOCS_COUNT, constants.WIKI_DS)`) and ran `python3 src/python/localrun.py -source wikimediumds` I commented out calls to `doUnionScoring` and `doDrillDownAdvanceScoring` and left only `doQueryFirstScoring` https://github.com/slow-J/lucene/commit/7423ddda30a2ca5e3fd90ee9743b3c40ad6b94e0 , it passes all unit tests. ``` TaskQPS baseline StdDevQPS my_modified_version StdDev Pct diff p-value HighTermEasyDD2 2718.56 (6.0%) 2656.82 (4.8%) -2.3% ( -12% - 9%) 0.189 MedTermEasyOrDD1 2454.91 (5.6%) 2417.95 (4.4%) -1.5% ( -10% - 8%) 0.343 MedTermMixedDD2 3184.16 (5.5%) 3137.70 (7.0%) -1.5% ( -13% - 11%) 0.465 LowTermEasyDD2 2602.71 (6.0%) 2570.49 (5.6%) -1.2% ( -12% - 11%) 0.503 LowTermEasyDD1 2583.23 (4.7%) 2553.44 (5.2%) -1.2% ( -10% - 9%) 0.465 HighTermMixedDD2 2673.50 (4.3%) 2643.56 (6.8%) -1.1% ( -11% - 10%) 0.536 LowTermHardDD1 2668.88 (5.9%) 2639.66 (4.0%) -1.1% ( -10% - 9%) 0.496 MedTermHardDD1 2320.13 (5.1%) 2298.27 (5.7%) -0.9% ( -11% - 10%) 0.580 HighTermHardOrDD2 2477.61 (4.8%) 2454.33 (3.6%) -0.9% ( -8% - 7%) 0.481 PKLookup 110.75 (12.0%) 109.98 (11.3%) -0.7% ( -21% - 25%) 0.849 HighTermEasyDD1 2777.91 (6.0%) 2762.76 (6.7%) -0.5% ( -12% - 12%) 0.787 MedTermHardOrDD1 2998.49 (5.2%) 2983.15 (5.8%) -0.5% ( -10% - 11%) 0.769 HighTermEasyOrDD1 2513.54 (4.9%) 2503.33 (4.4%) -0.4% ( -9% - 9%) 0.782 LowTermHardOrDD2 2349.94 (6.2%) 2346.39 (4.9%) -0.2% ( -10% - 11%) 0.932 LowTermHardDD2 2566.37 (4.5%) 2570.74 (5.5%) 0.2% ( -9% - 10%) 0.914 MedTermEasyDD2 3231.73 (6.8%) 3239.67 (6.4%) 0.2% ( -12% - 14%) 0.906 MedTermEasyDD1 3323.66 (7.4%) 3344.85 (5.4%) 0.6% ( -11% - 14%) 0.755 LowTermEasyOrDD1 2385.39 (5.0%) 2401.56 (5.6%) 0.7% ( -9% - 11%) 0.686 LowTermEasyOrDD2 2430.31 (4.7%) 2454.95 (5.6%) 1.0% ( -8% - 11%) 0.534 HighTermEasyOrDD2 2394.51 (5.0%) 2424.64 (6.2%) 1.3% ( -9% - 13%) 0.478 MedTermEasyOrDD2 2856.22 (7.0%) 2892.76 (5.1%) 1.3% ( -10% - 14%) 0.508 LowTermMixedDD2 2672.52 (5.9%) 2716.97 (5.9%) 1.7% ( -9% - 14%) 0.372 MedTermHardDD2 2319.03 (6.4%) 2361.41 (4.6%) 1.8% ( -8% - 13%) 0.299 HighTermHardOrDD1 2550.89 (5.4%) 2598.09 (2.7%) 1.9% ( -5% - 10%) 0.172 LowTermHardOrDD1 2429.07 (7.3%) 2484.87 (4.6%) 2.3% ( -8% - 15%) 0.234 HighTermHardDD2 2627.08 (5.8%) 2697.49 (5.7%) 2.7% ( -8% - 14%) 0.139 MedTermHardOrDD2 2846.21 (6.0%) 2956.36 (6.2%) 3.9% ( -7% - 17%) 0.045 HighTermHardDD1 2684.90 (4.4%) 2800.76 (3.3%) 4.3% ( -3% - 12%) 0.000 ``` I think from results we can see that there is no performance loss from removing doUnionScoring` and `doDrillDownAdvanceScoring`. I am wondering is the setup correct enough to trust the results? Does ([ds.tasks](https://github.com/mikemccand/luceneutil/blob/5fae60800edd84c70492ac8765824ca2e4a6a991/tasks/ds.tasks)) contain enough tasks? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
