[
https://issues.apache.org/jira/browse/LUCENE-10473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508728#comment-17508728
]
Adrien Grand commented on LUCENE-10473:
---------------------------------------
I had a quick look at this and I think that there are a few things we can do to
speed up these tests significantly before considering making them monsters.
What looks like the biggest offender to me is how {{tests.nightly}} interacts
with {{tests.multiplier}} for {{LuceneTestCase#atLeast}}. If you pass
{{tests.nightly}} and {{tests.multiplier=3}} then {{atLeast(x)}} returns a
number that is between 6*x and 9*x. This feels to me like {{tests.nightly}} is
increasing the multiplier too much on top of the configured tests multiplier.
Could we change this so that {{LuceneTestCase#atLeast}} ignores
{{tests.nightly}} or maybe {{tests.nightly}} just changes the default value of
{{tests.multiplier}} to 2 instead of 1? Or alternatively change
{{testRandomBig}} to pass count=100k instead of 200k knowing that the fact that
it runs with {{tests.nightly}} means that this number will be changed to 200k
anyway in practice.
Then the test also does wasteful stuff, like it uses
{{LuceneTestCase#newStringField}} for its ID field, which randomly enables term
vectors, so we end up spending non negligible time merging term vectors for no
reason.
These two changes seem to make {{TestDoubleRangeFieldQueries.testRandomBig}}
run 6x-10x faster on my machine with {{-Dtests.nightly=true
-Dtests.multiplier=3}}.
> Address slow testRandomBig runs
> -------------------------------
>
> Key: LUCENE-10473
> URL: https://issues.apache.org/jira/browse/LUCENE-10473
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Julie Tibshirani
> Priority: Major
>
> While working on the 9.1 release, we noticed the smoke tester sometimes
> taking several hours. It looks like some tests can take a really long time,
> especially when a higher "tests.multiplier" is used (than the default of 1):
> {code:java}
> The slowest tests (exceeding 500 ms) during this run:
> 3298.44s TestDoubleRangeFieldQueries.testRandomBig (:lucene:core)
> 2869.82s Test2BPostings.test (:lucene:core)
> 1951.74s TestLatLonDocValuesQueries.testRandomBig (:lucene:core)
> 1628.04s TestLatLonPointQueries.testRandomBig (:lucene:core)
> 1492.32s TestGeo3DPoint.testRandomBig (:lucene:spatial3d)
> 1481.19s TestXYDocValuesQueries.testRandomBig (:lucene:core)
> 1351.95s TestXYPointQueries.testRandomBig (:lucene:core)
> 940.30s TestLongRangeFieldQueries.testRandomBig (:lucene:core)
> 871.50s Test4GBStoredFields.test (:lucene:core)
> 743.00s TestFloatRangeFieldQueries.testRandomBig (:lucene:core)
> {code}
> -The main offender looks like {{{}BaseSpatialTestCase#testRandomBig{}}}, we
> should look into making this run faster.-
> Maybe relates to LUCENE-8643?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
