rmuir opened a new pull request #500:
URL: https://github.com/apache/lucene/pull/500
Recently when running tests, I noticed slowest-test/suite times were
dominated by term vectors merging tests:
Here's a typical run:
```
The slowest tests (exceeding 500 ms) during this run:
13.61s TestSimpleTextTermVectorsFormat.testMergeWithIndexSort
(:lucene:codecs)
11.73s TestNRTReplication.testCrashReplica (:lucene:replicator)
10.86s TestSimpleTextTermVectorsFormat.testMergeWithoutIndexSort
(:lucene:codecs)
6.44s TestGeo3dRpt.testOperationsFromFile (:lucene:spatial-extras)
6.15s TestAssertingTermVectorsFormat.testMergeWithIndexSort
(:lucene:test-framework)
5.99s TestCachePurging.testConcurrentPurges (:lucene:monitor)
5.99s TestCompressingTermVectorsFormat.testMergeWithIndexSort
(:lucene:test-framework)
5.70s TestBagOfPostings.test (:lucene:core)
5.39s TestAssertingTermVectorsFormat.testMergeWithoutIndexSort
(:lucene:test-framework)
5.38s TestCompressingTermVectorsFormat.testMergeWithoutIndexSort
(:lucene:test-framework)
The slowest suites (exceeding 1s) during this run:
36.35s TestSimpleTextTermVectorsFormat (:lucene:codecs)
17.50s TestSimpleTextDocValuesFormat (:lucene:codecs)
16.76s TestCompressingTermVectorsFormat (:lucene:test-framework)
15.44s TestAssertingTermVectorsFormat (:lucene:test-framework)
12.80s TestLucene90DocValuesFormat (:lucene:core)
12.58s TestAssertingDocValuesFormat (:lucene:test-framework)
11.91s TestLucene90TermVectorsFormat (:lucene:core)
11.81s TestNRTReplication (:lucene:replicator)
11.76s TestPerFieldDocValuesFormat (:lucene:core)
11.58s TestLucene90DocValuesFormatMergeInstance (:lucene:core)
```
So it doesn't help to just mark the tests nightly for SimpleText, as they
are slow for the other formats too. The main problem is each test indexes 100
docs many different ways (different index options X deletes X sort). Rather
than make things nightly, let's just use less documents for normal runs:
* Use less iterations locally so that term vector merging doesn't dominate
the list of slowest tests.
* Split out deletes/no-deletes into separate methods to improve
debuggability.
* Remove nightly from SimpleText term vectors merging tests, now that they
run much faster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
