rmuir commented on code in PR #633:
URL: https://github.com/apache/lucene/pull/633#discussion_r863840319
##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/MockRandomMergePolicy.java:
##########
@@ -86,6 +86,20 @@ public MergeSpecification findMerges(
return mergeSpec;
}
+ @Override
+ public MergeSpecification findMerges(CodecReader... readers) throws
IOException {
+ if (random.nextBoolean() == true) {
+ return super.findMerges(readers);
+ } else {
+ // create a oneMerge for each reader to let them get concurrently
processed by addIndexes()
Review Comment:
> Thanks for reviewing this @rmuir, @uschindler. I understand your concern.
Failures on concurrent code are already tough to reproduce; it gets much harder
to do this from a randomized test seed.
>
> It is worth noting that the `findMerges(CodecReader[])` API modified in
`MockRandomMergePolicy`, is only invoked within the
`addIndexes(CodecReaders[])` API, which should contain its scope of impact. But
I definitely don't want to add noise with random failures on distantly related
tests. I'll remove this new policy from `MockRandomMergePolicy`, which is
frequently used for `IndexWriterConfig` in multiple tests.
>
This is true, but also part of the risk: `addIndexes(CodecReaders[])` is
already pretty scary, correctness-wise. This is typically used to expertly
"transform" an index, otherwise you wouldn't use this API. It is probably
undertested today, I just don't want to mix concurrency randomly into the few
tests it has.
> For my awareness, what is the best practice in Lucene for testing
non-default, concurrency related code paths? Randomized tests help validate
infrequently triggered code, but concurrency poses has reproducibility
challenges. For such cases, do we try to rely entirely on change-specific
concurrency unit tests?
Well, we need to think about it. I actually hate that
ConcurrentMergeScheduler gets used the way it does in the tests, for this same
reason. And that's the actual problem here... I think, not your logic in the
mergepolicy? So really my concerns are around mergescheduler in tests
preventing reproducibility. CMS is the default if someone tests with just `new
IndexWriterConfig` and it is almost always the default if they use
`LuceneTestCase.newIndexWriterConfig`.
Maybe we can improve our tests to use `SerialMergeScheduler` and
`ConcurrentMergeScheduler(n=1)` more often, so that reproducibility is improved
across the board. We can have separate specific tests using
`ConcurrentMergeScheduler(n>1)`?
It really isn't anything new from your change here, I think i was just a bit
frustrated the last time i tried to reproduce a seed recently
(https://issues.apache.org/jira/browse/LUCENE-10529), so I want us to be
careful about blanket application of concurrency to a ton of existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]