rmuir commented on code in PR #633:
URL: https://github.com/apache/lucene/pull/633#discussion_r863840319


##########
lucene/test-framework/src/java/org/apache/lucene/tests/index/MockRandomMergePolicy.java:
##########
@@ -86,6 +86,20 @@ public MergeSpecification findMerges(
     return mergeSpec;
   }
 
+  @Override
+  public MergeSpecification findMerges(CodecReader... readers) throws 
IOException {
+    if (random.nextBoolean() == true) {
+      return super.findMerges(readers);
+    } else {
+      // create a oneMerge for each reader to let them get concurrently 
processed by addIndexes()

Review Comment:
   > Thanks for reviewing this @rmuir, @uschindler. I understand your concern. 
Failures on concurrent code are already tough to reproduce; it gets much harder 
to do this from a randomized test seed.
   > 
   > It is worth noting that the `findMerges(CodecReader[])` API modified in 
`MockRandomMergePolicy`, is only invoked within the 
`addIndexes(CodecReaders[])` API, which should contain its scope of impact. But 
I definitely don't want to add noise with random failures on distantly related 
tests. I'll remove this new policy from `MockRandomMergePolicy`, which is 
frequently used for `IndexWriterConfig` in multiple tests.
   > 
   
   This is true, but also part of the risk: `addIndexes(CodecReaders[])` is 
already pretty scary, correctness-wise. This is typically used to expertly 
"transform" an index, otherwise you wouldn't use this API. It is probably 
undertested today, I just don't want to mix concurrency randomly into the few 
tests it has.
   
   > For my awareness, what is the best practice in Lucene for testing 
non-default, concurrency related code paths? Randomized tests help validate 
infrequently triggered code, but concurrency poses has reproducibility 
challenges. For such cases, do we try to rely entirely on change-specific 
concurrency unit tests?
   
   Well, we need to think about it. I actually hate that 
ConcurrentMergeScheduler gets used the way it does in the tests, for this same 
reason. And that's the actual problem here... I think, not your logic in the 
mergepolicy? So really my concerns are around mergescheduler in tests 
preventing reproducibility. CMS is the default if someone tests with just `new 
IndexWriterConfig` and it is almost always the default if they use 
`LuceneTestCase.newIndexWriterConfig`.
   
   Maybe we can improve our tests to use `SerialMergeScheduler` and 
`ConcurrentMergeScheduler(n=1)` more often, so that reproducibility is improved 
across the board. We can have separate specific tests using 
`ConcurrentMergeScheduler(n>1)`? 
   
   It really isn't anything new from your change here, I think i was just a bit 
frustrated the last time i tried to reproduce a seed recently 
(https://issues.apache.org/jira/browse/LUCENE-10529), so I want us to be 
careful about blanket application of concurrency to a ton of existing tests.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to