rmuir commented on code in PR #11867:
URL: https://github.com/apache/lucene/pull/11867#discussion_r1001212579
##########
lucene/core/src/test/org/apache/lucene/document/TestManyKnnVectors.java:
##########
@@ -61,11 +61,13 @@
@Monster("takes ~2 hours and needs 2GB heap")
public class TestManyKnnVectors extends LuceneTestCase {
public void testLargeSegment() throws Exception {
- // Make sure to use the default codec instead of a random one
- IndexWriterConfig iwc =
newIndexWriterConfig().setCodec(TestUtil.getDefaultCodec());
+ IndexWriterConfig iwc = new IndexWriterConfig();
+ iwc.setCodec(TestUtil.getDefaultCodec()); // Make sure to use the default
codec instead of a random one
+ iwc.setRAMBufferSizeMB(3_000); // Use a 3GB buffer to create a single
large segment
Review Comment:
Maybe use a smaller value here... otherwise we need to change the docs of
`@Monster` annotation and your comments about configuring test heap sizes. And
I think its good to keep monster tests less monstrous.
I'm using 200MB buffer, still with your suggested 2GB heap. It seems to
flush about 600-700k docs in each segment. There's no merges happening until
the test asks for it with a forceMerge(1), which is running now. compared to
the 3GB buffer, yeah, I've gotta suffer the forceMerge in the end, not sure how
long that's gonna take, but it exercises the merge code as well as the flush
code, and keeps the heap memory usage lower, for less monstrosity in the test.
May be the right tradeoff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
