lsh1215 opened a new pull request, #16300:
URL: https://github.com/apache/lucene/pull/16300
### Description
This updates `TestTermInSetQuery.testDuel` to avoid avoidable memory
pressure in the stress reproduction from #14611.
The test checks query equivalence, not query-cache behavior. It still uses
the randomized test-framework `newSearcher(reader)`, but disables query caching
for this specific test so repeated no-score searches do not retain cached
doc-id sets across iterations.
It also changes the no-score comparison path to collect matching doc IDs
into a bit set instead of materializing full sorted `TopDocs`. For this branch
of the assertion, doc-id set equality is all that is needed.
This is intentionally different from prior range-cap attempts such as #15084
and #15118: it does not reduce randomized input sizes or the test coverage
shape. Instead, it removes the pressure sources observed in local profiling.
### Local investigation
I could not reproduce the OOM on my machine with the original issue command,
likely due to local environment differences. To make the failure mode visible
locally, I used the same stress shape and constrained the test JVM to a 16m
heap.
With the original code, the 16m squeeze reproduced:
```text
java.lang.OutOfMemoryError: Java heap space
```
Heap dump inspection, `jstat`, and JFR heap profiling pointed at two
avoidable sources of pressure in this equivalence test:
* the randomized test-framework query cache retaining cached doc-id sets
* the no-score branch materializing full sorted `TopDocs` even though it
only needs doc-id equality
I am not attaching local VisualVM/JFR screenshots since the commands and
results below are easier to reproduce and review.
### Verification
Ran with JDK 26:
```bash
export
JAVA_HOME=/opt/homebrew/opt/openjdk@26/libexec/openjdk.jdk/Contents/Home
export PATH="$JAVA_HOME/bin:$PATH"
```
```bash
./gradlew tidy
```
Result:
```text
BUILD SUCCESSFUL
```
```bash
./gradlew :lucene:core:compileTestJava
```
Result:
```text
BUILD SUCCESSFUL in 14s
```
Affected nightly test:
```bash
./gradlew :lucene:core:test --tests
"org.apache.lucene.search.TestTermInSetQuery.testDuel" \
-Ptests.filter=@Nightly \
-Ptests.jvms=1 \
-Ptests.seed=B03A5F38917C1431 \
-Ptests.useSecurityManager=false \
-Ptests.gui=true \
-Ptests.file.encoding=UTF-8 \
-Ptests.vectorsize=128 \
-Ptests.forceintegervectors=true
```
Result:
```text
:lucene:core:test (SUCCESS): 1 test
BUILD SUCCESSFUL in 3s
```
Class-level non-nightly coverage:
```bash
./gradlew :lucene:core:test --tests
"org.apache.lucene.search.TestTermInSetQuery" \
-Ptests.jvms=1 \
-Ptests.seed=B03A5F38917C1431 \
-Ptests.useSecurityManager=false \
-Ptests.gui=true \
-Ptests.file.encoding=UTF-8 \
-Ptests.vectorsize=128 \
-Ptests.forceintegervectors=true
```
Result:
```text
:lucene:core:test (SUCCESS): 15 tests, 1 skipped
BUILD SUCCESSFUL in 9s
```
Stress squeeze verification, using the same patch but temporarily changing
the inner `testDuel` loop from `100` to `10000` locally and constraining the
test JVM to 16m heap:
```bash
./gradlew :lucene:core:test --tests
"org.apache.lucene.search.TestTermInSetQuery.testDuel" \
-Ptests.filter=@Nightly \
-Ptests.jvms=1 \
"-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseG1GC -verbose:gc
-XX:ParallelGCThreads=1 -XX:ConcGCThreads=1" \
-Ptests.seed=B03A5F38917C1431 \
-Ptests.useSecurityManager=false \
-Ptests.gui=true \
-Ptests.file.encoding=UTF-8 \
-Ptests.vectorsize=128 \
-Ptests.forceintegervectors=true \
-Ptests.minheapsize=16m \
-Ptests.heapsize=16m
```
Result:
```text
:lucene:core:test (SUCCESS): 1 test
The slowest tests during this run:
286.51s TestTermInSetQuery.testDuel (:lucene:core)
BUILD SUCCESSFUL in 4m 56s
```
The 16m squeeze remains intentionally GC-heavy, but it completes instead of
reaching the unrecoverable Java heap OOM state observed with the original code.
Fixes #14611.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
