[
https://issues.apache.org/jira/browse/SOLR-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter reopened SOLR-14588:
---------------------------------------
TestCircuitBreaker.testBuildingMemoryPressure has failed several times in
non-reproducible ways on both regular jenkins run, and on Jira patch review
builds (when the patches do not in any way affect code executed by the test)
Based on the nature of the test, i'm guessing the problem is related to either:
concurrency bugs in the "real" code; timing assumptions that are violated by
the test (ie: is registering then ew circut breaker a non-blocking method
call?); or something about the combination of the test code + real cod thta is
finiky when dealing with low-resource build servers.
The failures always look like this...
{noformat}
[junit4] 2> 2568622 INFO
(TEST-TestCircuitBreaker.testBuildingMemoryPressure-seed#[52BF77C417B002F6]) [
] o.a.s.SolrTestCaseJ4 ###Starting testBuildingMemoryPressure
[junit4] 2> 2568623 INFO (TestCircuitBreaker-32014-thread-3) [ ]
o.a.s.c.S.Request [collection1] webapp=null path=null
params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
[junit4] 2> 2568623 INFO (TestCircuitBreaker-32014-thread-1) [ ]
o.a.s.c.S.Request [collection1] webapp=null path=null
params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
[junit4] 2> 2568623 INFO (TestCircuitBreaker-32014-thread-2) [ ]
o.a.s.c.S.Request [collection1] webapp=null path=null
params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
[junit4] 2> 2568623 INFO (TestCircuitBreaker-32014-thread-4) [ ]
o.a.s.c.S.Request [collection1] webapp=null path=null
params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
[junit4] 2> 2568624 INFO (TestCircuitBreaker-32014-thread-5) [ ]
o.a.s.c.S.Request [collection1] webapp=null path=null
params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
[junit4] 2> 2568625 INFO
(TEST-TestCircuitBreaker.testBuildingMemoryPressure-seed#[52BF77C417B002F6]) [
] o.a.s.SolrTestCaseJ4 ###Ending testBuildingMemoryPressure
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCircuitBreaker
-Dtests.method=testBuildingMemoryPressure -Dtests.seed=52BF77C417B002F6
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=bas-CM
-Dtests.timezone=America/Creston -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.01s J1 | TestCircuitBreaker.testBuildingMemoryPressure <<<
[junit4] > Throwable #1: java.lang.AssertionError: Number of failed
queries is not correct expected:<1> but was:<5>
[junit4] > at
__randomizedtesting.SeedInfo.seed([52BF77C417B002F6:ED325B14383CEE5F]:0)
[junit4] > at
org.apache.solr.util.TestCircuitBreaker.testBuildingMemoryPressure(TestCircuitBreaker.java:141)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
{noformat}
If the fix for this probablem is not obvious to the folks who worked on this
jira, then please PLEASE at least:
* update the test to log the specific details of these "expected" exceptions
that increment the fail counter
* keep an eye on the future jenkins builds looking for future failutes with
the modified logging to confirm what exactly is happening if/when this "all 5
requests" failed situation occurs.
> Circuit Breakers Infrastructure and Real JVM Based Circuit Breaker
> ------------------------------------------------------------------
>
> Key: SOLR-14588
> URL: https://issues.apache.org/jira/browse/SOLR-14588
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Atri Sharma
> Assignee: Atri Sharma
> Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 13h 50m
> Remaining Estimate: 0h
>
> This Jira tracks addition of circuit breakers in the search path and
> implements JVM based circuit breaker which rejects incoming search requests
> if the JVM heap usage exceeds a defined percentage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
