On 6/3/2013 2:39 AM, Bernd Fehling wrote:
> How are you handling "killer queries" with solr?
>
> While solr/lucene (currently 4.2.1) is trying to do its best I see sometimes
> stupid queries
> in my logs, located with extremly long query time.
>
> Example:
> q=???????+and+??+and+???+and+????+and+???????+and+??????????
>
> I even get hits for this (hits=34091309 status=0 QTime=88667).
>
> But the jetty log says:
> WARN:oejs.Response:Committed before 500 {msg=Datenübergabe unterbrochen
> (broken pipe),trace=org.eclipse.jetty.io.EofException...
> org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:838)|?...
> 35 more|,code=500}
> WARN:oejs.ServletHandler:/solr/base/select
> java.lang.IllegalStateException: Committed
> at org.eclipse.jetty.server.Response.resetBuffer(Response.java:1136)
>
> Because I get hits and qtime the search is successful, right?
>
> But jetty/http has already closed the connection and solr doesn't know about
> this?
>
> How are you handling "killer queries", just ignoring?
> Or something to tune (jetty config about timeout) or filter (query filtering)?
As you might know, EofException happens when one end (usually the
client) closes the TCP connection before the response is delivered.
This is usually caused by explicitly setting timeouts, or by using a
load balancer in front of Solr, because these will normally limit how
long the response can take. The timeout involved is probably 60 seconds
in this case, and the query took nearly 90 seconds.
It doesn't cause any *direct* problems for Solr, though the nasty
exception that gets logged every time is annoying. A query like that
does use a lot of resources, so if the server doesn't have a lot of
spare capacity, it can cause problems for everyone else.
Assuming that this isn't happening due to bugs in your application, the
only way to really handle this problem is to first locate the problem
user and educate them. If the problem continues and it's a viable
option, you might need to ban that user from your system.
Thanks,
Shawn
