We are working on optimizing query performance. My concern was to ensure
some stable QoS. Given our API and UI layout, user may generate an
expensive query. Given the nature of the service, user may want to
"hack" it. Currently, our Search API is a good point to try to inflict
DoS on our server. And even though search outage will not cause any real
security concern, it would be not nice.
That is why I wanted to put a hard limit on the query complexity. Thank
you for a hint on how to do it.
As a side note, search performance with Solr is great. It is only during
a good load test I am able to see those long running queries. When there
is no load, even the most expensive query I have takes less than 100ms
to be processed. As you said, 2.5M docs is not a very big index.
Thanks again for the reply. I am not sure if we are going to implement
custom component for Solr or put query complexity estimation code in our
application. But in any case your response was greatly appreciated,
because I was thinking that I am missing something.
-- Aleksey
On 12-08-30 05:51 AM, Erick Erickson wrote:
The first thing I'd do is run your query with &debguQuery=on and look
at the "timings" section. That'll tell you what component is taking all
the time and should help you figure out where the problem is....
But worst-case you could implement a custom component to stop
processing after some set number of responses..
2.5M docs isn't a very big index. So I'd look at the rest of the
tuning knobs before jumping to a solution. Also be aware that
the first time, for instance, a sort gets performed there's a lengthy
hit for warming the caches so you should disregard the first few
queries, or do appropriate autowarming.
Best
Erick
On Wed, Aug 29, 2012 at 1:26 PM, Aleksey Vorona <avor...@ea.com> wrote:
Hi, we are running Solr 3.6.1 and see an issue in our load tests. Some of
the queries our load test script produces result in huge number of hits. It
may go as high as 90% of all documents we have (2.5M). Those are all range
queries. I see in the log that those queries take much more time to execute.
Since such a query does not make any sense from the end user perspective, I
would like to limit its performance impact.
Is it possible to abort the query after certain number of document hits or
certain time elapsed and return a error? I would render that error as
"Please refine your search" message to the end user in my application. I
know that many sites on the web do that, and I guess most of them do that
with Solr.
I tried setting timeAllowed limit, but, for some reason, I did not see those
query times to go down. I suspect that most of the time is spent not in
Search phase (which is the only one respecting timeAllowed, as far as I
know), but in the sorting phase. And still, I want to abort any longer
running query. Otherwise they accumulate over time, pushing server's load
average sky high and killing performance even for regular queries.
-- Aleksey
