Thanks everyone for taking time to respond to my email. I think you are
correct in that the query results might be coming from main memory as I
only had around 7k queries.
However it is still not clear to me, given that everything was being
served from main memory, why is that I am not able to push
On 4/28/2017 12:43 PM, Toke Eskildsen wrote:
> Shawn Heisey wrote:
>> Adding more shards as Toke suggested *might* help,[...]
> I seem to have phrased my suggestion poorly. What I meant to suggest
> was a switch to a single shard (with 4 replicas) setup, instead of the
> current 2 shards (with 2
Beautiful, thank you.
-Original Message-
From: Walter Underwood [mailto:wun...@wunderwood.org]
Sent: Friday, April 28, 2017 3:07 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr Query Performance benchmarking
I use the JMeter plugins. They’ve been reorganized recently, so they
niel (NIH/NLM) [C]
> wrote:
>
> Walter,
>
> If you can share a pointer to that JMeter add-on, I'd love it.
>
> -Original Message-
> From: Walter Underwood [mailto:wun...@wunderwood.org]
> Sent: Friday, April 28, 2017 2:53 PM
> To: solr-user@lucene
Walter,
If you can share a pointer to that JMeter add-on, I'd love it.
-Original Message-
From: Walter Underwood [mailto:wun...@wunderwood.org]
Sent: Friday, April 28, 2017 2:53 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr Query Performance benchmarking
I use production
I use production logs to get a mix of common and long-tail queries. It is very
hard to get a realistic distribution with synthetic queries.
A benchmark run goes like this, with a big shell script driving it.
1. Reload the collection to clear caches.
2. Split the log into a cache warming set (usu
Shawn Heisey wrote:
> Adding more shards as Toke suggested *might* help,[...]
I seem to have phrased my suggestion poorly. What I meant to suggest was a
switch to a single shard (with 4 replicas) setup, instead of the current 2
shards (with 2 replicas).
- Toke
Well, the best way to get no cache hits is to set the cache sizes to
zero ;). That provides worst-case scenarios and tells you exactly how
much you're relying on caches. I'm not talking the lower-level Lucene
caches here.
One thing I've done is use the TermsComponent to generate a list of
terms ac
(aside: Using Gatling or Jmeter?)
Question: How can you easily randomize something in the query so you get no
cache hits? I think there are several levels of caching.
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
re: the q vs. fq question. My claim (not verified) is that the fastest
of all would be q=*:*&fq={!cache=false}. That would bypass the scoring
that putting it in the "q" clause would entail as well as bypass the
filter cache.
But I have to agree with Walter, this is very suspicious IMO. Here's
what
More “unrealistic” than “amazing”. I bet the set of test queries is smaller
than the query result cache size.
Results from cache are about 2 ms, but network communication to the shards
would add enough overhead to reach 40 ms.
wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunder
On 4/27/2017 5:20 PM, Suresh Pendap wrote:
> Max throughput that I get: 12000 to 12500 reqs/sec
> 95 percentile query latency: 30 to 40 msec
These numbers are *amazing* ... far better than I would have expected to
see on a 27GB index, even in a situation where it fits entirely into
available memor
On Thu, 2017-04-27 at 23:20 +, Suresh Pendap wrote:
> Number of Solr Nodes: 4
> Number of shards: 2
> replication-factor: 2
> Index size: 55 GB
> Shard/Core size: 27.7 GB
> maxConnsPerHost: 1000
The overhead of sharding is not trivial. Your overall index size is
fairly small, relative to your
We should consider adding another parameter "RealTime" in the log. That
would really help all of us trying to figure out how much time a query is
taking.
On Tue, Jun 4, 2013 at 5:14 PM, Otis Gospodnetic wrote:
> Right. The main takeway is that QTime is not exactly what user sees.
> What users
Right. The main takeway is that QTime is not exactly what user sees.
What users sees is always > QTime. To see what end users experience
one needs RUM (and I don't mean the tasty kind, but
http://en.wikipedia.org/wiki/Real_user_monitoring .
Otis
--
Solr & ElasticSearch Support
http://sematext.co
On Jun 4, 2013, at 8:28 AM, Shawn Heisey wrote:
> On 6/4/2013 8:45 AM, O'Regan, Mike wrote:
>> Can't a GC also kick in before the timer starts, resulting a short QTime but
>> a long time from the point of view of the client? Believe I have seen that
>> phenomenon in action.
>
> You are correct.
On 6/4/2013 8:45 AM, O'Regan, Mike wrote:
> Can't a GC also kick in before the timer starts, resulting a short QTime but
> a long time from the point of view of the client? Believe I have seen that
> phenomenon in action.
You are correct. I don't know why I didn't think of that.
Thanks,
Shawn
> From: Shawn Heisey [s...@elyograg.org]
>
>On 6/3/2013 3:33 PM, Greg Harris wrote:
>>
>> You have to be careful looking at the QTime's. They do not include garbage
>> collection. I've run into issues where QTime is short (cause it was), it
>> just happened that the query came in during a long ga
On 6/3/2013 3:33 PM, Greg Harris wrote:
You have to be careful looking at the QTime's. They do not include garbage
collection. I've run into issues where QTime is short (cause it was), it just
happened that the query came in during a long garbage collection where
everything was paused. So you
cene.apache.org
Subject: Re: Solr query performance tool
You can use this tool to analyze the logs..
https://github.com/dfdeshom/solr-loganalyzer
We use solrmeter to test the performance / Stress testing.
https://code.google.com/p/solrmeter/
--
View this message in context:
http://lucene.47206
You can use this tool to analyze the logs..
https://github.com/dfdeshom/solr-loganalyzer
We use solrmeter to test the performance / Stress testing.
https://code.google.com/p/solrmeter/
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-query-performance-tool-tp406690
The qtimes are in the solr log, you'll see lines like:
params={q=*:*} hits=32 status=0 QTime=5
QTime is the time spent serving the query but does NOT include
assembling the response.
Best
Erick
On Wed, May 29, 2013 at 5:58 PM, Spyros Lambrinidis
wrote:
> Hi,
>
> Lately we are seeing increased l
Hi,
The regular Solr log logs Qtime for each query.
Otis
Solr & ElasticSearch Support
http://sematext.com/
On May 29, 2013 5:59 PM, "Spyros Lambrinidis"
wrote:
> Hi,
>
> Lately we are seeing increased latency times on solr and we would like to
> know which queries / facet searches are the most
On 5/7/2013 8:45 AM, Kamal Palei wrote:
> When user clicks, 4, I will set "start" filter as 300, "rows" filter as 100
> and do the query. As query result, I am expecting row count as 1000, and
> 100 records data (row number 301 to 400).
This is what using the start and rows parameter with Solr wil
Thanks a lot Alex.
I will go and try to make use of start filter and update.
Meantime, if I need to know, how many total search records are there.
Example: Lets say I am searching key word "java".
There might be 1000 documents having java keyword. I need to show only 100
records at a time.
When
Yes, that's what the 'start' and 'rows' parameters do in the query
string. I would check the queries Solr sees when you do that long
request. There is usually a delay in retrieving items further down the
sorted list, but 15 seconds does feel excessive.
http://wiki.apache.org/solr/CommonQueryParame
> Subject: Re: Solr query performance issue
>
> We actually want OR operator on those values. Filters can only do AND,
> right?
>
> Is it better performance to have the query as field1:01 field1:02 field1:03
> instead of field1:(01 02 03)?
>
> BR,
> Larry
>
Another little optimization would be to flatten the query.
Instead of "field1:(02 04 05) field2:(01 02 03)"
use "field1:02 field1:04 field1:05 field2:01 field2:02 field2:03"
But I'd try and narrow down what queries are taking a long time, and
see if there is a common element that could be optimize
Team
> > To: solr-user@lucene.apache.org; yo...@lucidimagination.com
> > Sent: Tuesday, May 26, 2009 4:54:34 PM
> > Subject: Re: Solr query performance issue
> >
> > Yes, those terms are important in calculating the relevancy scores so
> they
> > are not in the filter queries.
on.com
> Sent: Tuesday, May 26, 2009 4:54:34 PM
> Subject: Re: Solr query performance issue
>
> Yes, those terms are important in calculating the relevancy scores so they
> are not in the filter queries. I was hoping if I can cache everything about
> a field, any combinations on
Yes, those terms are important in calculating the relevancy scores so they
are not in the filter queries. I was hoping if I can cache everything about
a field, any combinations on the field values will be read from cache. Then
it does not matter if I query for field1:(02 04 05), or field1:(01 02)
On Tue, May 26, 2009 at 3:42 PM, Larry He wrote:
> We have about 100 different fields and 1 million documents we indexed with
> Solr. Many of the fields are multi-valued, and some are numbers (for range
> search). We are expecting to perform solr queries contains over 30 terms
> and often the re
32 matches
Mail list logo
