I've run some test on both the versions of Solr we are testing... one is the
2010.12.10 build and the other is the 2012.02.16 build. The latter one is
where we were initially seeing poor response performance. I've attached 4
text files which have the results of a few runs against each of the buil
I will run some queries today, both with lazyfield loading on and off (for
the 2010 build we're using and the 2012 build we're using) and get you some
of the debug data.
On Sun, Feb 26, 2012 at 4:13 PM, Yonik Seeley-2-2 [via Lucene] <
ml-node+s472066n318...@n3.nabble.com> wrote:
> On Sun, F
On Sun, Feb 26, 2012 at 3:32 PM, Erick Erickson wrote:
> Would you hypothesize that lazy field loading could be that much
> slower if a large fraction of fields were selected?
If you actually use the lazy field later, it will cause an extra read
for each field.
If you don't have enough free RAM f
I sure can't reproduce this on an 11M document Wikipedia dump.
I added the "text" from the Wiki dump 49 extra times (i.e. there
are 50 copies of the text field in each document), and pulled
back 12000 documents from my test machine (a Macbook Pro
from 3 years ago). I also debugged the code a bit an
Obviously it'd be great if someone else was able to confirm this in their
setup as well.
But with different environments, payload sizes, etc., I'm not sure how
easily it can be tested in other environments.
On Fri, Feb 24, 2012 at 2:46 PM, Brian G wrote:
> Erick -
>
> That is exactly what we ar
Erick -
That is exactly what we are seeing.
this is in our solrconfig.xml:
false
and our response times have decreased drastically. I'm on my 40th-ish test
today and the response times are still 10+ seconds faster on the higher
payload than they were when it was set to true.
Smaller payloads a
Let me echo this back to see if I have it right, because it's *extremely*
weird if I'm reading it correctly.
In your solrconfig.xml file, you changed this line:
true
to this:
false
and your response time DECREASED? If you can confirm that
I'm reading it right, I'll open up a JIRA.
Best
Erick
On
I'm not sure what would constitute a low vs. high hit rate (and eviction
rate), so we've kept the setting at LRUCache instead of FastCache for now.
But I will say we did turn the LazyFieldLoading option off and wow - a huge
increase in performance on the newer nightly build we are using (the one
f
jconsole should just be there in your Java SDK, you shouldn't have to install
anything. Connecting remotely is a little trickier, here's the reference.
http://docs.oracle.com/javase/1.5.0/docs/guide/management/agent.html
I cheat and disable authentication, see the "disabling security" section, but
On Fri, Feb 24, 2012 at 11:24 AM, naptowndev wrote:
> Another question I have is regarding solr.LRUCache vs. solr.FastLRUCache.
> Would there be reason to implement (or not implement) fastLRU on the
> documentcache?
LRUCache can be faster if the hit rate is really low (i.e. the
eviction rate is h
Yonik -
Thanks, we'll give that a try (re: lazyfieldlaoding).
and no, the * is not in our config...that must have come over from pasting
it in from the file. Odd.
Another question I have is regarding solr.LRUCache vs. solr.FastLRUCache.
Would there be reason to implement (or not implement) fas
On Fri, Feb 24, 2012 at 10:25 AM, naptowndev wrote:
> Our current config for that is as follows:
> initialSize="*15000*"autowarmCount
> ="*0*" />
>
> It's the same for both instances
I assume the asterisks are for emphasis and are not actually present
in your config?
> And lazyfieldloading is e
Thanks again.
We're trying to get our ops team to install jconsole for us so we can take
a look at the GC stuff.
Your comment about the documentcache is intriguing for sure.
We just ran a couple of test against the older 4.x build we have (that's
been returning quicker) and the newer in that we
It's still worth looking at the GC characteristics, there's a possibility
that the newer build uses memory such that you're tripping over some
threshold, but that's grasping at straws. I'd at least hook up jConsole
for a sanity check...
But if your QTimes are fast, the next thing that comes to min
Erick -
Thanks. We've actually worked with Sematext to optimize the GC settings
and saw initial (and continued) performance boosts as a result...
The situation we're seeing now, has both versions of Solr running on the
same box under the same JVM, but we are undeploying an instance at a time
so
Ah, no, my mistake. The wildcards for the fl list won't matter re:
maxBooleanClauses,
I didn't read carefully enough.
I assume that just returning a field or two doesn't slow down
But one possible culprit, especially since you say this kicks in after
a while, is garbage collection. Here's an
Erick -
Agreed, it is puzzling.
What I've found is that it doesn't matter if I pass in wildcards for the
field list or not...but that the overall response time from the newer builds
of Solr that we've tested (e.g. 4.0.0.2012.02.16) is slower than the older
(4.0.0.2010.12.10.08.54.56) build.
If
It's pretty hard to say, even with the data you've provided. But,
try adding &debugQuery=on and look particularly down near the
bottom there'll be a "" section. That
section lists the time taken by all the components of a search,
not just the QTime. Things like highlighting etc. that can
often give
As an update to this... I tried running a query again the
4.0.0.2010.12.10.08.54.56 version and the newer 4.0.0.2012.02.16 (both on
the same box). So the query params were the same, returned results were the
same, but the 4.0.0.2010.12.10.08.54.56 returned the results in about 1.6
seconds and the
19 matches
Mail list logo
