Monitoring this thread make me ask the question of whether there are
standardized performance benchmarks for Solr.
Such that they are run and published with each new release. This would
affirm its performance under known circumstances,
with which people can try in their own environments and compare to their
application behavior.
I think it would be a good idea.
On 11/30/2011 04:12 PM, Pawel Rog wrote:
On Wed, Nov 30, 2011 at 9:05 PM, Chris Hostetter
<hossman_luc...@fucit.org> wrote:
: I tried to use index from 1.4 (load was the same as on index from 3.5)
: but there was problem with synchronization with master (invalid
: javabin format)
: Then I built new index on 3.5 with luceneMatchVersion LUCENE_35
why would you need to re-replicate from the master?
You already have a copy of the Solr 1.4 index on the slave machine where
you are doing testing correct? Just (make sure Solr 1.4 isn't running
and) point Solr 3.5 at that solr home directory for the configs and data
and time that. (Just because Solr 3.5 can't replicate from Solr 1.4
over HTTP doesn't mean it can't open indexes built by Solr 1.4)
I made It before sending earlier e-mail. Efect was the same.
It's important to understand if the discrepencies you are seeing have to
do with *building* the index under Solr 3.5, or *searching* in Solr 3.5.
: reader :
SolrIndexReader{this=8cca36c,r=ReadOnlyDirectoryReader@8cca36c,refCnt=1,segments=4}
: readerDir :
org.apache.lucene.store.NIOFSDirectory@/data/solr_data/itemsfull/index
:
: solr 3.5
: reader :
SolrIndexReader{this=3d01e178,r=ReadOnlyDirectoryReader@3d01e178,refCnt=1,segments=14}
: readerDir :
org.apache.lucene.store.MMapDirectory@/data/solr_data_350/itemsfull/index
: lockFactory=org.apache.lucene.store.NativeFSLockFactory@294ce5eb
As mentioned, the difference in the number of segments may be contributing
to the perf differences you are seeing, so optimizing both indexes (or
doing a partial optimize of your 3.5 index down to 4 segments) for
comparison would probably be worthwhile. (and if that is the entirety of
hte problem, then explicitly configuring a MergePolicy may help you in the
long run)
but independent of that I would like to suggest that you first try
explicitly configuring Solr 3.5 to use NIOFSDirectory so it's consistent
with what Solr 1.4 was doing (I'm told MMapDirectory should be faster, but
maybe there's something about your setup that makes that not true) So it
would be helpful to also try adding this to your 3.5 solrconfig.xml and
testing ...
<directoryFactory name="DirectoryFactory" class="solr.NIOFSDirectoryFactory"/>
: I made some test with quiet heavy query (with frange). In both cases
: (1.4 and 3.5) I used the same newSearcher queries and started solr
: without any load.
: Results of debug timing
Ok, well ... honestly: giving us *one* example of the timing data for
*one* query (w/o even telling us what the exact query was) ins't really
anything we can use to help you ... the crux of the question was: "was the
slow performance you are seeing only under heavy load or was it also slow
when you did manual testing?"
: When I send fewer than 60 rps I see that in comparsion to 1.4 median
: response time is worse, avarage is worse but maximum time is better.
: It doesn't change propotion of cpu usage (3.5 uses much more cpu).
How much "fewer then 60 rps" ? ... I'm trying to understand if the
problems you are seeing are solely happening under "heavy" concurrent
load, or if you are seeing Solr 3.5 consistently respond much slower then
Solr 1.4 even with a single client?
Also: I may still be missunderstanding how you are generating load, and
wether you are throttling the clients, but seeing higher CPU utilization
in Solr 3.5 isn't neccessarily an indication of something going wrong --
in some cases higher CPU% (particularly under heavy concurrent load on a
multi-core machine) could just mean that Solr is now capable of utilizing
more CPU to process parallel request, where as previous versions might have
been hitting other bottle necks. -- but that doesn't explain the slower
response times. that's what concerns me the most.
I don't think that 1200% CPU usage with the same traffic is better
then 200%. I think you are wrong :) Using solr 1.4 I can reach 300rps
and then reach 1200% on cpu and only 60rps in solr 3.5
FWIW: I'm still wondering what the stats from your caches wound up looking
like on both Solr 1.4 and Solr 3.5...
7) What do the cache stats look like on your Solr 3.5 instance after
you've done some of this timing testing? the output of...
http://localhost:8983/solr/admin/mbeans?cat=CACHE&stats=true&wt=json&indent=true
...would be helpful. NOTE: you may need to add this to your
solrconfig.xml
for that URL to work...
<requestHandler name="/admin/" class="solr.admin.AdminHandlers" />'
...but i don't think /admin/mbeans exists in Solr 1.4, so you may just
have to get the details from stats.jsp.
I forgot to write it earlier. QueryCache hit rate was about 0.03 (in
solr 1.4 and 3.5). Filter cache hitrate was abaout 0.35 in both cases.
Document hit rate was about 0.55 in both cases.
Trace from thread wasn't helpful to diagnose problem? As I mentioned
before - almost all threads were in the same line of code in
SolrIndexSearcher.
