Perhaps a really silly question, but... Is your batch job sending queries serially? In which case this is understandable and would be sensitive to the lower hit ratios in your caches.
If this is the case, you still won't get 2x the performance with two servers, I'm guessing your total time to run the batch job with 2 threads would be closer to 3 minutes rather than 2.5.... Which brings up another question, if you fired 4 threads at your two servers (or 8 or...) I think you'd see something of a throughput increase but it wouldn't be linear once you start getting up past the number of CPU cores available. Which suggests another set of tests.... 1> single node with 2 (or 4 or 8 or whatever) threads 2> same thing with a load balancer. FWIW, Erick On Fri, Oct 10, 2014 at 2:30 AM, Yannick <yann1...@yahoo.com.invalid> wrote: > Hi guys, > > Thanks a lot for the cogent feedback, much appreciated. I'll group my answers > in one message. > > 1- (Charlie): regarding the type of queries I make: here is a typical one; > it's fairly vanilla, no faceting or anything fancy. > > q=+((titleStart:"was sixteen")(subTitleStart:"was > sixteen"))+(participantStart:"fairport convention")&fq=+(type:COCV) > > 2- (Erick): correct, my index is not sharded, and there is no SolrCloud for > the test. In a production env, I will setup a SolrCloud, but I think I need > to understand what's going on here first. It's a medium-size index (30 > million docs). I did a sharded test and the performance was worse on 2 > servers / 2 shards, than 2 servers / same single shard on each. > > 3- (Walter): regarding the cache situation : you may be on to something; this > is what I observed: > > document cache: hitrate 0.38 (1 server condition) / 0.26 (on each of the 2 > servers) > filter cache : hitrate 1 in both conditions > query result cache : hitrate 0.24 (1 server) / 0.14 (on each of the 2 servers) > > Is it possible this is the issue? My batch issues ~300,000 queries, > corresponding to ~150,000 unique queries. > > Will look into Varnish, thanks for the pointer. > > Yannick
