The thing that jumps out at me is "&rows=2000". If you documentCache in solrconfig.xml is still the defaults, it only holds 512. So you're running all over your disk gathering up the fields to return, especially since you also specified "fl=*,score". And if you have large fields stored, you're doing an awful lot of disk reading.....
simple tests to see if this is on the right track, try these, singly and in combination. 1> try with "&rows=10" 2> try with "&fl=id" assuming "id" is your <uniqueKey> Best Erick On Thu, Dec 15, 2011 at 5:00 PM, ku3ia <dem...@gmail.com> wrote: > Hi, all! > > I have a problem with distributed search. I downloaded one shard from my > production. It has: > * ~29M docs > * 11 fields > * ~105M terms > * size of shard is: 13GB > On production there are near 30 the same shards. I split this shard to 4 > more smaller shards, so now I have: > small shard1: > docs: 6.2M > terms: 27.2M > size: 2.89GB > small shard2: > docs: 6.3M > terms: 28.7M > size: 2.98GB > small shard3: > docs: 7.9M > terms: 32.8M > size: 3.60GB > small shard4: > docs: 8.2M > terms: 32.6M > size: 3.70GB > > My machine confguration: > ABIT AX-78 > AMD Athlon 64 X2 5200+ > DDR2 Kingston 2x2G+2x1G = 6G > WDC WD2500JS (System here) > WDC WD20EARS (6 partitions = 30 GB for shards at begin of drive, and other > empty, all partitions are well aligned) > GNU/Linux Debian Squeeze > Tomcat 6.0.32 with JAVA_OPTS: > JAVA_OPTS="$JAVA_OPTS -XX:+DisableExplicitGC -server \ > -XX:PermSize=512M -XX:MaxPermSize=512M -Xmx4096M -Xms4096M > -XX:NewSize=128M -XX:MaxNewSize=128M \ > -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled \ > -XX:CMSInitiatingOccupancyFraction=50 -XX:GCTimeRatio=9 > -XX:MinHeapFreeRatio=25 -XX:MaxHeapFreeRatio=25 \ > -verbose:gc -XX:+PrintGCTimeStamps -Xloggc:$CATALINA_HOME/logs/gc.log" > Solr 3.5 > > I configured 4 cores and start Tomcat. I write a bash script. It's runing > during 300 seconds and sending every 6 seconds queries like > http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(assistants)&rows=2000&start=0&fl=*,score&qt=requestShards > where qt=requestShards is my 4 shards. After test I have the results: > > Elapsed time: 299 secs > --- solr --- > Queries processed: 21 << this is full response file > Queries cancelled: 29 << this is number of killed curls > Average QTime is: 59645.6 ms > Average RTime is: 59.7619 sec(s) << this is average time difference between > start and stop the curl. There is a part of script: > # >>dcs=`date +%s` > # >>curl ${url} -s -H 'Content-type:text/xml; charset=utf-8' > > ${F_DATADIR}/$dest.fdata > # >>dce=`date +%s` > # >>dcd=$(echo "$dce - $dcs" | bc) > Size of data-dir is: 3346766 bytes << this is response dir size > > I'm using nmon to to monitor R/W disk speed, and I was surprised that read > speed of my shards volumes WDC20EAR's drive was nearly 3 MB/s when script is > working. After this I run benchmark test from disk utility. Here is results: > Minimum read rate: 53.2MB/s > Maximum Read rate: 126.4 MB/s > Average Read rate: 95.8 MB/s > > But from the other side I tested queries like > http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(assistants)&rows=2000&start=0&fl=*,score > results is: > Elapsed time: 299 secs > --- solr --- > Queries processed: 50 > Queries cancelled: 0 > Average QTime is: 139.76 ms > Average RTime is: 2.2 sec(s) > Size of data-dir is: 6819259 bytes > > and quesries like > http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(assistants)&rows=2000&start=0&fl=*,score&shards=127.0.0.1:8080/solr/shard1 > and result is: > Elapsed time: 299 secs > --- solr --- > Queries processed: 49 > Queries cancelled: 1 > Average QTime is: 1878.37 ms > Average RTime is: 1.95918 sec(s) > Size of data-dir is: 4274099 bytes > So we see the results are the same. > > My big question is: why is so slow drive read speed when Solr is working? > Thanks for any replies > > P.S. And maybe my general problem is too much terms in shard, for example, > query > http://127.0.0.1:8080/solr/shard1/terms?terms.fl=field1 > shows: > <lst name="field1"> > <int name="a">58641</int> > <int name="the">45022</int> > <int name="i">36339</int> > <int name="s">35637</int> > <int name="d">34247</int> > <int name="m">33869</int> > <int name="b">28961</int> > <int name="r">28147</int> > <int name="e">27654</int> > <int name="n">26940</int> > </lst> > > Thanks. > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3590028.html > Sent from the Solr - User mailing list archive at Nabble.com.
