Here's another test. Add &debugQuery=on to your query and post the results. I believe you'll find that the QTime parameter returned in the packet is quite small. That is the amount of time spent in the query, NOT the time spent reading the docs form disk to return. And if that number remains small whether you search 10 or 2,000 docs, then your problem is just that you're reading 2,000 documents from disk and returning them. And that has nothing to do with sharding, it's just how things work....
Although there's one other possibility: By returning 2,000 rows, you require that each shard assemble a list of the top 2,000 documents and then they are collated into a single packet, so you're asking the system to do a lot of list processing. Also, I really have a hard time seeing what advantage you get from putting all those shards on the same machine, you're just creating extra work. Best Erick On Sat, Dec 17, 2011 at 4:11 PM, ku3ia <dem...@gmail.com> wrote: > Hi, Erick! > >>>Right, are you falling afoul of the recursive shard thing? That is, >>>if you shards point back to itself. As far as I understand, your >>>shards parameter in your request handler shouldn't point back >>>to itself.... > No, my request handler don't points itself cause default is false. But I > tested today, using classic shards parameter. Here is results: > > successfully started at 22:25:58 > Queries interval is: 10 queries per minute > http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(jurisdictions)&rows=2000&start=0&fl=*,score&shards=127.0.0.1:8080/solr/shard1,127.0.0.1:8080/solr/shard2,127.0.0.1:8080/solr/shard3,127.0.0.1:8080/solr/shard4 > ... > http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(mainstreaming)&rows=2000&start=0&fl=*,score&shards=127.0.0.1:8080/solr/shard1,127.0.0.1:8080/solr/shard2,127.0.0.1:8080/solr/shard3,127.0.0.1:8080/solr/shard4 > utility successfully stopped at 22:30:58 > Elapsed time: 300 secs > --- solr --- > Queries processed: 11 > Queries cancelled: 39 > Average QTime is: 55966.8 ms > Average RTime is: 56.2727 sec(s) > Size of data-dir is: 496726 bytes > > And test with RecordID only: > successfully started at 22:17:16 > Queries interval is: 10 queries per minute > http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(simulating)&rows=2000&start=0&fl=RecordID&shards=127.0.0.1:8080/solr/shard1,127.0.0.1:8080/solr/shard2,127.0.0.1:8080/solr/shard3,127.0.0.1:8080/solr/shard4 > ... > http://127.0.0.1:8080/solr/shard1/select/?ident=true&q=(multimillionaire's)&rows=2000&start=0&fl=RecordID&shards=127.0.0.1:8080/solr/shard1,127.0.0.1:8080/solr/shard2,127.0.0.1:8080/solr/shard3,127.0.0.1:8080/solr/shard4 > utility successfully stopped at 22:22:16 > Elapsed time: 300 secs > --- solr --- > Queries processed: 23 > Queries cancelled: 27 > Average QTime is: 13991.8 ms > Average RTime is: 14.1739 sec(s) > Size of data-dir is: 223580 bytes > > And read speed is the same. > > On production I create a RAMDisk (mount -t tmpfs -o size=xxxM tmpfs > /mnt/tmpfs/) and put my shards onto it. The QTime/RTime was magnificent. But > the problem is, I dont have much RAM for it( > > Can it be my huge index, terms or maybe it's to much queries per minute, or > row count or I made something wrong in my configs? And, can you please watch > drive speed on your indexes? Is it the same? > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3594683.html > Sent from the Solr - User mailing list archive at Nabble.com.
