Any ideas?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p4106968.html
Sent from the Solr - User mailing list archive at Nabble.com.
on merge Ids:
resultIds.put(shardDoc.id.toString(), shardDoc);
So seems it try to use stored "id", not docvalue? I tried different codecs:
Direct, Disk, Memory, Lucene45. Results are nearby the same.
Can you advise me, may be I'm wrong in my configs or I'm not understand when
us
On Wed, Dec 28, 2011 at 5:47 AM, ku3ia wrote:
> So, based on p.2) and on my previous researches, I conclude, that the more
> documents I want to retrieve, the slower is search and main problem is the
> cycle in writeDocs method. Am I right? Can you advice something in this
> situation?
For the fi
72066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3616192.html
Sent from the Solr - User mailing list archive at Nabble.com.
e on any keyword the
situations is similar.
Thanks.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3605074.html
Sent from the Solr - User mailing list archive at Nabble.com.
: I had a similar requirement in my project, where a user might ask for up
: to 3000 results. What I did was change SolrIndexSearcher.doc(int, Set)
: to retrieve the unique key from the field cache instead of retrieving it
: as a stored field from disk. This resulted in a massive speed
: impro
: So why do you have this 2,000 requirement in the first
: place? This really sounds like an XY problem.
I would really suggest re-visiting this question. No sinle user is going
to look at 2000 docs on a single page, and in your previous email you said
there was a requirement to ask solr for 2
: For example I have 4 shards. Finally, I need 2000 docs. Now, when I'm using
:
&shards=127.0.0.1:8080/solr/shard1,127.0.0.1:8080/solr/shard2,127.0.0.1:8080/solr/shard3,127.0.0.1:8080/solr/shard4
: Solr gets 2000 docs from each shard (shard1,2,3,4, summary we have 8000
: docs) merge and sort it,
t's true. But from the other side, the second month I'm trying to
resolve this problem on my huge data, so I'm attempting to raise any ideas,
maybe, even these ideas are incorrect.
Thanks.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-dis
On Mon, Dec 19, 2011 at 01:32:22PM -0800, ku3ia wrote:
> >>Uhm, either I misunderstand your question or you're doing
> >>a lot of extra work for nothing
>
> >>The whole point of sharding it exactly to collect the top N docs
> >>from each shard and merge them into a single result [...]
> >>
f I recall correctly).
>
> -Michael
>
Thanks Michel, I'll try it.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3599752.html
Sent from the Solr - User mailing list archive at Nabble.com.
I had a similar requirement in my project, where a user might ask for up to
3000 results. What I did was change SolrIndexSearcher.doc(int, Set) to retrieve
the unique key from the field cache instead of retrieving it as a stored field
from disk. This resulted in a massive speed improvement for t
have 2000 rows
(maybe less), but not 8000... That was my question.
Thanks.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3599636.html
Sent from the Solr - User mailing list archive at Nabble.com.
(2000/4=500) and sends to each shard queries with rows=500, but not
rows=2000, so finally, summary after merging and sorting I'll have 2000 rows
(maybe less), but not 8000... That was my question.
Thanks.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3599636.html
Sent from the Solr - User mailing list archive at Nabble.com.
g
>>>extra work.
> Yeah, on my production I have 5 servers and 6 shards (big shards) on each.
> But I tried to use only one shard for each server (summary five shards) but
> results wasn't fine.
>
>>>Although there's one other possibility: By returning 2,000 ro
ined at query filed or score and pull result to
the user?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3597893.html
Sent from the Solr - User mailing list archive at Nabble.com.
nto it. The QTime/RTime was magnificent. But
> the problem is, I dont have much RAM for it(
>
> Can it be my huge index, terms or maybe it's to much queries per minute, or
> row count or I made something wrong in my configs? And, can you please watch
> drive speed on your indexes? Is it the same?
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3594683.html
> Sent from the Solr - User mailing list archive at Nabble.com.
my shards onto it. The QTime/RTime was magnificent. But
the problem is, I dont have much RAM for it(
Can it be my huge index, terms or maybe it's to much queries per minute, or
row count or I made something wrong in my configs? And, can you please watch
drive speed on your indexes?
>
>
> explicit
> 10
> name="shards">127.0.0.1:8080/solr/shard1,127.0.0.1:8080/solr/shard2,127.0.0.1:8080/solr/shard3,127.0.0.1:8080/solr/shard4
>
>
>
> This request handler is defined at shard1's solrconfig.
>
> --
27.0.0.1:8080/solr/shard2,127.0.0.1:8080/solr/shard3,127.0.0.1:8080/solr/shard4
This request handler is defined at shard1's solrconfig.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3592734.html
Sent from the Solr - User mailing list archive at Nabble.com.
e: 301 secs
> --- solr ---
> Queries processed: 15
> Queries cancelled: 35
> Average QTime is: 52775.7 ms
> Average RTime is: 53.2667 sec(s)
> Size of data-dir is: 212978 bytes
>
> In first test disk usage by nmon: ~30-40% and in the second - 100%. Drive
> read speed start
the second - 100%. Drive
read speed starting from 3-5 MB/s and falls to 500-700 KB/s in both tests.
Have you any ideas?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3592364.html
Sent from the Solr - User mailing list archive at Nabble.com.
s: 1.95918 sec(s)
> Size of data-dir is: 4274099 bytes
> So we see the results are the same.
>
> My big question is: why is so slow drive read speed when Solr is working?
> Thanks for any replies
>
> P.S. And maybe my general problem is too much terms in shard, for example,
> query
> http://127.0.0.1:8080/solr/shard1/terms?terms.fl=field1
> shows:
>
> 58641
> 45022
> 36339
> 35637
> 34247
> 33869
> 28961
> 28147
> 27654
> 26940
>
>
> Thanks.
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3590028.html
> Sent from the Solr - User mailing list archive at Nabble.com.
r any replies
P.S. And maybe my general problem is too much terms in shard, for example,
query
http://127.0.0.1:8080/solr/shard1/terms?terms.fl=field1
shows:
58641
45022
36339
35637
34247
33869
28961
28147
27654
26940
Thanks.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Poor-performance-on-distributed-search-tp3590028p3590028.html
Sent from the Solr - User mailing list archive at Nabble.com.
24 matches
Mail list logo
