Re: Slow performance on distributed search

2013-03-26 Thread Erick Erickson
ssage- > From: qungg [mailto:qzheng1...@gmail.com] > Sent: Tuesday, March 26, 2013 2:55 PM > To: solr-user@lucene.apache.org > Subject: RE: Slow performance on distributed search > > for start=100,000&row=10. event though each individual shard take only < 10ms > to qu

RE: Slow performance on distributed search

2013-03-26 Thread Michael Ryan
olr-user@lucene.apache.org Subject: RE: Slow performance on distributed search for start=100,000&row=10. event though each individual shard take only < 10ms to query, the merging process done by controller would take about a minutes. By looking at logs, each shard is giving the controller shard

Re: Slow performance on distributed search

2013-03-26 Thread Michael Della Bitta
From: Walter Underwood >> Sent: Tuesday, March 26, 2013 3:47 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Slow performance on distributed search >> >> Why on earth are you starting at row 100,000? What use case is hat? --wunder >> >> On

Re: Slow performance on distributed search

2013-03-26 Thread Walter Underwood
, 2013, at 1:14 PM, Jack Krupansky wrote: > (You mean, other than "deep paging".) > > -- Jack Krupansky > > -Original Message- From: Walter Underwood > Sent: Tuesday, March 26, 2013 3:47 PM > To: solr-user@lucene.apache.org > Subject: Re: Slow performance

Re: Slow performance on distributed search

2013-03-26 Thread Jack Krupansky
(You mean, other than "deep paging".) -- Jack Krupansky -Original Message- From: Walter Underwood Sent: Tuesday, March 26, 2013 3:47 PM To: solr-user@lucene.apache.org Subject: Re: Slow performance on distributed search Why on earth are you starting at row 100,000? What u

Re: Slow performance on distributed search

2013-03-26 Thread Walter Underwood
Why on earth are you starting at row 100,000? What use case is that? --wunder On Mar 26, 2013, at 11:55 AM, qungg wrote: > for start=100,000&row=10. event though each individual shard take only < 10ms > to query, the merging process done by controller would take about a minutes. > > By looking

Re: Slow performance on distributed search

2013-03-26 Thread Joel Bernstein
Take a look at this jira: https://issues.apache.org/jira/browse/SOLR-659 I have not tried this out myself but it looks promising. On Tue, Mar 26, 2013 at 2:55 PM, qungg wrote: > for start=100,000&row=10. event though each individual shard take only < > 10ms > to query, the merging process don

RE: Slow performance on distributed search

2013-03-26 Thread qungg
for start=100,000&row=10. event though each individual shard take only < 10ms to query, the merging process done by controller would take about a minutes. By looking at logs, each shard is giving the controller shard 100,010 rows of data, and because there are 40 shards in total, the controller i

RE: Slow performance on distributed search

2013-03-26 Thread Michael Ryan
What are the values of the start and rows parameters you are using? When you say the controller shard takes a long time, how long is it taking - 100ms, 1s, 10s...? -Michael -Original Message- From: qungg [mailto:qzheng1...@gmail.com] Sent: Tuesday, March 26, 2013 11:17 AM To: solr-user

Re: Slow performance on distributed search

2013-03-26 Thread qungg
Thank you for reply. Queries do need to go to all 40 shards, though 40 shards is not a final number, my setup can be changed if search time decreases. Im using 40 shards because we have a tool that can index faster with more shards. -- View this message in context: http://lucene.472066.n3.nabb

Re: Slow performance on distributed search

2013-03-26 Thread Otis Gospodnetic
Hi, Does your query really need to search all 40 shards? If not, dispatching the query only to shards that need to be queried will help. Otis -- SOLR Performance Monitoring - http://sematext.com/spm/index.html On Tue, Mar 26, 2013 at 11:17 AM, qungg wrote: > Hi, > > I have 40 shards runnin