Re: Sorting performance

2012-06-08 Thread Dmitry Kan
Hi, probably this may help you start: https://issues.apache.org/jira/browse/SOLR-1297 Dmitry On Mon, Jun 4, 2012 at 9:51 PM, Gau wrote: > Here is the usecase: > I am using synonym expansion at query time to get results. this is > essentially a name search, so a search for Jim may be expanded

Re: Sorting performance + replication of index between cores

2009-09-03 Thread Sreeram Vaidyanathan
g it took from 30 mins up to 2 hours). >>> >>> Slave caches are configured to have autowarmCount="0" and >>> maxWarmingSearchers=1 , and I have new data 1 second after snapshoot is >>> done. I haven't noticed any huge delays while serving search r

Re: Sorting performance + replication of index between cores

2009-04-17 Thread sunnyfr
xWarmingSearchers=1 , and I have new data 1 second after snapshoot is >> done. I haven't noticed any huge delays while serving search request. >> Try to use those values - may be they'll help in your case too. >> >> Ben Janicki >> >> >> -Origin

Re: Sorting performance + replication of index between cores

2008-10-27 Thread christophe
From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: 22 October 2008 04:56 To: solr-user@lucene.apache.org Subject: Re: Sorting performance : The problem is that I will have hundreds of users doing queries, and a : continuous flow of document coming in. : So a delay in warming up a cache "co

Re: Sorting performance

2008-10-23 Thread christophe
Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: 22 October 2008 04:56 To: solr-user@lucene.apache.org Subject: Re: Sorting performance : The problem is that I will have hundreds of users doing queries, and a : continuous flow of document coming in. : So a delay in warming up a cache "could&qu

RE: Sorting performance

2008-10-22 Thread Beniamin Janicki
lp in your case too. Ben Janicki -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: 22 October 2008 04:56 To: solr-user@lucene.apache.org Subject: Re: Sorting performance : The problem is that I will have hundreds of users doing queries, and a : continuous flow of d

Re: Sorting performance

2008-10-21 Thread Chris Hostetter
: The problem is that I will have hundreds of users doing queries, and a : continuous flow of document coming in. : So a delay in warming up a cache "could" be acceptable if I do it a few times : per day. But not on a too regular basis (right now, the first query that loads : the cache takes 150s)

Re: Sorting performance

2008-10-21 Thread christophe
I'm now considering if Solr (Lucene) is a good choice when we have a huge number of indexed document and a large number of new documents needs to be indexed everyday. Maybe I'm wrong, but my feeling is that the way the sort caches are handled (recreated after new commit, not shared between Sea

RE: Sorting performance

2008-10-20 Thread Lance Norskog
inal Message- From: Mark Miller [mailto:[EMAIL PROTECTED] Sent: Monday, October 20, 2008 6:24 AM To: solr-user@lucene.apache.org Subject: Re: Sorting performance christophe wrote: > When I start indexing new documents, searches are taking long time > again: is the sort cache flushed when new

Re: Sorting performance

2008-10-20 Thread christophe
The problem is that I will have hundreds of users doing queries, and a continuous flow of document coming in. So a delay in warming up a cache "could" be acceptable if I do it a few times per day. But not on a too regular basis (right now, the first query that loads the cache takes 150s). Howe

Re: Sorting performance

2008-10-20 Thread Norberto Meijome
On Mon, 20 Oct 2008 16:28:23 +0300 christophe <[EMAIL PROTECTED]> wrote: > Hum. this mean I have to wait before I index new documents and avoid > indexing when they are created (I have about 50 000 new documents > created each day and I was planning to make those searchable ASAP). you can a

Re: Sorting performance

2008-10-20 Thread christophe
Hum. this mean I have to wait before I index new documents and avoid indexing when they are created (I have about 50 000 new documents created each day and I was planning to make those searchable ASAP). Too bad there is no way to have a centralized cache that can be shared AND updated when n

Re: Sorting performance

2008-10-20 Thread Mark Miller
christophe wrote: When I start indexing new documents, searches are taking long time again: is the sort cache flushed when new documents are indexed ? When you commit, a new Reader will be opened (or reopened) so that the freshly added docs can be seen. This would make the first search slow a

Re: Sorting performance

2008-10-20 Thread Erick Erickson
Caches are specific to opening a searcher. So whenever you open a reader, the caches are rebuilt for that server. If you are picking up your changes, you MUST be opening a new reader so yes, indeed, your caches are being flushed. You can get around this by firing a few warmup queries at the server

Re: Sorting performance

2008-10-20 Thread christophe
When I start indexing new documents, searches are taking long time again: is the sort cache flushed when new documents are indexed ? Thanks Christophe Mark Miller wrote: You need to setup a warming query that sorts so that the initial long query is done behind the scenes. Users first query wil

Re: Sorting performance

2008-10-20 Thread christophe
Will do so. Thanks. Are there any metrics on how to compute memory requirements (based on doc average size, number of sorted fields, number of indexed documents + number of new document / day) ? Thanks Christophe Mark Miller wrote: You need to setup a warming query that sorts so that the ini

Re: Sorting performance

2008-10-18 Thread Mark Miller
You need to setup a warming query that sorts so that the initial long query is done behind the scenes. Users first query will then be fast. Solrconfig. - Mark On Oct 18, 2008, at 1:34 AM, christophe <[EMAIL PROTECTED]> wrote: Here are the memory parameters I'm using now(Tomcat): -Xms202

Re: Sorting performance

2008-10-18 Thread christophe
Here are the memory parameters I'm using now(Tomcat): -Xms2024m -Xmx2024m With those values, the second query is way faster. Only the first one is very slow. Thanks for the tip. However, I'm wondering if will be enough and I will not hit the same issues when I will have many users searching at

Re: Sorting performance

2008-10-17 Thread christophe
It is slow each time I run it. (I test it from the Solr admin console or from a JAVA program using the Http client). I do not get the OOM each time. Thx Christophe Otis Gospodnetic wrote: Is the sorted query slow only the first time or every time you run it? You got an OOM? What -Xmx value a

Re: Sorting performance

2008-10-17 Thread Otis Gospodnetic
Is the sorted query slow only the first time or every time you run it? You got an OOM? What -Xmx value are you using? Try increasing it. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: christophe <[EMAIL PROTECTED]> > To: solr-user@lucene