RE: Performance help for heavy indexing workload

2008-02-12 Thread Lance Norskog
an impressive app. Lance -Original Message- From: James Brady [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 12, 2008 12:41 PM To: solr-user@lucene.apache.org Subject: Re: Performance help for heavy indexing workload Hi - thanks to everyone for their responses. A couple of extra pieces

Re: Performance help for heavy indexing workload

2008-02-12 Thread James Brady
Hi - thanks to everyone for their responses. A couple of extra pieces of data which should help me optimise - documents are very rarely updated once in the index, and I can throw away index data older than 7 days. So, based on advice from Mike and Walter, it seems my best option will be t

Re: Performance help for heavy indexing workload

2008-02-12 Thread Mike Klaas
On 11-Feb-08, at 11:38 PM, James Brady wrote: Hello, I'm looking for some configuration guidance to help improve performance of my application, which tends to do a lot more indexing than searching. At present, it needs to index around two documents / sec - a document being the stripped c

Re: Performance help for heavy indexing workload

2008-02-12 Thread Walter Underwood
On 2/12/08 7:40 AM, "Ken Krugler" <[EMAIL PROTECTED]> wrote: > In general immediate updating of an index with a continuous stream of > new content, and fast search results, work in opposition. The > searcher's various caches are getting continuously flushed to avoid > stale content, which can easi

Re: Performance help for heavy indexing workload

2008-02-12 Thread Walter Underwood
ring by a user_id field. >> >> This seems to be the case for every sort option except score asc and >> score desc. Please tell me Solr doesn't sort all matching documents >> before applying boolean filters? >> >> James >> >> Begin forwarded message: &

Re: Fwd: Performance help for heavy indexing workload

2008-02-12 Thread Ken Krugler
Hi James, I'm looking for some configuration guidance to help improve performance of my application, which tends to do a lot more indexing than searching. At present, it needs to index around two documents / sec - a document being the stripped content of a webpage. However, performance was

Re: Performance help for heavy indexing workload

2008-02-12 Thread Erick Erickson
; above) and filtering by a user_id field. > > This seems to be the case for every sort option except score asc and > score desc. Please tell me Solr doesn't sort all matching documents > before applying boolean filters? > > James > > Begin forwarded message: > >

Fwd: Performance help for heavy indexing workload

2008-02-12 Thread James Brady
essage: From: James Brady <[EMAIL PROTECTED]> Date: 11 February 2008 23:38:16 GMT-08:00 To: solr-user@lucene.apache.org Subject: Performance help for heavy indexing workload Hello, I'm looking for some configuration guidance to help improve performance of my application, which tends to

Performance help for heavy indexing workload

2008-02-11 Thread James Brady
Hello, I'm looking for some configuration guidance to help improve performance of my application, which tends to do a lot more indexing than searching. At present, it needs to index around two documents / sec - a document being the stripped content of a webpage. However, performance was so