Re: Debugging Solr memory usage/heap problems

2007-02-07 Thread Otis Gospodnetic
To help find leaks I had good luck with jmap and even jhat in Java 1.6. Otis - Original Message From: Graham Stead <[EMAIL PROTECTED]> To: solr-user@lucene.apache.org Sent: Wednesday, February 7, 2007 12:49:47 AM Subject: RE: Debugging Solr memory usage/heap problems Thanks, Ch

RE: Debugging Solr memory usage/heap problems

2007-02-06 Thread Graham Stead
Thanks, Chris. I will test with vanilla Solr to clarify whether the problem occurs with it, or only in the version where we have made changes. -Graham > : To tweak our scoring, a custom hit collector in > SolrIndexSearcher creates 1 > : fieldCache and 3 ValueSources from 3 fields: > : - an inte

RE: Debugging Solr memory usage/heap problems

2007-02-06 Thread Chris Hostetter
: To tweak our scoring, a custom hit collector in SolrIndexSearcher creates 1 : fieldCache and 3 ValueSources from 3 fields: : - an integer field with many unique values (order 10^4) : - another integer field with many unique values (order 10^4) : - an integer field with hundreds of unique values

RE: Debugging Solr memory usage/heap problems

2007-02-06 Thread Graham Stead
> > Our queries do not sort by any field. However, we do make use of > > FunctionQueries and a typical query is something like: > > > > users_query AND (+linear_function_query > +recip_function_query > > +language:english^0 -flags:spam^0) > > Function queries often build fieldCaches--on

Re: Debugging Solr memory usage/heap problems

2007-02-06 Thread Mike Klaas
On 2/6/07, Graham Stead <[EMAIL PROTECTED]> wrote: Our queries do not sort by any field. However, we do make use of FunctionQueries and a typical query is something like: users_query AND (+linear_function_query +recip_function_query +language:english^0 -flags:spam^0) Function queries

RE: Debugging Solr memory usage/heap problems

2007-02-06 Thread Graham Stead
Mike, Yonik, thanks for the quick reply. > I think it is in your queries. Are you sorting on many > fields? What is a typical query? I'm not a lucene expert, > but there are lucene experts on this list. Our queries do not sort by any field. However, we do make use of FunctionQueries and a

Re: Debugging Solr memory usage/heap problems

2007-02-06 Thread Yonik Seeley
On 2/6/07, Graham Stead <[EMAIL PROTECTED]> wrote: Hi everyone, My Solr JVM runs out of heap space quite frequently. I'm trying to understand Solr/Lucene's memory usage so I can address the problem correctly. Otherwise, I feel I'm taking random shots in the dark. I've tried previous troubleshoo

Re: Debugging Solr memory usage/heap problems

2007-02-06 Thread Mike Klaas
On 2/6/07, Graham Stead <[EMAIL PROTECTED]> wrote: Hi everyone, My Solr JVM runs out of heap space quite frequently. I'm trying to understand Solr/Lucene's memory usage so I can address the problem correctly. Otherwise, I feel I'm taking random shots in the dark. <> 4) In solrconfig.xml, I s