Re: Facets with 5000 facet fields - Out of memory error during the query time

2013-04-25 Thread sivaprasad
I got more information with the responses.Now, It's time to re look into the number of facets to be configured. Thanks, Siva http://smarttechies.wordpress.com/ -- View this message in context: http://lucene.472066.n3.nabble.com/Facets-with-5000-facet-fields-Out-of-memory-error-during-the-quer

Re: Facets with 5000 facet fields

2013-03-21 Thread Andy
solr-user@lucene.apache.org Sent: Thursday, March 21, 2013 9:04 AM Subject: Re: Facets with 5000 facet fields as was said below, add facet.method=fcs to your query URL. Upayavira On Thu, Mar 21, 2013, at 09:41 AM, Andy wrote: > What do I need to do to use this new per segment fa

Re: Facets with 5000 facet fields

2013-03-21 Thread Upayavira
apache.org > Sent: Wednesday, March 20, 2013 1:09 PM > Subject: Re: Facets with 5000 facet fields > > > On Mar 20, 2013, at 11:29 AM, Chris Hostetter > wrote: > > > Not true ... per segment FIeldCache support is available in solr > > faceting, you just have to sp

Re: Facets with 5000 facet fields

2013-03-21 Thread Andy
What do I need to do to use this new per segment faceting method? From: Mark Miller To: solr-user@lucene.apache.org Sent: Wednesday, March 20, 2013 1:09 PM Subject: Re: Facets with 5000 facet fields On Mar 20, 2013, at 11:29 AM, Chris Hostetter wrote

Re: Facets with 5000 facet fields

2013-03-21 Thread John Nielsen
It looks like docvalues might solve a problem we have. (sorry for the thread jacking) I looked for info on it on the wiki, but could not find any. Is there any documentation done on it yet? On Wed, Mar 20, 2013 at 6:09 PM, Mark Miller wrote: > > On Mar 20, 2013, at 11:29 AM, Chris Hostetter

Re: Facets with 5000 facet fields

2013-03-20 Thread Mark Miller
On Mar 20, 2013, at 11:29 AM, Chris Hostetter wrote: > Not true ... per segment FIeldCache support is available in solr > faceting, you just have to specify facet.method=fcs (FieldCache per > Segment) Also, if you use docvalues in 4.2, Robert tells me it is uses a new per seg faceting method

Re: Facets with 5000 facet fields

2013-03-20 Thread Chris Hostetter
: > I seem to recall that facet cache is not per segment so every time the : > index is updated the facet cache will need to be re-computed. : : That is correct. We haven't experimented with segment based faceting Not true ... per segment FIeldCache support is available in solr faceting, you ju

Re: Facets with 5000 facet fields

2013-03-20 Thread Toke Eskildsen
On Wed, 2013-03-20 at 10:12 +0100, Andy wrote: > Are you doing NRT updates? No. Startup/re-open time is around 1 minute for the Solr instance, but due to we are currently doing nightly updates only. > I seem to recall that facet cache is not per segment so every time the > index is updated the f

Re: Facets with 5000 facet fields

2013-03-20 Thread Andy
at problem? From: Toke Eskildsen To: "solr-user@lucene.apache.org" ; Andy Sent: Wednesday, March 20, 2013 4:06 AM Subject: Re: Facets with 5000 facet fields On Wed, 2013-03-20 at 07:19 +0100, Andy wrote: > What about the case where there's only a small number of fields (a &g

Re: Facets with 5000 facet fields

2013-03-20 Thread Toke Eskildsen
On Wed, 2013-03-20 at 07:19 +0100, Andy wrote: > What about the case where there's only a small number of fields (a > dozen or two) but each field has hundreds of thousands or millions of > values? Would Solr be able to handle that? We do that on a daily basis at State and University Library, Denm

Re: Facets with 5000 facet fields

2013-03-19 Thread Andy
esday, March 19, 2013 6:09 PM Subject: Re: Facets with 5000 facet fields : In order to support faceting, Solr maintains a cache of the faceted : field. You need one cache for each field you are faceting on, meaning : your memory requirements will be substantial, unless, I guess, your 1) you can con

Re: Facets with 5000 facet fields

2013-03-19 Thread Chris Hostetter
: In order to support faceting, Solr maintains a cache of the faceted : field. You need one cache for each field you are faceting on, meaning : your memory requirements will be substantial, unless, I guess, your 1) you can consider trading ram for time by using "facet.method=enum" (and disabling

RE: Facets with 5000 facet fields

2013-03-19 Thread Toke Eskildsen
Toke Eskildsen [t...@statsbiblioteket.dk] wrote: [Solr, 11M documents, 5000 facet fields, 12GB RAM, OOM] > 5000 fields @ 9 MByte is about 45GB for faceting. > If you are feeling really adventurous, take a look at > https://issues.apache.org/jira/browse/SOLR-2412 I tried building a test-index wi

Re: Facets with 5000 facet fields

2013-03-18 Thread Toke Eskildsen
On Mon, 2013-03-18 at 08:34 +0100, sivaprasad wrote: > We have configured solr for 5000 facet fields as part of request handler.We > have 10811177 docs in the index. > > The solr server machine is quad core with 12 gb of RAM. > > When we are querying with facets, we are getting out of memory erro

Re: Facets with 5000 facet fields

2013-03-18 Thread Upayavira
I'd be very surprised if this were to work. I recall one situation in which 24 facets in a request placed too much pressure on the server. In order to support faceting, Solr maintains a cache of the faceted field. You need one cache for each field you are faceting on, meaning your memory requireme