Take a look at using DocValues for facets that are problematic. It not only
moves the memory off-heap, but stores values in a much more optimal manner.
-- Jack Krupansky
-----Original Message-----
From: Toke Eskildsen
Sent: Thursday, June 20, 2013 3:26 AM
To: solr-user@lucene.apache.org
Subject: RE: yet another optimize question
Petersen, Robert [robert.peter...@mail.rakuten.com] wrote:
We actually have hundreds of facet-able fields, but most are specialized
and are only faceted upon if the user has drilled into the particular
category
to which they are applicable and so they are only indexed for products
in those categories. I guess it is the facets that eat up so much of our
memory.
As Andre mentions, the problem is that the fc facet method maintains a list
of values (or pointers to values, if we're talking text) for each document
in the whole index. Faceting on a field that only has a single value in a
single document in the whole index still allocates memory linear to the
total number of documents. You are in the same situation as John Nielsen in
the thread "Solr using a ridiculous amount of memory"
http://lucene.472066.n3.nabble.com/Solr-using-a-ridiculous-amount-of-memory-tt4050840.html#none
You could try and change the way you index the facet information to get
around this waste, but it is quite a lot of work:
http://sbdevel.wordpress.com/2013/04/16/you-are-faceting-itwrong/
It was suggested that if I use facet method = enum for those particular
specialized facets then my memory usage would go down.
If the number of unique values in the individual facets is low, this could
work. If nothing else, it is very easy to try.
- Toke Eskildsen=
