Thanks, guys. Now I know a little more about DocValues and realize that
they will do the job wrt FieldCache.
Regards, Per Steffensen
On 9/12/13 3:11 AM, Otis Gospodnetic wrote:
Per, check zee Wiki, there is a page describing docvalues. We used them
successfully in a solr for analytics scenario.
Otis
Solr & ElasticSearch Support
http://sematext.com/
On Sep 11, 2013 9:15 AM, "Michael Sokolov" <msoko...@safaribooksonline.com>
wrote:
On 09/11/2013 08:40 AM, Per Steffensen wrote:
The reason I mention sort is that we in my project, half a year ago, have
dealt with the FieldCache->OOM-problem when doing sort-requests. We
basically just reject sort-requests unless they hit below X documents - in
case they do we just find them without sorting and sort them ourselves
afterwards.
Currently our problem is, that we have to do a group/distinct (in
SQL-language) query and we have found that we can do what we want to do
using group
(http://wiki.apache.org/solr/**FieldCollapsing<http://wiki.apache.org/solr/FieldCollapsing>)
or facet - either will work for us. Problem is that they both use
FieldCache and we "know" that using FieldCache will lead to OOM-execptions
with the amount of data each of our Solr-nodes administrate. This time we
have really no option of just "limit" usage as we did with sort. Therefore
we need a group/distinct-functionality that works even on huge data-amounts
(and a algorithm using FieldCache will not)
I believe setting facet.method=enum will actually make facet not use the
FieldCache. Is that true? Is it a bad idea?
I do not know much about DocValues, but I do not believe that you will
avoid FieldCache by using DocValues? Please elaborate, or point to
documentation where I will be able to read that I am wrong. Thanks!
There is Simon Willnauer's presentation http://www.slideshare.net/**
lucenerevolution/willnauer-**simon-doc-values-column-**
stride-fields-in-lucene<http://www.slideshare.net/lucenerevolution/willnauer-simon-doc-values-column-stride-fields-in-lucene>
and this blog post http://blog.trifork.com/2011/**
10/27/introducing-lucene-**index-doc-values/<http://blog.trifork.com/2011/10/27/introducing-lucene-index-doc-values/>
and this one that shows some performance comparisons:
http://searchhub.org/2013/04/**02/fun-with-docvalues-in-solr-**4-2/<http://searchhub.org/2013/04/02/fun-with-docvalues-in-solr-4-2/>
