On Thu, Sep 16, 2010 at 11:48 AM, Peter Karich <peat...@yahoo.de> wrote:
> Hi Kjetil, > > is this custom component (which performes groub by + calcs stats) > somewhere available? > I would like to do something similar. Would you mind to share if it > isn't already available? > > The grouping stuff sounds similar to > https://issues.apache.org/jira/browse/SOLR-236 > > where you can have mem problems too ;-) or see: > https://issues.apache.org/jira/browse/SOLR-1682 > > Thanks for the links! These patches seem to provide somewhat similar functionality, I'll investigate if they're implemented in a similar way too. We've developed this component for a client, so while I'd like to share it I can't make any promises. Sorry. > > Any tips or similar experiences? > > you want to decrease memory usage? Yes. Specifically, I would like to keep the heap at 4 GB. Unfortunately I'm still seeing some OutOfMemoryErrors so I might have to up the heap size again. I guess what I'm really wondering is if there's a way to keep memory use down, while at the same time not sacrificing the performance of our queries. The queries have to run through all values for a field in order to calculate the sum, so it's not enough to just cache a few values. The code which fetches values from the index uses FieldCache.DEFAULT.getStringIndex for a field, and then indexes like this: FieldType fieldType = searcher.getSchema().getFieldType(fieldName); fieldType.indexedToReadable(stringIndex.lookup[stringIndex.order[documentId]]); Is there a better way to do this? Thanks. ---Kjetil
