Patches welcome <G>... But yeah, the fieldCache is pretty much controlled by Lucene. I rather doubt there's a lot of interest in flushing it as you're asking, just because of the concentration on making Solr/Lucene _fast_. Especially if you think about running 10 simultaneous queries sorting on the 10 fields in your example. At least one of them would have to block until enough of the others had completed that it would be OK to flush some values.... it just seems like a nightmare both code-wise and speed-wise.
But then I don't really understand the low-level details well enough to say for sure.... Best Erick On Mon, May 14, 2012 at 8:18 AM, Mathias Hodler <mathias.hod...@gmail.com> wrote: > Hi Erick, > > I'm sorting on 10 different fields (string, date and floats) with 90% > - 100% unique values and 50k indexed documents. > > You're right - cleaning the cache would slow down the next queries > until the fields are cached again. But it would be nice to run Solr on > systems with limited memory and accept the longer query time. > > I don't can say you an exact size of memory I can use for solr because > it should be runnable on different systems. Thats the reason why I > analyzed the behavior of solr on systems with little memory. > > So reducing the number of available sort fields is the only > possibility if you can't increase memory. > > Thanks. > > Mathias > > 2012/5/14 Erick Erickson <erickerick...@gmail.com>: >> But consider what would happen if the cache was cleaned up.... the next >> query in would require that the terms be re-loaded. I guess it's possible >> that some people would be willing to sacrifice speed in constrained >> situations... >> >> Meanwhile, you have two options >> 1> increase memory >> 2> sort on fewer unique values. Have you examined your index to see >> how many _unique_ values are in these fields? And is it possible to >> reduce that number? >> >> Best >> Erick >> >> P.S. How much memory are you working with, and how many unique values >> of what types are you sorting on? And what version of Solr are you using? >> >> On Fri, May 11, 2012 at 5:38 AM, Mathias Hodler >> <mathias.hod...@gmail.com> wrote: >>> Hi, >>> >>> sorting on a field increases the Lucene FieldCache. If I'm starting 10 >>> queries and each query sorting on a different field, 9 queries could >>> be executed but then the Lucene FieldCache exceeds max memory and OOM >>> occurs. >>> In my opinion Lucene Field Cache should be cleaned up if there is not >>> enough memory left. But instead of that, Field Cache will always >>> remains in "Old Generation GC". >>> >>> Could this be fixed or is the only way out to get more memory? >>> >>> Thanks. >>> >>> Mathias
