Update on the my initial performance findings for off-heap filters: http://heliosearch.org/off-heap-filters/
-Yonik http://heliosearch.org -- making solr shine On Tue, Jan 7, 2014 at 1:53 PM, Yonik Seeley <ysee...@gmail.com> wrote: > Off-Heap Filters: > JVMs have never been good at dealing with large heaps. Large heaps > mean the JVM needs to do a lot of garbage collection work, and often > means some pretty long stop-the-world GC pauses. > > Filters (Solr DocSets) stored in the filterCache are now allocated > off-heap and reference counted so they can be freed as soon as they > are no longer needed. The JVM no longer needs to waste time copying > around these potentially long-lived blocks of memory. This should both > help eliminate the long GC pauses as well as increase request > throughput. > > Performance Results: > I'm still putting together a blog on the results, but they look good! > It was pretty trivial to reproduce >1s stop-the-world GC pauses with a > 4GB heap, and then see those pauses completely go away when I switched > to off-heap filters. Throughput also increased since much less time > was spent doing GC.
