Yes and no, The FieldCache is the big culprit. We do a huge amount of faceting so it seems right. Unfortunately I am super swamped at work so I have precious little time to work on this, which is what explains my silence.
Out of desperation, I added another 32G of memory to each server and increased the JVM size to 64G from 25G. The servers are running with 96G memory right now (this is the max amount supported by the hardware) which leaves solr somewhat starved for memory. I am aware of the performance implications of doing this but I have little choice. The extra memory helped a lot, but it still OOM with about 180 clients using it. Unfortunately I need to support at least double that. After upgrading the RAM, I ran for almost two weeks with the same workload that used to OOM a couple of times a day, so it doesn't look like a leak. Today I finally managed to set up a test core so I can begin to play around with docValues. I actually have a couple of questions regarding docValues: 1) If I facet on multible fields and only some of those fields are using docValues, will I still get the memory saving benefit of docValues? (one of the facet fields use null values and will require a lot of work in our product to fix) 2) If i just use docValues on one small core with very limited traffic at first for testing purposes, how can I test that it is actually using the disk for caching? I really appreciate all the help I have received on this list so far. I do feel confident that I will be able to solve this issue eventually. On Mon, Apr 15, 2013 at 9:00 AM, Toke Eskildsen <t...@statsbiblioteket.dk>wrote: > On Sun, 2013-03-24 at 09:19 +0100, John Nielsen wrote: > > Our memory requirements are running amok. We have less than a quarter of > > our customers running now and even though we have allocated 25GB to the > JVM > > already, we are still seeing daily OOM crashes. > > Out of curiosity: Did you manage to pinpoint the memory eater in your > setup? > > - Toke Eskildsen > > -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk
