Hi! We're currently facing a strange memory issue we can't explain, so I'd like to kindly ask if anyone is able to shed a light an the behavour we encounter.
We use a Solr 3.5 instance on a Windows Server 2008 machine equipped with 16GB of ram. The index uses 8 cores, 10 million documents, disk size of 180 GB in total. The machine is only used for searches, text extraction is done on another box. We run Solr on Jetty, installed as a service using procrun. The only adjusted JVM parameter is -XX:MaxPermSize=256M When we connect to the Jetty / Solr instance with JVisualVM for debugging purposes we see that MaxHeapSize is selected to 3.8 GB automatically. After approx. 48 hours after startup and frequent search activity, we see the following: The "JettyService.exe" process eats up nearly all available memory. Windows Resource Monitor reports for this process: Working Set (KB) 14.317.768 Shareable (KB) 9.928.032 Private (KB) 4.389.736 This results in machine totals of: Hardware Reserved: 3MB In Use: 16155 MB Modified: 7 MB Standby: 206 MB Free: 13 MB Available: 219 MB Cached: 213 MB Total: 16381 MB Installed: 16384 Actually there's no free memory for other processes left. We are currently quite unsure how to interpret these values. To my mind the total memory limit of Solr should be Max Heap Size + PermSpace + Stack + JVM stuff with JVM stuff and Stack being a few hundred megabytes or so, but not gigabytes. The "private" value, being around 4.3 GB is the size I expected for memory consumption (i.e. 3.8 GB heap + few hundred megs on top for other stuff). Usually I'd expect the huge amount of memory listed as "shareable" to appear under "standby" as well, meaning that it denotes cached memory that is used by java, but might be freed if requested. The small amount of only 213 MB cached memory indicates that there's no headroom left for file system cache etc. My questions: 1) Is the equation Max Heap Size + PermSpace + Stack + JVM stuff roughly correct or is there anything fundamental missing that can consume that much memory? 2) Since the private memory seems about to be ok, can anyone explain why there is such a huge amount of shareable memory used that does not seem to be available to other processed (i.e. locked in any way?!) 3) Is there something Solr-specific that adds to the memory equation, i.e. uses memory but not from the heap pool? Any help is appreciated! Thanks! Greetings Nico
