On 4/13/2016 12:57 AM, Bastien Latard - MDPI AG wrote: > Thank you Shawn & Reth! > > So I have now some questions, again > > > Remind: I have only Solr running on this server (i.e.: java + tomcat). > > /BTW: I needed to increase previously the java heap size because I > went out of memory. Actually, you only see here 2Gb (8Gb previously) > for JVM because I automatically restart tomcat for a better > performance every 30 minutes if no DIH running./
If you size your heap appropriately and properly tune garbage collection, restarting like this should be unnecessary. > Question #1: > From the picture above, we see Physical memory: ~60Gb > * -> is this because of -Xmx40960m AND -XX:MaxPermSize=20480m ? * I don't actually know whether permgen is allocated from the heap, or *in addition* to the heap. Your current allocated heap size is 20GB, which means that at most Java is taking up 30GB, but it might be just 20GB. The other 30-40GB is used by the operating system -- for disk caching (the page cache). It's perfectly normal for physical memory to be almost completely maxed out. The physical memory graph is nearly useless for troubleshooting. Here's a screenshot of one of my servers: https://www.dropbox.com/s/55d4x33tpyyaoff/solr-dashboard-physical-mem.png?dl=0 Notice that the max heap here is 8GB ... yet physical memory has 59GB allocated -- 95 percent. There are some additional java processes taking up a few GB, but the vast majority of the memory is used by the OS page cache. > Question #2: > /"The OS caches the actual index files"./ > > *Does this mean that OS will try to cache 47.48Gb for this index? (if > not, how can I know the size of the cache) > */Or are you speaking about page cache > <https://en.wikipedia.org/wiki/Page_cache>?/ I am talking about the page cache, also known as the disk cache. The OS will potentially use *all* unassigned memory for the page cache. You can ask your operating system how much memory is being used for this purpose. > Question #3: > /"documentCache does live in Java heap" > /*Is there a way to know the real size used/needed by this caching?* Solr does not report memory usage with that much detail. Perhaps one day it will, but we're not there yet. The size of an entry in the documentCache should be approximately the size of the stored data for that document, plus Java overhead required to hold the data. The filterCache is the one that usually uses a large amount of memory. Thanks, Shawn
