Hendrik: There are a limited number of threads that load in parallel when starting up, depends on the configuration. The defaults are 3 threads in stand-alone and 8 in Cloud (see: NodeConfig.java)
public static final int DEFAULT_CORE_LOAD_THREADS = 3; public static final int DEFAULT_CORE_LOAD_THREADS_IN_CLOUD = 8; Not sure whether that's germane, but thought it worth mentioning just in case. Best, Erick On Sun, Mar 18, 2018 at 1:11 PM, Hendrik Haddorp <hendrik.hadd...@gmx.net> wrote: > I increased the metaspace size to 2GB. This way I could do multiple rounds > of reloading all collections already. The GC logs do show now an almost > stable metaspace size. So maybe I did just set the limits too low. Still a > bit odd that reloading the collections results in a higher memory usage. > Shouldn't all collections be loaded during the startup? > > On 18.03.2018 17:22, Hendrik Haddorp wrote: >> >> Hi, >> >> I did a simple test on a three node cluster using Solr 7.2.1. The JVMs >> (Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_162 25.162-b12) >> have about 6.5GB heap and 1.5GB metaspace. In my test I have 1000 >> collections with only 1000 simple documents each. I'm then triggering >> collections reloads via SolrJ using a fixed number of threads, as this has >> shown memory issues in the past. Even with two threads the nodes eventually >> die with an OOM Error as they are running out of metaspace. I found the >> following Jiras that might be about the same issue: >> https://issues.apache.org/jira/browse/SOLR-10506 >> https://issues.apache.org/jira/browse/SOLR-9117 >> https://issues.apache.org/jira/browse/SOLR-6678 >> >> The first two are flagged as fixed in 7.0. >> >> Any ideas, beside not doing reloads? >> >> regards, >> Hendrik > >
