Hi! (Solr 7.6 , Tlog replicas) I have an issue while reloading collection with 100 shards and 3 replicas per shard residing on 5 nodes. Configuration of that collection is pretty complex (90 external file fields) When node starts cores load always successfully.
When I reload collection with collection api command: /admin/collections?action=RELOAD&name=col all 5 nodes stop responding and I have dead cluster. Only restarting solr on all nodes revives it. When I decreased number of shards/cores by 5 times (to 20 shards instead of 100) Collection reloaded successfully. My guess is that during Collection RELOAD , limit on threads is not honored and all cores try to reload simultaneously. Erick wrote here ( http://lucene.472066.n3.nabble.com/collection-reload-leads-to-OutOfMemoryError-td4380754.html#a4380791 ) ➢ There are a limited number of threads that load in parallel when ➢ starting up, depends on the configuration. The defaults are 3 threads ➢ in stand-alone and 8 in Cloud (see: NodeConfig.java) ➢ ➢ public static final int DEFAULT_CORE_LOAD_THREADS = 3; ➢ public static final int DEFAULT_CORE_LOAD_THREADS_IN_CLOUD = 8; But unfortunately stumbling about source I can't find out the place and approve whether these "threads limit" plays any role in reload collection or not... though I lack the necessary skills in java Maybe somebody can give a hint where to look? There was discussion here as well http://lucene.472066.n3.nabble.com/Solr-reload-process-flow-td4379966.html#none -- Vadim
