Azazel K <am_tech_mon...@outlook.com> wrote: [Toke: 1 shard instead of multiple?]
> The nodes were unstable when we had single shard setup. > It used to run OOM frequently. Fair enough. [Toke: Use a queue instead of 1000+ concurrent requests?] > There are 16CPU on each node. Requests are live with > upstream client impact so they cannot be put in a queue(!!). Using more threads only helps is there are unused resources. With 4*16CPU cores, I would expect CPU & IO-system to be saturated well before 1000 concurrent requests. Having no real limit on the number of concurrent requests makes is very hard to handle sudden spikes: Unless each call has trivial memory overhead, there must be a lot of free room on the heap for those special occasions. Related: https://issues.apache.org/jira/browse/SOLR-7344 > Also in one of the other threads, Shawn Heisey mentions to increase thread > size to 100000 and tweaking process limit settings on OS. Since you are running distributed searches, the number of ingoing connections (the maxThreads parameter) should be "high enough" to avoid deadlocks. 100K is fine - it is practically the same as infinite as things will break down elsewhere before that number is reached. Limiting the number of concurrent requests to SolrCloud must currently be done from the outside. - Toke Eskildsen
