On 10/18/2017 2:41 PM, Walter Underwood wrote: > Jetty maxThreads is set to 10,000 which seams way too big. > > The comment suggests 5X the number of CPUs. We have 36 CPUs, which would mean > 180 threads, which seems more reasonable.
I have not seen any evidence that maxThreads at 10000 causes memory issues. The out-of-the-box heap size for all recent releases is 512MB, and Solr starts up just fine with 10000 maxThreads. Most containers (including Jetty and Tomcat) default to a maxThreads value of 200. The Jetty included with Solr has had a setting of 10000 since I first started testing with version 1.4. Users who provide their own containers frequently run into problems where the container will not allow Solr to start the threads it needs to run properly, so they must increase the value. This is a graph of threads on a couple of my Solr servers: https://www.dropbox.com/s/4ux2y3xwvsjjrmt/solr-thread-graph.png?dl=0 The server named bigindy5 (rear graph in the screenshot) is my dev server, running 6.6.2-SNAPSHOT. The server named idxb6 is running 5.3.2-SNAPSHOT and is a backup production server. The dev server has 8 CPU cores without hyperthreading and 24 Solr cores (indexes). Most of those cores have index data in them -- the dev server has copies of *all* my indexes onboard. It has very little activity though -- aside from once-a-minute maintenance updates and a monitoring server, there's virtually no query activity. The production server has 20 CPU cores with hyperthreading (so it looks like 40 to the OS), the same 24 Solr cores, but only a handful of those cores have data, the rest are idle. There's one critical difference in activity for this server compared to the dev server -- four of the cores on the machine are actively indexing from MySQL with the dataimport handler, because I'm doing a full rebuild on that index. Because this server is in a backup role currently, its query load is similar to the dev server -- almost nothing. These servers handle distributed indexes, but they are NOT running in cloud mode. If there were active queries, then more threads would be needed than currently are running. If there were more Solr cores (indexes), then more threads would be needed. My installation is probably bigger than typical, but is definitely not what I would call large. As you can see from the screenshot, these servers have reached thread counts in the 300 range, and are currently sitting at about 250. If I followed that recommendation of 5 threads per CPU, I would configure a value of 40 on the dev server, which wouldn't be anywhere near enough. I've got another server running version 4.7.2 with 8 CPU cores (no hyperthreading) and slightly fewer Solr cores. This is a server that actively gets queries at a fairly low QPS rate. It shows a steady thread count of around 200, with a peak thread count of 1032. That instance of Solr has an uptime of 208 days. Based on what I have seen on my servers, I would not run with maxThreads less than 2000, and I don't see any reason to change it from the provided default of 10000. Thanks, Shawn
