On 2/2/2018 10:00 AM, mmb1234 wrote:
In our solr non-cloud env., we are seeing lots of CLOSE_WAIT, causing jvm to
stop "working" with 3 mins of solr start.
solr [ /opt/solr ]$ netstat -anp | grep 8983 | grep CLOSE_WAIT | grep
10.xxx.xxx.xxx | wc -l
9453
Solr isn't handling network services. That's being done by the servlet
container. The servlet container included with Solr is Jetty. Since
version 5.0, Jetty is the only option with official support.
There are a number of issues against Jetty where people are seeing many
CLOSE_WAIT problems. At least one of them does acknowledge that there
is a bug in Jetty, but in discussion with them on IRC, they have
apparently never been able to actually reproduce the problem. One issue
says that they don't think Jetty 9.4.x will have the problem, but Solr
won't have Jetty 9.4.x until Solr 7.3.0 is released.
You said that you're running Solr 6.2.2, but there is no 6.2.2 version.
There is 6.4.2 and 6.6.2. Solr 6.6.x and 6.4.x include Jetty 9.3.14,
6.2.x includes Jetty 9.3.8.
Another oddity: Your info says that Solr has a 30GB heap, but the JVM
argument list includes "-Xmx512m" which is a 512MB heap.
One issue that a many open connections causes is a large number of open
file handles, and that can apparently cause software to become
unresponsive. Are you seeing any errors in Solr's logfile before the
process becomes unresponsive?
Thanks,
Shawn
