Thanks! I understand now, let's focus on next question. I configured the "maxConnections" and "maxConnectionsPerHost" in solr.xml as you said before. But I got the same failure result. Looks like the http connection pool is exhausted and most threads are waiting to get a free connection. the configuration is as below
<shardHandlerFactory name="shardHandlerFactory" class="HttpShardHandlerFactory"> <int name="socketTimeout">60000</int> <int name="connTimeout">30000</int> <int name="maxConnections">10000</int> <int name="maxConnectionsPerHost">500</int> </shardHandlerFactory> I posted the problem in JIRA before, the link is https://issues.apache.org/jira/browse/SOLR-9253 I think you still remember that. you can get the java core log in the attachments. please help me analysis it. 2016-07-04 13:13 GMT+08:00 Shalin Shekhar Mangar <shalinman...@gmail.com>: > You are asking many different questions. Lets take them up one by one: > > You say that you use the singleton pattern to create and get the Solr > server connection. Yes, that is fine. The CloudSolrServer is thread safe > and can be shared across requests/threads. When your application exits, you > should shutdown that object. > > Once you shutdown ZooKeeper, Solr cannot process any updates. This is by > design. A running and functional Zookeeper is necessary for SolrCloud. > > On Fri, Jul 1, 2016 at 2:15 PM, 穆 俊峰 <kent...@live.cn> wrote: > > > Dear Mr. Experts! > > > > > > I came across a problem when use solrcloud, it goes down. The version is > > 4.9.1. > > > > we got the java core log, it looks like the http connection pool is > > exhausted and most threads are waiting to get a free connection. > > > > I have increased http connection defaults for the SolrJ client, and also > > configed the connection defaults in solr.xml for all shard servers as > below. > > > > <shardHandlerFactory name="shardHandlerFactory" > > class="HttpShardHandlerFactory"> > > <int name="socketTimeout">60000</int> > > <int name="connTimeout">30000</int> > > <int name="maxConnections">10000</int> > > <int name="maxConnectionsPerHost">500</int> > > </shardHandlerFactory> > > > > > > > > the java stack as below > > > > "httpShardExecutor-3-thread-541" prio=10 tid=0x00007f7b1c02b000 > nid=0x20af > > waiting on condition [0x00007f79fd490000] > > java.lang.Thread.State: WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x0000000605710068> (a > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) > > at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:133) > > at > > > org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:282) > > at > > > org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64) > > at > > > org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:177) > > at > > > org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:170) > > at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:102) > > at > > > org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:208) > > at > > > org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:195) > > at > > > org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:422) > > at > > > org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) > > at > > > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) > > at > > > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106) > > at > > > org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57) > > at > > > org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:452) > > at > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210) > > at > > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206) > > at > > > org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:340) > > at > > > org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:301) > > at > > > org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:205) > > at > > > org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:162) > > at > > > org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119) > > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > > at > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:724) > > > > > > > > > > besides, we use the singleton pattern to create and get the solr server > > connection, I wonder if this pattern is OK. Once I shutdown the > zookeeper, > > the application can not do the solr query, the error as "no live > > SolrServers available to handle this request". > > > > so I need to restart our connection to reconnect solrcloud. > > > > as we use the singleton pattern, we do not use the method of "shutdown" > to > > release the solrserver connection, will this be a problem? or we need to > > create the connection on demand every time? > > > > > > please help me, look forward to your reply. Thanks a lot! > > > > > > Best Regards! > > Kent Mu > > > > > > -- > Regards, > Shalin Shekhar Mangar. >