So, I tried few things and it seems like there are more page faults after the solr6 upgrade. Even when there is no update or query activity (Except the periodic commit), the pagefaults are little higher than they used to be .
Any suggestions in this area? Thanks Nawab On Tue, Aug 15, 2017 at 4:09 PM, Nawab Zada Asad Iqbal <khi...@gmail.com> wrote: > Hi Rick > > My software is not very sophisticated. I have picked some queries from > production logs, which I am replaying against this solr installation. It is > not a SolrCloud but i specify "shards=" in the query to gather results > from all shards. > > I found some values to tweak e.g. > <int name="maxConnectionsPerHost">1500</int> > <int name="maxConnections">150000</int> > > After doing this, the "Unable to write response, client closed connection > or we are shutting down" error is mostly gone. > > However, the query perf is bad. The server is not using all the assigned > memory. However CPU usage is reaching 80%. > The query response time is 50+ times worse (e.g., 1400 msec vs 20 msec for > 75th percentile) . > What can I do to use more memory and hopefully alleviate some of this bad > performance? > > My cache settings are identical to older setup. > > > Thanks > Nawab > > > > > > On Mon, Aug 14, 2017 at 9:01 AM, Rick Leir <rl...@leirtech.com> wrote: > >> Nawab >> What test software do you use? What else is happening when the exception >> occurs? >> Cheers -- Rick >> >> On August 12, 2017 1:48:19 PM EDT, Nawab Zada Asad Iqbal < >> khi...@gmail.com> wrote: >> >Hi, >> > >> >I am executing a query performance test against my solr 6.6 setup and I >> >noticed following exception every now and then. What do I need to do? >> > >> >Aug 11, 2017 08:40:07 AM INFO (qtp761960786-250) [ x:filesearch] >> >o.a.s.s.HttpSolrCall Unable to write response, client closed connection >> >or >> >we are shutting down >> >org.eclipse.jetty.io.EofException >> >at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199) >> > at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420) >> > at org.eclipse.jetty.io.WriteFlusher.completeWrite( >> >WriteFlusher.java:375) >> > at org.eclipse.jetty.io.SelectChannelEndPoint$3.run( >> >SelectChannelEndPoint.java:107) >> > at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected( >> >SelectChannelEndPoint.java:193) >> > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer. >> >processSelected(ManagedSelector.java:283) >> > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce( >> >ManagedSelector.java:181) >> > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume. >> >executeProduceConsume(ExecuteProduceConsume.java:249) >> > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume. >> >produceConsume(ExecuteProduceConsume.java:148) >> > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run( >> >ExecuteProduceConsume.java:136) >> > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( >> >QueuedThreadPool.java:671) >> > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run( >> >QueuedThreadPool.java:589) >> > at java.lang.Thread.run(Thread.java:748) >> >Caused by: java.io.IOException: Broken pipe >> > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) >> > at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) >> > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) >> > at sun.nio.ch.IOUtil.write(IOUtil.java:51) >> > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) >> >at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177) >> > >> > >> > >> >Apart from that, I also noticed that the query response time is longer >> >than >> >I expected, while the memory utilization stays <= 35%. I thought that >> >somewhere I have set maxThreads (Jetty) to a very low number, however I >> >am >> >falling back on default which is 10000 (so that shouldn't be a >> >problem). >> > >> > >> >Thanks >> >Nawab >> >> -- >> Sorry for being brief. Alternate email is rickleir at yahoo dot com > > >
