I really appreciate if somebody can help me to solve this problem. Regards.
On Tue, Dec 8, 2015 at 9:22 PM, Ali Nazemian <alinazem...@gmail.com> wrote: > I did that already. The situation was worse. The autocommit part makes > solr unavailable. > On Dec 8, 2015 7:13 PM, "Emir Arnautovic" <emir.arnauto...@sematext.com> > wrote: > >> Hi Ali, >> Can you try without explicit commits and see if threads will still be >> blocked. >> >> Thanks, >> Emir >> >> On 08.12.2015 16:19, Ali Nazemian wrote: >> >>> The indexing load is as follows: >>> - Around 1000 documents every 5 mins. >>> - The indexing speed is slow because of the complicated analyzer which is >>> applied to each document. It takes around 60 seconds to index 1000 >>> documents with applying this analyzer (It is really slow. However, based >>> on >>> the analyzing part I think it would be acceptable). >>> - The concurrentsolrclient is used in all the indexing/updating cases. >>> >>> Regards. >>> >>> On Tue, Dec 8, 2015 at 6:36 PM, Ali Nazemian <alinazem...@gmail.com> >>> wrote: >>> >>> Dear Emir, >>>> Hi, >>>> There are some cases that I have soft commit in my application. However, >>>> the bulk update part has only hard commit for a bulk of 2500 documents. >>>> Here are some information about the whole indexing/updating scenarios: >>>> - Indexing part uses soft commit. >>>> - In a single update cases soft commit is used. >>>> - For bulk update batch hard commit is used (on 2500 documents) >>>> - Auto hard commit :120 sec >>>> - Auto soft commit: disable >>>> >>>> Best regards. >>>> >>>> >>>> On Tue, Dec 8, 2015 at 12:35 PM, Emir Arnautovic < >>>> emir.arnauto...@sematext.com> wrote: >>>> >>>> Hi Ali, >>>>> This thread is blocked because cannot obtain update lock - in this >>>>> particular case when doing soft commit. I am guessing that there >>>>> others are >>>>> blocked for the same reason. Can you tell us bit more about your setup >>>>> and >>>>> indexing load and procedure? Do you do explicit commits? >>>>> >>>>> Regards, >>>>> Emir >>>>> >>>>> -- >>>>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >>>>> Solr & Elasticsearch Support * http://sematext.com/ >>>>> >>>>> >>>>> >>>>> On 08.12.2015 08:16, Ali Nazemian wrote: >>>>> >>>>> Hi, >>>>>> There is a while since I have had problem with Solr 5.2.1 and I could >>>>>> not >>>>>> fix it yet. The only think that is clear to me is when I send bulk >>>>>> update >>>>>> to Solr the commit thread will be blocked! Here is the thread dump >>>>>> output: >>>>>> >>>>>> "qtp595445781-8207" prio=10 tid=0x00007f0bf68f5800 nid=0x5785 waiting >>>>>> for >>>>>> monitor entry [0x00007f081cf04000] >>>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:608) >>>>>> - waiting to lock <0x000000067ba2e660> (a java.lang.Object) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1635) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1612) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:270) >>>>>> at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) >>>>>> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064) >>>>>> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654) >>>>>> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) >>>>>> at >>>>>> >>>>>> >>>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) >>>>>> at >>>>>> >>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >>>>>> at org.eclipse.jetty.server.Server.handle(Server.java:497) >>>>>> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) >>>>>> at >>>>>> org.eclipse.jetty.io >>>>>> .AbstractConnection$2.run(AbstractConnection.java:540) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) >>>>>> at >>>>>> >>>>>> >>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) >>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>> >>>>>> Locked ownable synchronizers: >>>>>> - None >>>>>> >>>>>> FYI there are lots of blocked thread in thread dump report and Solr >>>>>> becomes >>>>>> really slow in this case. The temporary solution would be restarting >>>>>> Solr. >>>>>> But, I am really sick of restarting! I really appreciate if somebody >>>>>> can >>>>>> help me to solve this problem? >>>>>> >>>>>> Best regards. >>>>>> >>>>>> >>>>>> >>>> -- >>>> A.Nazemian >>>> >>>> >>> >>> >> -- >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >> Solr & Elasticsearch Support * http://sematext.com/ >> >> -- A.Nazemian
