+1 for more investigation
Bill Bell
Sent from mobile
On Feb 22, 2011, at 7:13 AM, Rachita Choudhary <rachita.choudh...@burrp.com>
wrote:
> Hi Solr Users,
>
> We are upgrading from Solr 1.3 to Solr 1.4.1.
> While using Solr 1.3 , we were seeing multiple blocking active threads on
> "org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal() ".
>
> To utilize the benefits of NIO, on upgrading to Solr 1.4.1, we see other
> type of multiple blocking threads on
> "org.apache.solr.request.UnInvertedField.getUnInvertedField() &
>
> SegmentReader$CoreReaders.getTermsReader".
> Due to this, the QTimes shoots up from few hundreds to thousand of
> msec.. even going upto 30-40 secs for a single query.
>
> - The multiple blocking threads show up after few thousands of queries.
> - We do not have faceting and sorting on the same fields.
> - Our facet fields are multivalued text fields, but no large text values are
> present.
> - Index size - around 10 GB
> - We have not specified any method for faceting in our schema.xml.
> - Our field value cache settings are:
> <fieldValueCache
> class="solr.FastLRUCache"
> size="175"
> autowarmCount="0"
> showItems="10"
> />
>
> Can someone please tell us the why we are seeing these blocked threads ?
> Also if they are related to our field value cache , then a cache of size 175
> will be filled up with very few initial queries and right after that we
> should see multiple blocking threads ?
> What difference it will make if we have "facet.method = enum" ?
> Is this all related to fieldValueCache or is there some other configuration
> which we need to set to avoid these blocking threads?
>
> Thanks,
> Rachita
>
> *Cache values example:
> *facetField1_27443 :
> {field=facet1_27443,memSize=4214884,tindexSize=52,time=22,phase1=15,nTerms=4,bigTerms=0,termInstances=6,uses=1}
>
> facetField1_70 :
> {field=facetField1_70,memSize=4223310,tindexSize=308,time=28,phase1=21,nTerms=636,bigTerms=0,termInstances=14404,uses=1}
>
> facetField2 :
> {field=facetField2,memSize=4262644,tindexSize=3156,time=273,phase1=267,nTerms=12188,bigTerms=0,termInstances=1255522,uses=7031}
> *
> Stack trace for
> "org.apache.solr.request.UnInvertedField.getUnInvertedField() -
> BLOCKED"*
>
> at org.apache.solr.request.UnInvertedField.getUnInvertedField
> (UnInvertedField.java:837)
> at org.apache.solr.request.SimpleFacets.getTermCounts (SimpleFacets.java:250)
> at org.apache.solr.request.SimpleFacets.getFacetFieldCounts
> (SimpleFacets.java:283)
> at org.apache.solr.request.SimpleFacets.getFacetCounts (SimpleFacets.java:166)
> at org.apache.solr.handler.component.FacetComponent.process
> (FacetComponent.java:72)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody
> (SearchHandler.java:195)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest
> (RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute (SolrCore.java:1316)
> at org.apache.solr.servlet.SolrDispatchFilter.execute
> (SolrDispatchFilter.java:338)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter
> (SolrDispatchFilter.java:241)
> at com.caucho.server.dispatch.FilterFilterChain.doFilter
> (FilterFilterChain.java:87)
> at com.caucho.server.webapp.WebAppFilterChain.doFilter
> (WebAppFilterChain.java:187)
> at com.caucho.server.dispatch.ServletInvocation.service
> (ServletInvocation.java:266)
> at com.caucho.server.http.HttpRequest.handleRequest (HttpRequest.java:270)
> at com.caucho.server.port.TcpConnection.run (TcpConnection.java:678)
> at com.caucho.util.ThreadPool$Item.runTasks (ThreadPool.java:721)
> at com.caucho.util.ThreadPool$Item.run (ThreadPool.java:643)
> at java.lang.Thread.run (Thread.java:595)
>
>
> *org.apache.lucene.index.SegmentReader$CoreReaders.getTermsReader() -
> BLOCKED*
>
> at org.apache.lucene.index.SegmentReader$CoreReaders.getTermsReader
> (SegmentReader.java:170)
> at org.apache.lucene.index.SegmentTermDocs. (SegmentTermDocs.java:52)
> at org.apache.lucene.index.SegmentReader.termDocs (SegmentReader.java:987)
> at org.apache.lucene.index.IndexReader.termDocs (IndexReader.java:1102)
> at org.apache.lucene.index.SegmentReader.termDocs (SegmentReader.java:981)
> at org.apache.solr.search.SolrIndexReader.termDocs (SolrIndexReader.java:320)
> at org.apache.solr.search.SolrIndexSearcher.getDocSetNC
> (SolrIndexSearcher.java:640)
> at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet
> (SolrIndexSearcher.java:563)
> at org.apache.solr.search.SolrIndexSearcher.numDocs
> (SolrIndexSearcher.java:1422)
> at com.askme.solrenhancements.facet.ExtendedFacet.getCustomFacetCount
> (ExtendedFacet.java:132)
> at com.askme.solrenhancements.facet.ExtendedFacet.getCustomFacetCount
> (ExtendedFacet.java:92)
> at com.askme.solrenhancements.facet.ExtendedFacet.getFacetAdditionalInfo
> (ExtendedFacet.java:69)
> at com.askme.solrenhancements.facet.ExtendedFacet.getFacetInfo
> (ExtendedFacet.java:56)
> at com.askme.solrenhancements.facet.CustomFacetComponent.process
> (CustomFacetComponent.java:43)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody
> (SearchHandler.java:195)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest
> (RequestHandlerBase.java:131)
> at org.apache.solr.core.SolrCore.execute (SolrCore.java:1316)
> at org.apache.solr.servlet.SolrDispatchFilter.execute
> (SolrDispatchFilter.java:338)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter
> (SolrDispatchFilter.java:241)
> at com.caucho.server.dispatch.FilterFilterChain.doFilter
> (FilterFilterChain.java:87)
> at com.caucho.server.webapp.WebAppFilterChain.doFilter
> (WebAppFilterChain.java:187)
> at com.caucho.server.dispatch.ServletInvocation.service
> (ServletInvocation.java:266)
> at com.caucho.server.http.HttpRequest.handleRequest (HttpRequest.java:270)
> at com.caucho.server.port.TcpConnection.run (TcpConnection.java:678)
> at com.caucho.util.ThreadPool$Item.runTasks (ThreadPool.java:721)
> at com.caucho.util.ThreadPool$Item.run (ThreadPool.java:643)
> at java.lang.Thread.run (Thread.java:595)
