Well done Mikhail, curious to see the performance! Apart the disk usage ( of course building docValues will cost more space), taking in consideration the Field cardinality, in the past when the field cardinality was low ( few unique values in the field), the enum approach was suggested ( so DocValues were non necessary strictly talking about faceting).
After this improvement, in your opinion, what is the current status ? after the 5.4 release, will enum faceting approach make sense anymore ? Has anyone benchmarked this ? Cheers 2015-09-23 9:02 GMT+01:00 Mikhail Khludnev <mkhlud...@griddynamics.com>: > Uwe, > > I'm sorry for confusion https://issues.apache.org/jira/browse/SOLR-7730 > goes in 5.4 only. Hence, to get fast DV facets you need to apply patch > (it's pretty small). > Accelerating non-DV facets is not so clear so far. Please show profiler > snapshot for non-DV facets if you wish to go this way. > > > On Tue, Sep 22, 2015 at 5:36 PM, Uwe Reh <r...@hebis.uni-frankfurt.de> > wrote: > > > The exact version as shown by the UI is: > > - solr-impl 5.3.0 1696229 - noble - 2015-08-17 17:10:43 > > - lucene-impl 5.3.0 1696229 - noble - 2015-08-17 16:59:03 > > > > Unfortunately my skills in debugging are limited. So I'm not sure about a > > 'deeper caller stack'. > > Did you mean the attached snapshot from VirtualVM, a stack trace like > > below or something else? Please give me a hint. > > > > uwe > > > > "qtp1734853116-68" #68 prio=5 os_prio=64 tid=0x00000000117fd800 nid=0x77 > >> runnable [0xfffffd7f991fc000] > >> java.lang.Thread.State: RUNNABLE > >> at java.util.HashMap.resize(HashMap.java:734) > >> at java.util.HashMap.putVal(HashMap.java:662) > >> at java.util.HashMap.put(HashMap.java:611) > >> at > >> > org.apache.lucene.index.FieldInfos$Builder.addOrUpdateInternal(FieldInfos.java:344) > >> at > >> org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:366) > >> at > >> org.apache.lucene.index.FieldInfos$Builder.add(FieldInfos.java:304) > >> at > >> > org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:245) > >> at > >> > org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:237) > >> at > >> > org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedSetDocValues(SlowCompositeReaderWrapper.java:174) > >> at > >> > org.apache.solr.request.DocValuesFacets.getCounts(DocValuesFacets.java:72) > >> at > >> > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:492) > >> at > >> > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:385) > >> at > >> org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:628) > >> at > >> org.apache.solr.request.SimpleFacets$3.call(SimpleFacets.java:619) > >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) > >> at > >> org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:573) > >> at > >> > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:644) > >> at > >> > org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:294) > >> at > >> > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:256) > >> at > >> > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:285) > >> at > >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) > >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068) > >> at > >> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669) > >> at > >> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462) > >> at > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210) > >> at > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179) > >> at > >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) > >> at > >> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) > >> at > >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > >> at > >> > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) > >> at > >> > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) > >> at > >> > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) > >> at > >> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) > >> at > >> > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) > >> at > >> > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) > >> at > >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > >> at > >> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) > >> at > >> > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) > >> at > >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > >> at org.eclipse.jetty.server.Server.handle(Server.java:499) > >> at > >> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) > >> at > >> > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) > >> at org.eclipse.jetty.io > >> .AbstractConnection$2.run(AbstractConnection.java:540) > >> at > >> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > >> at > >> > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) > >> at java.lang.Thread.run(Thread.java:745) > >> > > > > > > > > Am 22.09.2015 um 12:56 schrieb Mikhail Khludnev: > > > >> It's quite strange > >> https://issues.apache.org/jira/browse/SOLR-7730 significantly optimized > >> DV > >> facets at 5.3.0 exactly by avoiding FileInfo merge. > >> Would you mind to provide deeper caller stack for > >> org.apache.lucene.index.FileInfos.MultibleFields.getMergedFieldInfos()? > >> Or a time spend in SlowCompositeReaderWrapper, DocValuesFacets, > >> MultiDocValues and their hot methods. > >> Which version you exactly on? and how do you know that? > >> Thanks > >> > >> > > > > > -- > Sincerely yours > Mikhail Khludnev > Principal Engineer, > Grid Dynamics > > <http://www.griddynamics.com> > <mkhlud...@griddynamics.com> > -- -------------------------- Benedetti Alessandro Visiting card - http://about.me/alessandro_benedetti Blog - http://alexbenedetti.blogspot.co.uk "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
