Hello Yonik, I took one node of the 7.2.1 cluster out of the load balancer so it would only receive shard queries, this way i could kind of 'safely' disable our custom components one by one, while keeping functionality in place by letting the other 7.2.1 nodes continue on with the full configuration.
I am now at a point where literally all custom components are deleted or commented out in the config for the node running 7.4. The only custom stuff left is an extension of SearchHandler that only writes numFound to the response headers, and all the token filters in our schema. You were right, it was leaking exactly one SolrIndexSearcher instance on each commit. But, with all our stuff gone, the leak is still there! I triple checked it! Of course, the bastard is locally still not reproducible. So, what is next? I have no clues left. Many, many thanks, Markus -----Original message----- > From:Markus Jelsma <markus.jel...@openindex.io> > Sent: Thursday 28th June 2018 23:52 > To: solr-user@lucene.apache.org > Subject: RE: 7.3 appears to leak > > Hello Yonik, > > If leaking a whole SolrIndexSearcher would cause this problem, then the only > custom component would be our copy/paste-and-enhance version of the elevator > component, is the root of all problems. It is a direct copy of the 7.2 source > where only things like getAnalyzedQuery, the ElevationObj and the loop over > the map entries is changed. > > There are no changes to code related to the searcher. Other component where > we get a RefCount of searcher is used without issues, we always decrement the > reference after using it. But those components are not in use in this > collection. > > The source has changed a lot with 7.4 but we still use the old code. I will > investigate the component thoroughly, even revert to the old 7.2 vanilla > component for a brief period in production for one machine. It may not be a > problem if i don't let our load balancer access it directly, so it only > serves shard queries. > > I will get back to this topic tomorrow! > > Many thanks, > Markus > > > > -----Original message----- > > From:Yonik Seeley <ysee...@gmail.com> > > Sent: Thursday 28th June 2018 23:30 > > To: solr-user@lucene.apache.org > > Subject: Re: 7.3 appears to leak > > > > > * SortedIntDocSet instances ánd ConcurrentLRUCache$CacheEntry instances > > > are both leaked on commit; > > > > If these are actually filterCache entries being leaked, it stands to > > reason that a whole searcher is being leaked somewhere. > > > > -Yonik > > >
