Hi Toke, Thanks for the post. Good that things are moving forward! It took a while!
Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 5 Feb 2020, at 15:23, Toke Eskildsen <t...@kb.dk> wrote: > > On Wed, 2020-02-05 at 13:00 +0100, Emir Arnautović wrote: >> I was thinking in that direction. Do you know where it is in the >> codebase or which structure is used - I am guessing some array of >> objects? > > Yeah. More precisely a priority queue of Objects, initialized with > sentinel Objects. rows=1000000 is bad both from a memory allocation POW > and because the heap-structure of the priority queue implementation has > extremely bad memory locality when it is being updated. > > I performed some measurements and did some experiments a few years ago: > https://sbdevel.wordpress.com/2015/10/05/speeding-up-core-search/ > and there is https://issues.apache.org/jira/browse/LUCENE-8875 which > takes care of the Sentinel thing in solr 8.2. > > - Toke Eskildsen, Royal Danish Library > >
