Hi Toke, Please see below. We reindexed the solr8 cluster to make sure it was up to date with content.
* What is your Xmx for the Solrs? - solr8 SOLR_JAVA_MEM="-Xms11235m -Xmx11235m" - 70% of OS Memory solr4 export CATALINA_OPTS="-Xms11235m -Xmx11235m" - - 70% of OS Memory (If you use most of the physical memory for Java, there might be too little left for OS-caching) * Do you have a large number of stored or docValued fields? solr8 not sure if this would be considered a large amount or not. Posted schema in dropbox https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 solr4 posted schema in dropbox https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 (Solr 8 returns docValued fields per default while Solr 4 does not) * How large is a response? One simple way to check is curl ' http://solr.obscured.com:8990/solr/content/select?q=*%3A*&wt=json&indent=true ' | wc -c solr8 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 18830 100 18830 0 0 245 0 0:01:16 0:01:16 --:--:-- 5822 18830 solr4 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 13149 0 13149 0 0 807 0 --:--:-- 0:00:16 --:--:-- 3273 13149 (Solr 4 and 8 compress stored content differently and a very large result set requires more CPU power to uncompress in Solr 8 (but less IO)) * Do you have any response related defaults in your solrconfig.xml, such as faceting or grouping? solr8 posted config https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 solr4 posted config https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 (You might be doing heavy aggregation even if you don't explicitly ask for it) *Manzama*a MODERN GOVERNANCE company Russell Bahr Lead Infrastructure Engineer USA & CAN Office: +1 (541) 306 3271 USA & CAN Support: +1 (541) 706 9393 UK Office & Support: +44 (0)203 282 1633 AUS Office & Support: +61 (0) 2 8417 2339 543 NW York Drive, Suite 100, Bend, OR 97703 LinkedIn <http://www.linkedin.com/company/manzama> | Twitter <https://twitter.com/ManzamaInc> | Facebook <http://www.facebook.com/manzamainc> | YouTube <https://www.youtube.com/channel/UCBo3QoqewyNoo7HiT_BFuRw> On Wed, Sep 4, 2019 at 1:43 AM Toke Eskildsen <t...@kb.dk> wrote: > On Tue, 2019-09-03 at 12:35 -0700, Russell Bahr wrote: > > Also, if it helps, the content on each server is between around 6.2Gb > > and 7.8Gb. > > We're still missing something here. The trivial query > > > http://solr.obscured.com:8990/solr/content/select?q=*%3A*&wt=json&indent=true > on such a modest index size & document count should not take seconds to > complete. So we must widen our questioning: > > * What is your Xmx for the Solrs? > (If you use most of the physical memory for Java, there might be too > little left for OS-caching) > > * Do you have a large number of stored or docValued fields? > (Solr 8 returns docValued fields per default while Solr 4 does not) > > * How large is a response? One simple way to check is > curl ' > > http://solr.obscured.com:8990/solr/content/select?q=*%3A*&wt=json&indent=true' > > | wc -c > (Solr 4 and 8 compress stored content differently and a very large > result set requires more CPU power to uncompress in Solr 8 (but less > IO)) > > * Do you have any response related defaults in your solrconfig.xml, > such as faceting or grouping? > (You might be doing heavy aggregation even if you don't explicitly ask > for it) > > > - Toke Eskildsen, Royal Danish Library > > >
