Hi Toke and Shawn, Any thoughts on what I sent? Thanks in advance, Russ *Manzama*a MODERN GOVERNANCE company
Russell Bahr Lead Infrastructure Engineer USA & CAN Office: +1 (541) 306 3271 USA & CAN Support: +1 (541) 706 9393 UK Office & Support: +44 (0)203 282 1633 AUS Office & Support: +61 (0) 2 8417 2339 543 NW York Drive, Suite 100, Bend, OR 97703 LinkedIn <http://www.linkedin.com/company/manzama> | Twitter <https://twitter.com/ManzamaInc> | Facebook <http://www.facebook.com/manzamainc> | YouTube <https://www.youtube.com/channel/UCBo3QoqewyNoo7HiT_BFuRw> On Wed, Sep 4, 2019 at 2:55 PM Russell Bahr <r...@manzama.com> wrote: > Hi Toke, > Please see below. We reindexed the solr8 cluster to make sure it was up > to date with content. > > * What is your Xmx for the Solrs? - > solr8 > > SOLR_JAVA_MEM="-Xms11235m -Xmx11235m" - 70% of OS Memory > > > solr4 > > export CATALINA_OPTS="-Xms11235m -Xmx11235m" - - 70% of OS Memory > > > (If you use most of the physical memory for Java, there might be too > little left for OS-caching) > > * Do you have a large number of stored or docValued fields? > solr8 > not sure if this would be considered a large amount or not. Posted schema > in dropbox > https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 > solr4 > posted schema in dropbox > https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 > > (Solr 8 returns docValued fields per default while Solr 4 does not) > > * How large is a response? One simple way to check is > curl ' > > http://solr.obscured.com:8990/solr/content/select?q=*%3A*&wt=json&indent=true > ' > | wc -c > > solr8 > > % Total % Received % Xferd Average Speed Time Time Time > Current > > Dload Upload Total Spent Left > Speed > > 100 18830 100 18830 0 0 245 0 0:01:16 0:01:16 --:--:-- > 5822 > > 18830 > > solr4 > > % Total % Received % Xferd Average Speed Time Time Time > Current > > Dload Upload Total Spent Left > Speed > > 100 13149 0 13149 0 0 807 0 --:--:-- 0:00:16 --:--:-- > 3273 > > 13149 > > > > (Solr 4 and 8 compress stored content differently and a very large > result set requires more CPU power to uncompress in Solr 8 (but less > IO)) > > * Do you have any response related defaults in your solrconfig.xml, > such as faceting or grouping? > > solr8 > posted config > https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 > > solr4 > posted config > https://www.dropbox.com/sh/hslknixd3azj7mi/AABnCXex_HInCvRz3kuKLwNna?dl=0 > > (You might be doing heavy aggregation even if you don't explicitly ask > for it) > > > *Manzama*a MODERN GOVERNANCE company > > Russell Bahr > Lead Infrastructure Engineer > > USA & CAN Office: +1 (541) 306 3271 > USA & CAN Support: +1 (541) 706 9393 > UK Office & Support: +44 (0)203 282 1633 > AUS Office & Support: +61 (0) 2 8417 2339 > > 543 NW York Drive, Suite 100, Bend, OR 97703 > > LinkedIn <http://www.linkedin.com/company/manzama> | Twitter > <https://twitter.com/ManzamaInc> | Facebook > <http://www.facebook.com/manzamainc> | YouTube > <https://www.youtube.com/channel/UCBo3QoqewyNoo7HiT_BFuRw> > > > On Wed, Sep 4, 2019 at 1:43 AM Toke Eskildsen <t...@kb.dk> wrote: > >> On Tue, 2019-09-03 at 12:35 -0700, Russell Bahr wrote: >> > Also, if it helps, the content on each server is between around 6.2Gb >> > and 7.8Gb. >> >> We're still missing something here. The trivial query >> >> >> http://solr.obscured.com:8990/solr/content/select?q=*%3A*&wt=json&indent=true >> on such a modest index size & document count should not take seconds to >> complete. So we must widen our questioning: >> >> * What is your Xmx for the Solrs? >> (If you use most of the physical memory for Java, there might be too >> little left for OS-caching) >> >> * Do you have a large number of stored or docValued fields? >> (Solr 8 returns docValued fields per default while Solr 4 does not) >> >> * How large is a response? One simple way to check is >> curl ' >> >> http://solr.obscured.com:8990/solr/content/select?q=*%3A*&wt=json&indent=true' >> >> | wc -c >> (Solr 4 and 8 compress stored content differently and a very large >> result set requires more CPU power to uncompress in Solr 8 (but less >> IO)) >> >> * Do you have any response related defaults in your solrconfig.xml, >> such as faceting or grouping? >> (You might be doing heavy aggregation even if you don't explicitly ask >> for it) >> >> >> - Toke Eskildsen, Royal Danish Library >> >> >>
