> How many zookeepers do you have? How many collections? What is there size?
> How much CPU / memory do you give per container? How much heap in comparison
> to total memory of the container ?
3 Zookeepers.
733 containers/nodes
735 total cores. Each core ranges from ~4-10GB of index. (Autoscaling splits at
12GB)
10 collections, ranging from 147 shards at most, to 3 at least. Replication
factor of 2 other than .system which has 3 replicas.
Each container has a min/max heap of 750MB other than the overseer containers
which have a min/max of 3GB.
Containers aren't hard limited by K8S on memory or CPU but the machines the
containers are on have 4 cores and ~13GB of ram.
Now that I look at the CPU usage on a per container basis, it looks like it is
maxing out all four cores on the VM that is hosting the overseer container.
Barely using the heap (300MB).
I suppose that means that if we put the overseers on machines with more cores,
it might be able to get things done a bit faster. Though that still seems like
a limited solution as we are going to grow this cluster at least double in size
if not larger.
We are using the solr:7.7.2 container.
Java Options on the home page are below:
-DHELM_CHART=overseer
-DSTOP.KEY=solrrocks
-DSTOP.PORT=7983
-Dhost=overseer-solr-0.solr.DOMAIN
-Djetty.home=/opt/solr/server
-Djetty.port=8983
-Dsolr.data.home=
-Dsolr.default.confdir=/opt/solr/server/solr/configsets/_default/conf
-Dsolr.install.dir=/opt/solr
-Dsolr.jetty.https.port=8983
-Dsolr.log.dir=/opt/solr/server/logs
-Dsolr.log.level=INFO
-Dsolr.solr.home=/opt/solr/server/home
-Duser.timezone=UTC
-DzkClientTimeout=60000
-DzkHost=zookeeper-1.DOMAIN:2181,zookeeper-2.DOMAIN:2181,zookeeper-3.DOMAIN:2181/ZNODE
-XX:+CMSParallelRemarkEnabled
-XX:+CMSScavengeBeforeRemark
-XX:+ParallelRefProcEnabled
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+UseConcMarkSweepGC
-XX:-OmitStackTraceInFastThrow
-XX:CMSInitiatingOccupancyFraction=50
-XX:CMSMaxAbortablePrecleanTime=6000
-XX:ConcGCThreads=4
-XX:MaxTenuringThreshold=8
-XX:NewRatio=3
-XX:ParallelGCThreads=4
-XX:PretenureSizeThreshold=64m
-XX:SurvivorRatio=4
-XX:TargetSurvivorRatio=90
-Xlog:gc*:file=/opt/solr/server/logs/solr_gc.log:time,uptime:filecount=9,filesize=20M
-Xmx3g
-Xmx3g
-Xss256k
evolve24 Confidential & Proprietary Statement: This email and any attachments
are confidential and may contain information that is privileged, confidential
or exempt from disclosure under applicable law. It is intended for the use of
the recipients. If you are not the intended recipient, or believe that you have
received this communication in error, please do not read, print, copy,
retransmit, disseminate, or otherwise use the information. Please delete this
email and attachments, without reading, printing, copying, forwarding or saving
them, and notify the Sender immediately by reply email. No confidentiality or
privilege is waived or lost by any transmission in error.
