Hi Shwan, I am providing the data asked .In case any thing else is required please let me know.
*If you get the maxDoc number from every core (index) in that Solr * *instance, and add those numbers up, you'll get a total document count * *for the whole node. What are those numbers?* -This solr server is running single Core.This cloud instance is having 3 servers on different machines with maxshardperNode set to 1 and having a replication factor of 3. Full collection is present on all 3 nodes.I have checked max docs on every node and they were around 1.5 million on each node with 0.9 Millions active records. *How much disk space do all the indexes take?* -> index size is around 2GB/per node. *What is Solr's max heap size? Are there any other programs running on * *that node other than the one Solr instance? This would include multiple * *Solr instances.* ->Maximum heap size for solr is set to 12GB but each node generally take 3 to 5 GB.These solr instances are hosted on servers where we are running other services too. each machine on which we are running solr is having 64GB of RAM and 24 core cpu.During the peek CPU usage i have seen solr consuming 2000% cpu that causes issue. *If you can get the screenshot mentioned at the link below, that can * *reveal some of the information I have asked for above, but not all of it.* *https://wiki.apache.org/solr/SolrPerformanceProblems#Asking_for_help_on_a_memory.2Fperformance_issue <https://wiki.apache.org/solr/SolrPerformanceProblems#Asking_for_help_on_a_memory.2Fperformance_issue>* -> What specific information is required ? *What query rate is Solr handling?* -> 80-100 query per second on each solr node during peek time.It sums up to 250-300 request on 3 replicas of same index. *Do you have multiple Solr servers for this use, or just one?* -> We do use single solr cluster with three nodes in it. I generally face issue once in every few days and one thing that is common on all such happenings is high traffic .I am suspecting that due to dynamic nature of index.during high traffic , cache miss iincreases and solr start doing run time comuptations resulting in high CPU usage.I am not being able to find any other explanation as of now. Thanks Saurabh Sharma On Sat, May 11, 2019 at 8:58 PM Shawn Heisey <apa...@elyograg.org> wrote: > On 5/11/2019 8:05 AM, Saurabh Sharma wrote: > > I have been observing a very unique pattern in our solr resource usage. > > I am running a cluster with 3 nodes and RAM on each node is 12GB. > > We are doing hard commits every 1 minute and soft commits every 15 > seconds. > > Under normal circumstances solr response time is ~15 ms and CPU usage of > > around 200% (~2 cores) . But we have few instances where CPU load > suddenly > > increase to around 2000%(~ 20 cores) and response time increase to 1000ms > > .At the time this situation happen , almost every query start taking time > > and CPU usage keeps on increasing for around 30 mins and even restart > don't > > help. RAM usage remains constant for this duration.Do solr start behaving > > like this under high traffic situations ? what can be the possible > reasons > > of very high CPU usage in real time solr. > > If you get the maxDoc number from every core (index) in that Solr > instance, and add those numbers up, you'll get a total document count > for the whole node. What are those numbers? > > How much disk space do all the indexes take? > > What is Solr's max heap size? Are there any other programs running on > that node other than the one Solr instance? This would include multiple > Solr instances. > > If you can get the screenshot mentioned at the link below, that can > reveal some of the information I have asked for above, but not all of it. > > > https://wiki.apache.org/solr/SolrPerformanceProblems#Asking_for_help_on_a_memory.2Fperformance_issue > > What query rate is Solr handling? > > Do you have multiple Solr servers for this use, or just one? > > Thanks, > Shawn >
