Hi Shawn, I again faced the issue and restarting the leader worked for me this time. Please find attached the top command for further insights.
First java process in screenshot is solr. Can it be a possibility that there are some issue with this particular node? Looking forward to hearing from you. Regards Saurabh On Sun, May 12, 2019 at 2:51 AM Shawn Heisey <apa...@elyograg.org> wrote: > On 5/11/2019 12:49 PM, Saurabh Sharma wrote: > > Full collection is present on all 3 nodes.I have checked max docs on > every > > node and they were around 1.5 million on each node with 0.9 Millions > active > > records. > > > > *How much disk space do all the indexes take?* > > -> index size is around 2GB/per node. > > > > *What is Solr's max heap size? Are there any other programs running on * > > *that node other than the one Solr instance? This would include > multiple * > > *Solr instances.* > > ->Maximum heap size for solr is set to 12GB but each node generally take > 3 > > to 5 GB.These solr instances are hosted on servers where we are running > > other services too. > > each machine on which we are running solr is having 64GB of RAM and 24 > core > > cpu.During the peek CPU usage i have seen solr consuming 2000% cpu that > > causes issue. > > A 12GB heap seems excessive for 1.5 million docs taking up 2GB of space, > unless you are running extremely resource intensive queries -- facets or > grouping on high cardinality fields, for instance. > > With other programs on the server, the systems memory may not be fully > available for the operating system to cache the index data. That is the > secret to good Solr performance -- getting relevant parts of the index > into the OS disk cache so that Solr doesn't need to actually read the > data off the disk. > > > *If you can get the screenshot mentioned at the link below, that can * > > *reveal some of the information I have asked for above, but not all of > it.* > > > > * > https://wiki.apache.org/solr/SolrPerformanceProblems#Asking_for_help_on_a_memory.2Fperformance_issue > > < > https://wiki.apache.org/solr/SolrPerformanceProblems#Asking_for_help_on_a_memory.2Fperformance_issue > >* > > -> What specific information is required ? > > The specific information required is the screenshot of a process listing > as described at that page. You'll need to use a file sharing site, as > this mailing list typically eats email attachments. This screenshot > provides a very good overview of the system that we can use to determine > whether we expect good performance. > > > *What query rate is Solr handling?* > > -> 80-100 query per second on each solr node during peek time.It sums up > to > > 250-300 request on 3 replicas of same index. > > That's a very high query rate. It will be even more important for the > system to have the index data in memory. > > Thanks, > Shawn >
