Can you share the cache stats from the admin panel? Also how much load are you talking about here? (Queries/second) How many documents do you have? Are you fetching any large stored fields?
On Thu, 3 Mar 2016, 12:31 Maulin Rathod, <[email protected]> wrote: > Adding extra information. > > Our index size is around 120 GB (2 shard + 2 replica). > We have 400 GB RAM on our windows server. Solr is assigned 50 GB RAM. So > there is huge amount of free RAM (>300 GB) is available for OS. > > We have very simple query which returns only 5 solr documents. Under load > condition it takes 100 ms to 2000 ms. > > > -----Original Message----- > From: Maulin Rathod > Sent: 03 March 2016 12:24 > To: [email protected] > Subject: RE: Solr Configuration (Caching & RAM) for performance Tuning > > we do soft commit when we insert/update document. > > //Insert Document > > UpdateResponse resp = cloudServer.add(doc, 1000); if (resp.getStatus() == > 0) { > success = true; > } > > //Update Document > > UpdateRequest req = new UpdateRequest(); req.setCommitWithin(1000); > req.add(docs); UpdateResponse resp = req.process(cloudServer); if > (resp.getStatus() == 0) { > success = true; > } > > Here is commit settings in solrconfig.xml. > > <autoCommit> > <maxTime>600000</maxTime> > <maxDocs>20000</maxDocs> > <openSearcher>false</openSearcher> > </autoCommit> > > <autoSoftCommit> > <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime> > </autoSoftCommit> > > > > > -----Original Message----- > From: Binoy Dalal [mailto:[email protected]] > Sent: 03 March 2016 11:57 > To: [email protected] > Subject: Re: Solr Configuration (Caching & RAM) for performance Tuning > > 1) Experiment with the autowarming settings in solrconfig.xml. Since in > your case, you're indexing so frequently consider setting the count to a > low number, so that not a lot of time is spent warming the caches. > Alternatively if you're not very big on initial query response times being > small, you could turn off auto warming all together. > Also how are your commit settings configured? > Do you do a hard commit every 10 seconds or do you have soft committing > enabled? > > 2) 50Gb memory is way to high to assign to just solr and it is unnecessary. > Solr loads your index into the OS cache. The index is not held in the JVM > heap. > So it is essential that your OS have enough free memory available to load > the entire index. > Since you're only seeing about a 2gb use of your JVM memory, set your heap > size to something around 4gbs. > > Also, how big is your index? > > On Thu, 3 Mar 2016, 11:39 Maulin Rathod, <[email protected]> wrote: > > > Hi, > > > > We are using Solr 5.2 (on windows 2012 server/jdk 1.8) for document > > content indexing/querying. We found that querying slows down > > intermittently under load condition. > > > > In our analysis we found two issues. > > > > 1) Solr is not effectively using caching. > > > > Whenever new document indexed, it opens new searcher and hence cache > > will become invalid (as cache was associated with old Index Searcher). > > In our scenario, new documents are indexed very frequently (at least > > 10 document are indexed per minute). So effectively cache will not be > > useful as it will open new searcher frequently to make new documents > available for searching. > > How can improve caching usage? > > > > > > 2) RAM is not utilized > > > > We observed that Solr is using only 1-2 GB of heap even though we have > > assign 50 GB. Seems like it is not loading index into RAM which leads > > to high IO. Is it possible to configure Solr to fully load indexes in > memory? > > Don't find any documentation about this. How can we increase RAM usage > > to improve Solr performance? > > > > > > Regards, > > > > Maulin > > > > [CC Award Winners 2015] > > > > -- > Regards, > Binoy Dalal > -- Regards, Binoy Dalal
