Committing after every doc is an anti-pattern. All the in-memory structures
are being thrown away after each update/insert.
Why do you think you need to do this? The usual pattern is to just let your
autocommit parameters in Solr config.XML do this for you.
Ditto with specifying commitWithin on each add unless you have a pretty
unusual installation.
I'd recommend setting your commit intervals to be as long as you can stand
and not do anything from the client, including specifying commitWithin.
Best
Erick
On Mar 3, 2016 8:47 PM, "Binoy Dalal" <binoydala...@gmail.com> wrote:
> Can you share the cache stats from the admin panel?
> Also how much load are you talking about here? (Queries/second)
> How many documents do you have?
> Are you fetching any large stored fields?
>
> On Thu, 3 Mar 2016, 12:31 Maulin Rathod, <mrat...@asite.com> wrote:
>
> > Adding extra information.
> >
> > Our index size is around 120 GB (2 shard + 2 replica).
> > We have 400 GB RAM on our windows server. Solr is assigned 50 GB RAM.
> So
> > there is huge amount of free RAM (>300 GB) is available for OS.
> >
> > We have very simple query which returns only 5 solr documents. Under load
> > condition it takes 100 ms to 2000 ms.
> >
> >
> > -----Original Message-----
> > From: Maulin Rathod
> > Sent: 03 March 2016 12:24
> > To: solr-user@lucene.apache.org
> > Subject: RE: Solr Configuration (Caching & RAM) for performance Tuning
> >
> > we do soft commit when we insert/update document.
> >
> > //Insert Document
> >
> > UpdateResponse resp = cloudServer.add(doc, 1000); if (resp.getStatus() ==
> > 0) {
> > success = true;
> > }
> >
> > //Update Document
> >
> > UpdateRequest req = new UpdateRequest(); req.setCommitWithin(1000);
> > req.add(docs); UpdateResponse resp = req.process(cloudServer); if
> > (resp.getStatus() == 0) {
> > success = true;
> > }
> >
> > Here is commit settings in solrconfig.xml.
> >
> > <autoCommit>
> > <maxTime>600000</maxTime>
> > <maxDocs>20000</maxDocs>
> > <openSearcher>false</openSearcher>
> > </autoCommit>
> >
> > <autoSoftCommit>
> > <maxTime>${solr.autoSoftCommit.maxTime:-1}</maxTime>
> > </autoSoftCommit>
> >
> >
> >
> >
> > -----Original Message-----
> > From: Binoy Dalal [mailto:binoydala...@gmail.com]
> > Sent: 03 March 2016 11:57
> > To: solr-user@lucene.apache.org
> > Subject: Re: Solr Configuration (Caching & RAM) for performance Tuning
> >
> > 1) Experiment with the autowarming settings in solrconfig.xml. Since in
> > your case, you're indexing so frequently consider setting the count to a
> > low number, so that not a lot of time is spent warming the caches.
> > Alternatively if you're not very big on initial query response times
> being
> > small, you could turn off auto warming all together.
> > Also how are your commit settings configured?
> > Do you do a hard commit every 10 seconds or do you have soft committing
> > enabled?
> >
> > 2) 50Gb memory is way to high to assign to just solr and it is
> unnecessary.
> > Solr loads your index into the OS cache. The index is not held in the JVM
> > heap.
> > So it is essential that your OS have enough free memory available to load
> > the entire index.
> > Since you're only seeing about a 2gb use of your JVM memory, set your
> heap
> > size to something around 4gbs.
> >
> > Also, how big is your index?
> >
> > On Thu, 3 Mar 2016, 11:39 Maulin Rathod, <mrat...@asite.com> wrote:
> >
> > > Hi,
> > >
> > > We are using Solr 5.2 (on windows 2012 server/jdk 1.8) for document
> > > content indexing/querying. We found that querying slows down
> > > intermittently under load condition.
> > >
> > > In our analysis we found two issues.
> > >
> > > 1) Solr is not effectively using caching.
> > >
> > > Whenever new document indexed, it opens new searcher and hence cache
> > > will become invalid (as cache was associated with old Index Searcher).
> > > In our scenario, new documents are indexed very frequently (at least
> > > 10 document are indexed per minute). So effectively cache will not be
> > > useful as it will open new searcher frequently to make new documents
> > available for searching.
> > > How can improve caching usage?
> > >
> > >
> > > 2) RAM is not utilized
> > >
> > > We observed that Solr is using only 1-2 GB of heap even though we have
> > > assign 50 GB. Seems like it is not loading index into RAM which leads
> > > to high IO. Is it possible to configure Solr to fully load indexes in
> > memory?
> > > Don't find any documentation about this. How can we increase RAM usage
> > > to improve Solr performance?
> > >
> > >
> > > Regards,
> > >
> > > Maulin
> > >
> > > [CC Award Winners 2015]
> > >
> > > --
> > Regards,
> > Binoy Dalal
> >
> --
> Regards,
> Binoy Dalal
>
