Hi Emir, Thanks for the response.
We have specific near realtime search requirements, that is why we are explicitly invoking Solr commits. However we will try to improve on reducing the commits from application. In the meantime, the errors/warnings I mentioned in my previous mail; are they really due to frequent commits..? We are having plenty of requests taking more seconds to deliver from solr irrespective of network overhead, so any thoughts whether commit frequency affects solr latency..? Thanks, Leo Prince On Fri, Oct 13, 2017 at 2:46 PM, Emir Arnautović < emir.arnauto...@sematext.com> wrote: > Hi Leo, > It is considered a bad practice to commit from your application. You > should let Solr handle commits. There is a great article about soft and > hard commits: https://lucidworks.com/2013/08/23/understanding- > transaction-logs-softcommit-and-commit-in-sorlcloud/ < > https://lucidworks.com/2013/08/23/understanding- > transaction-logs-softcommit-and-commit-in-sorlcloud/> > > If you really want to commit from your application, then you should use > commitWithin parameter that would group your commits in a single commit, > and you would be sure that your changes are committed within some time (you > can do that with autocommit as well). > > Opening searcher can be fast but it can also last for a while - it depends > on warming parameters that you set. In any case, I would not recommend that > you focus on making opening last less, but to see what are your NRT > requirements and commit as rare as possible. > > HTH, > Emir > -- > Monitoring - Log Management - Alerting - Anomaly Detection > Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > > > > > On 13 Oct 2017, at 10:30, Leo Prince <leoprince.discussi...@gmail.com> > wrote: > > > > Hi, > > > > I am new to community and thank you for letting me in. > > > > Let me get into my concern real quick. Please find my OS and Solr > versions > > > > Ubuntu 14.04.4 LTS > > solr-spec 4.10.2 > > solr-impl 4.10.2 1634293 - mike - 2014-10-26 05:56:21 > > lucene-spec 4.10.2 > > lucene-impl 4.10.2 1634293 - mike - 2014-10-26 05:51:56 > > > > I am getting the following errors/warnings from Solr > > > > 1, ERROR: org.apache.solr.client.solrj.impl.HttpSolrServer$ > RemoteSolrException: > > Error opening new searcher. exceeded limit of maxWarmingSearchers=2, > > try again later. > > 2, PERFORMANCE WARNING: Overlapping onDeckSearchers=2 > > 3, WARN: DistributedUpdateProcessor error sending update > > org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: > > Error opening new searcher. exceeded limit of maxWarmingSearchers=2, > > try again later. > > > > I have gone through various discussion threads and was able to get a > > minor idea on scenarios where maxWarmingSearchers exceeds. We have > > commit doing from application for individual requests and autocommit > > enabled. > > > > So my concern is, is there any chance of performance issues when > > number of commits are high at a particular point of time. In our > > application, we are approximating like 100-500 commits can happen > > simultaneously from application and autocommit too for those > > individual requests which are not committing individually after the > > write. > > > > Autocommit is configured as follows, > > > > <autoCommit> > > <maxTime>15000</maxTime> > > <openSearcher>false</openSearcher> > > </autoCommit> > > > > > > When worked on the openSearcher parameter, I see it makes solr not to > > open a new search while commit is undergoing. How many seconds or > > milliseconds, solr takes to complete a commit and allow further new > > searches to spawn..? Considering we have a massive numerous individual > > commits and massive doc autocommit, is setting up openSearcher to > > false, a good practice..? What happens to the new requests when a > > commit is going on. Will it be queued for new search to spawn or will > > it be delivered with the old search existed..? > > > > Thanks in advance, > > > > Leo Prince. > >
