I agree with the suggestions so far. The cache auto-warming doesn't seem the problem as the index is not massive and the auto-warm is for only 10 docs. Are you using any warming query for the new searcher ?
Are you using soft or hard commit ? This can make the difference ( soft are much cheaper, not free but cheaper) . You said : " Actually earlier it was taking less but suddenly it has increased " What happened ? Anyway, there are a lot of questions to answer before we can help you... Cheers On Fri, Aug 12, 2016 at 4:58 AM, Esther-Melaine Quansah < esther.quan...@lucidworks.com> wrote: > Midas, > > I’d like further clarification as well. Are you sending commits along with > each document that you’re POSTing to Solr? If so, you’re essentially either > opening a new searcher or flushing to disk with each POST which could > explain latency between each request. > > Thanks, > > Esther > > On Aug 11, 2016, at 12:19 PM, Erick Erickson <erickerick...@gmail.com> > wrote: > > > > bq: we post json documents through the curl it takes the time (same > time i > > would like to say that we are not hard committing ). that curl takes time > > i.e. 1.3 sec. > > > > OK, I'm really confused. _what_ is taking 1.3 seconds? When you said > > commit, I was thinking of Solr's commit operation, which is totally > distinct > > from just adding a doc to the index. But I read the above statement > > as you're saying it takes 1.3 seconds just to send a doc to Solr. > > > > Let's see the exact curl command you're using please? > > > > Best, > > Erick > > > > > > On Thu, Aug 11, 2016 at 5:32 AM, Emir Arnautovic > > <emir.arnauto...@sematext.com> wrote: > >> Hi Midas, > >> > >> 1. How many indexing threads? > >> 2. Do you batch documents and what is your batch size? > >> 3. How frequently do you commit? > >> > >> I would recommend: > >> 1. Move commits to Solr (set auto soft commit to max allowed time) > >> 2. Use batches (bulks) > >> 3. tune bulk size and number of threads to achieve max performance. > >> > >> Thanks, > >> Emir > >> > >> > >> > >> On 11.08.2016 08:21, Midas A wrote: > >>> > >>> Emir, > >>> > >>> other queries: > >>> > >>> a) Solr cloud : NO > >>> b) <filterCache class="solr.FastLRUCache" > >>> size="5000" initialSize="5000" autowarmCount="10"/> > >>> c) <queryResultCache class="solr.LRUCache" > >>> size="1000" initialSize="1000" autowarmCount="10"/> > >>> d) <documentCache class="solr.LRUCache" > >>> size="1000" initialSize="1000" autowarmCount="10"/> > >>> e) we are using multi threaded system. > >>> > >>> On Thu, Aug 11, 2016 at 11:48 AM, Midas A <test.mi...@gmail.com> > wrote: > >>> > >>>> Emir, > >>>> > >>>> we post json documents through the curl it takes the time (same time i > >>>> would like to say that we are not hard committing ). that curl takes > time > >>>> i.e. 1.3 sec. > >>>> > >>>> On Wed, Aug 10, 2016 at 2:29 PM, Emir Arnautovic < > >>>> emir.arnauto...@sematext.com> wrote: > >>>> > >>>>> Hi Midas, > >>>>> > >>>>> According to your autocommit configuration and your worry about > commit > >>>>> time I assume that you are doing explicit commits from client code > and > >>>>> that > >>>>> 1.3s is client observed commit time. If that is the case, than it > might > >>>>> be > >>>>> opening searcher that is taking time. > >>>>> > >>>>> How do you index data - single threaded or multithreaded? How > frequently > >>>>> do you commit from client? Can you let Solr do soft commits instead > of > >>>>> explicitly committing? Do you have warmup queries? Is this SolrCloud? > >>>>> What > >>>>> is number of servers (what spec), shards, docs? > >>>>> > >>>>> In any case monitoring can give you more info about server/Solr > behavior > >>>>> and help you diagnose issues more easily/precisely. One such > monitoring > >>>>> tool is our SPM <http://sematext.com/spm>. > >>>>> > >>>>> Regards, > >>>>> Emir > >>>>> > >>>>> -- > >>>>> Monitoring * Alerting * Anomaly Detection * Centralized Log > Management > >>>>> Solr & Elasticsearch Support * http://sematext.com/ > >>>>> > >>>>> On 10.08.2016 05:20, Midas A wrote: > >>>>> > >>>>>> Thanks for replying > >>>>>> > >>>>>> index size:9GB > >>>>>> 2000 docs/sec. > >>>>>> > >>>>>> Actually earlier it was taking less but suddenly it has increased . > >>>>>> > >>>>>> Currently we do not have any monitoring tool. > >>>>>> > >>>>>> On Tue, Aug 9, 2016 at 7:00 PM, Emir Arnautovic < > >>>>>> emir.arnauto...@sematext.com> wrote: > >>>>>> > >>>>>> Hi Midas, > >>>>>>> > >>>>>>> Can you give us more details on your index: size, number of new > docs > >>>>>>> between commits. Why do you think 1.3s for commit is to much and > why > >>>>>>> do > >>>>>>> you > >>>>>>> need it to take less? Did you do any system/Solr monitoring? > >>>>>>> > >>>>>>> Emir > >>>>>>> > >>>>>>> > >>>>>>> On 09.08.2016 14:10, Midas A wrote: > >>>>>>> > >>>>>>> please reply it is urgent. > >>>>>>>> > >>>>>>>> On Tue, Aug 9, 2016 at 11:17 AM, Midas A <test.mi...@gmail.com> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi , > >>>>>>>> > >>>>>>>>> commit is taking more than 1300 ms . what should i check on > server. > >>>>>>>>> > >>>>>>>>> below is my configuration . > >>>>>>>>> > >>>>>>>>> <autoCommit> <maxTime>${solr.autoCommit.maxTime:15000}</maxTime> > < > >>>>>>>>> openSearcher>false</openSearcher> </autoCommit> <autoSoftCommit> > >>>>>>>>> <maxTime> > >>>>>>>>> ${solr.autoSoftCommit.maxTime:-1}</maxTime> </autoSoftCommit> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>> > >>>>>>> Monitoring * Alerting * Anomaly Detection * Centralized Log > Management > >>>>>>> Solr & Elasticsearch Support * http://sematext.com/ > >>>>>>> > >>>>>>> > >>>>>>> > >> > >> -- > >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management > >> Solr & Elasticsearch Support * http://sematext.com/ > >> > > -- -------------------------- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
