Thanks Otis, for this clarification! That means I will have to descrease the commit frequency to speed up indexing. How could I do this if I don't want to introduce an artificial delay time? ... via increasing the batch size?
Today I have read in another thread[1] that one should univert the field? What is it and how can I do this? Regards, Peter. [1] http://www.mail-archive.com/solr-user@lucene.apache.org/msg36113.html > As you make changes to your index, you probably want to see the new/modified > documents in your search results. In order to do that, the new searcher > needs > to be reopened, and this happens on commit. > Otis > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > ----- Original Message ---- > >> From: Peter Karich <peat...@yahoo.de> >> To: solr-user@lucene.apache.org >> Sent: Fri, July 30, 2010 6:19:03 PM >> Subject: Re: Solr Indexing slows down >> >> Hi Otis, >> >> does it mean that a new searcher is opened after I commit? >> I thought only on startup...(?) >> >> Regards, >> Peter. >> >> >>> Peter, there are events in solrconfig where you define warm up queries >>> when >>> >> a >> >> >>> new searcher is opened. >>> >>> There are also cache settings that play a role here. >>> >>> 30-60 seconds is pretty frequent for Solr. >>> >>> Otis >>> ---- >>> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch >>> Lucene ecosystem search :: http://search-lucene.com/ >>> >>> >>> >>> ----- Original Message ---- >>> >>> >>>> From: Peter Karich <peat...@yahoo.de> >>>> To: solr-user@lucene.apache.org >>>> Sent: Fri, July 30, 2010 4:06:48 PM >>>> Subject: Re: Solr Indexing slows down >>>> >>>> Hi Erick! >>>> >>>> thanks for the response! >>>> I will answer your questions ;-) >>>> >>>> >>>> >>>>> How often are you making changes to your index? >>>>> >>>>> >>>> Every 30-60 seconds. Too heavy? >>>> >>>> >>>> >>>> >>>>> Do you have autocommit on? >>>>> >>>>> >>>> No. >>>> >>>> >>>> >>>> >>>>> Do you commit when updating each document? >>>>> >>>>> >>>> No. I commit after a batch update of 200 documents >>>> >>>> >>>> >>>> >>>>> Committing too often and consequently firing off warmup queries is the >>>>> >> first >> >> >>>>> >>>>> >>>> place I'd look. >>>> >>>> Why is commiting firing warmup queries? Is there any documentation about >>>> this subject? >>>> How can I be sure that the previous commit has done its magic? >>>> >>>> >>>> >>>>> there are several config values that influence the commit frequency >>>>> >>>>> >>>> I now know the autowarm and the mergeFactor config. What else? Is this >>>> documentation complete: >>>> http://wiki.apache.org/lucene-java/ImproveIndexingSpeed ? >>>> >>>> Regards, >>>> Peter. >>>> >>>> >>>> >>>>> See the subject about 1500 threads. The first place I'd look is how >>>>> often you're committing. If you're committing before the warmup queries >>>>> from the previous commit have done their magic, you might be getting >>>>> into a death spiral. >>>>> >>>>> HTH >>>>> Erick >>>>> >>>>> On Thu, Jul 29, 2010 at 7:02 AM, Peter Karich <peat...@yahoo.de> >>>>> > wrote: > >>>>> >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> I am indexing a solr 1.4.0 core and commiting gets slower and slower. >>>>>> Starting from 3-5 seconds for ~200 documents and ending with over 60 >>>>>> seconds after 800 commits. Then, if I reloaded the index, it is as >>>>>> > fast > >>>>>> as before! And today I have read a similar thread [1] and indeed: if I >>>>>> set autowarming for the caches to 0 the slowdown disappears. >>>>>> >>>>>> BUT at the same time I would like to offer searching on that core, >>>>>> > which > >>>>>> would be dramatically slowed down (due to no autowarming). >>>>>> >>>>>> Does someone know a better solution to avoid index-slow-down? >>>>>> >>>>>> Regards, >>>>>> Peter. >>>>>> >>>>>> [1] http://www.mail-archive.com/solr-user@lucene.apache.org/msg20785.html >>>>>> >>>>>> >>>>>>
