Hoss, Using Solr 1.4, I see constant index growth until an optimize. I commit (hundreds of updates) every 5 minutes and have a mergefactor of 10, but every 50 minutes I don't see the index collapse down to its original size -- it's slightly larger.
Over the course of a week, the index grew from 4.5 gigs to 6 gigs, growing and shrinking in file size and count but generally upward. Only when I manually optimized did the index return to 4.5 gigs. So -- I thought I understood you to mean that if I frequently merge, it's basically the same as an optimize, and cruft will get purged. Am I misunderstanding you? Michael PS: The extra 1.5G actually matters, as this is one of 8 cores and I'm trying to keep it all in RAM. On Tue, Nov 17, 2009 at 2:37 PM, Israel Ekpo <israele...@gmail.com> wrote: > On Tue, Nov 17, 2009 at 2:24 PM, Chris Hostetter > <hossman_luc...@fucit.org>wrote: > >> >> : Basically, search entries are keyed to other documents. We have finite >> : storage, >> : so we purge old documents. My understanding was that deleted documents >> : still >> : take space until an optimize is done. Therefore, if I don't optimize, >> the >> : index >> : size on disk will grow without bound. >> : >> : Am I mistaken? If I don't ever have to optimize, it would make my life >> : easier. >> >> deletions are purged as segments get merged. if you want to force >> deleted documents to be purged, the only way to do that at the >> moment is to optimize (which merges all segments). but if you are >> continually deleteing/adding documents, the deletions will eventaully get >> purged even if you never optimize. >> >> >> >> >> -Hoss >> >> > > Chris, > > Since the mergeFactor controls the segment merge frequency and size and the > number of segments is limited to mergeFactor - 1. > > Would one be correct to state that if some documents have been deleted from > the index and the changes finalized with a call to commit, as more documents > are added to the index, eventually the index will be implicitly "*optimized > *" and the deleted documents will be purged even without explicitly issuing > an optimize statement? > > > -- > "Good Enough" is not good enough. > To give anything less than your best is to sacrifice the gift. > Quality First. Measure Twice. Cut Once. >
