Thats brilliant. I am just starting to wonder if there anything at all that you guys haven't thought about ;) Thanks that setting should be really useful.
> Date: Wed, 10 Sep 2008 15:26:57 -0700 > From: [EMAIL PROTECTED] > To: solr-user@lucene.apache.org > Subject: RE: Question on how index works - runs out of disk space! > > > : OPtimize solved it . Thanks Jason. I am surprised on why solr does this? > > this gets into some complicated discussions about the underlying Lucnee > index format, this is discussed at a very low level in the Lucene docs... > > http://lucene.apache.org/java/2_3_2/fileformats.html > > ...but at a slightly higher level the issue comes from the basic nature of > an inverted index. even though you have a uniqueKey, and are "replacing" > an existing document, there is no easy way to reclaim the space used by > the previous version of the document in realtime -- instead a single bit > records that the old version was deleted, and the new version is added to > the end. > > the space used by those deleted docs is reclaimed when "segments" get > "merged". All segments are merged into one compact segment when you do an > optimize -- but an optimize isn't actaully neccessary to ensure that the > deleted docs are *eventually* purged, as documents are added, incremental > merges are constantly taking place. How often they take place (as a > function of docs added) can be controlled with various settings in > solrconfig.xml > > That is the root of why you can see an index grow even though you only > "replace" existing docswithout adding new docs ... it will grow and then > it will shrnk again once merging happens. > > On a slightly related topic: if you really want to explicitly forge some > segment merging, but a full optimize takes longer then you are willing to > wait, there is a new option in Solr 1.3 to support to support partial > optimiation... > > <optimize maxSegments="5" /> > > > -Hoss > _________________________________________________________________ Searching for weekend getaways? Try Live.com http://www.live.com/?scope=video&form=MICOAL
