Wow, this is weird. I commit before I optimize. In fact, I bounce
tomcat before I optimize just in case. It makse sense, as you say, that
then "the open reader can only be holding references to segments that
wouldn't be deleted until the optimize is complete anyway".
But we're still exceeding 2x. And after the optimize fails, if we then
do a commit or bounce tomcat, a bunch of segments disappear. I am stumped.
Yonik Seeley wrote:
On Wed, Oct 7, 2009 at 1:50 PM, Phillip Farber <pfar...@umich.edu> wrote:
So this implies that for a "normal" optimize, in every case, due to the
Searcher holding open the existing segment prior to optimize that we'd
always need 3x even in the normal case.
This seems wrong since it is repeated stated that in the normal case only 2x
is needed and I have successfully optimized a similar sized 192G index on
identical hardware with a 400G capacity.
2x for the IndexWriter only.
Having an open index reader can increase that somewhat... 3x is the
absolute worst case I think and that can currently be avoided by first
calling commit and then calling optimize I think. This way the open
reader will only be holding references to segments that wouldn't be
deleted until the optimize is complete anyway.
-Yonik
http://www.lucidimagination.com
