I've now worked on three different search engines and they all have a
3X worst
case on space, so I'm familiar with this case. --wunder
On Oct 1, 2009, at 7:15 AM, Mark Miller wrote:
Nice one ;) Its not technically a case where optimize requires > 2x
though in case the user asking gets confused. Its a case unrelated to
optimize that can grow your index. Then you need < 2x for the
optimize,
since you won't copy the deletes.
It also requires that you jump hoops to delete everything. If you
delete
everything with *:*, that is smart enough not to just do a delete on
every document - it just creates a new index, allowing the removal of
the old very efficiently.
Def agree on the more disk space.
Walter Underwood wrote:
Here is how you need 3X. First, index everything and optimize. Then
delete everything and reindex without any merges.
You have one full-size index containing only deleted docs, one
full-size index containing reindexed docs, and need that much space
for a third index.
Honestly, disk is cheap, and there is no way to make Lucene work
reliably with less disk. 1TB is a few hundred dollars. You have a
free
search engine, buy some disk.
wunder
On Oct 1, 2009, at 6:25 AM, Grant Ingersoll wrote:
151GB or as little as from 183GB to 182GB. Is that size after a
commit close to the size the index would be after an optimize? For
that matter, are there cases where optimization can take more than
2x? I've heard of cases but have not observed them in my system.
I seem to recall a case where it can be 3x, but I don't know that it
has been observed much.
--
- Mark
http://www.lucidimagination.com
