First, don't optimize after every chunk, it's just making extra work for your system. If you're using a 3.x or trunk build, optimizing doesn't do much for you anyway, but if you must, just optimize after your entire import is done.
Optimizing will pretty much copy the old index into a new set of files, so you can expect your disk space to at least double because Solr/Lucene doesn't delete anything until it's sure that the optimize finished successfully. Imagine the consequence of deleting files as they were copied to save disk space. Now hit a program error, power glitch or ctrl-c. Your indexes would be corrupted. Best Erick On Mon, Nov 29, 2010 at 6:07 AM, stockii <st...@shopgate.com> wrote: > > Hello. > > i have ~37 Million Docs that i want to index. > > when i starte a full-import i importing only every 2 Million Docs, because > of better controll over solr and space/heap .... > > so when i import 2 million docs and solr start the commit and the optimize > my used disc-space jumps into the sky. reacten: solr restart and space the > used space goes down. > > why is using solr so many space ? > > can i optimize that ? > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Large-Hdd-Space-using-during-commit-optimize-tp1985807p1985807.html > Sent from the Solr - User mailing list archive at Nabble.com. >
