Well, you'd have to understand the whole way the index structure is laid out to do binary editing, and I don't know it well enough to even offer a rough idea. There are detailed docs hanging around _somewhere_ that will give you the formats, or you could go at the code. But that's probably pretty hairy.
I'm surprised you're getting that many files, something sounds really screwy. I'm not at all surprised that the index tries to double _temporarily_ when optimizing, that's expected behavior. But it should go back down after the optimization/forcemerge is complete. Problem is that now you've got some garbage left around that prevents your original index from being copied while optimizing, and getting rid of the unnecessary files seems fraught. I'm also surprised that you wound up with a space problem when you tried to optimize unless you have compound file format turned on, which I doubt. Could you wipe the index from that disk and re-copy the original and try again? So I'm going to chicken out and leave it to my betters to offer any advice, I don't play in the lower-level file structures. Sorry I can't be more help Erick On Tue, Jun 26, 2012 at 11:46 AM, avenka <ave...@gmail.com> wrote: > So, I tried 'optimize', but it failed because of lack of space on the first > machine. I then moved the whole thing to a different machine where the index > was pretty much the only thing and was using about 37% of disk, but it still > failed because of a "No space left on device" IOException. Also, the size of > the index has since doubled to roughly 74% of the disk on this second > machine now and the number of files has increased from 3289 to 3329. > Actually even the 3289 files on the first machine were after I tried > optimize on it once, so the "original" size must have been even smaller. > > I don't think I can afford any more space and am close to giving up and > reclaiming space on the two machines. A couple more questions before that: > > 1) I am tempted to try editing binary--the "magnetic needle" option. Could > you elaborate on this? Would there be a way to go back to an index that is > the original size from its super-sized current form(s)? > > 2) Will CheckIndex also need more than twice the space? Would there be a way > to bring down the size to the original size without running 'optimize' if I > try that route? Also how exactly do I run CheckIndex, e.g., the exact URL I > need to hit? > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/solr-java-lang-NullPointerException-on-select-queries-tp3989974p3991400.html > Sent from the Solr - User mailing list archive at Nabble.com.
