Re: 'Optimizing' Solr Index Size

2013-08-07 Thread Brendan Grainger
Thanks Erick, our index is relatively static. I think the deletes must be coming from 'reindexing' the same documents so definitely handy to recover the space. I've seen that video before. Definitely very interesting. Brendan On Wed, Aug 7, 2013 at 8:04 AM, Erick Erickson wrote: > The general

Re: 'Optimizing' Solr Index Size

2013-08-07 Thread Erick Erickson
The general advice is to not merge (optimize) unless your index is relatively static. You're quite correct, optimizing simply recovers the space from deleted documents, otherwise it won't change much (except having fewer segments). Here's a _great_ video that Mike McCandless put together: http://b

Re: 'Optimizing' Solr Index Size

2013-08-06 Thread Brendan Grainger
To maybe answer another one of my questions about the 50Gb recovered when running: curl ' http://localhost:8983/solr/update?optimize=true&maxSegments=10&waitFlush=false ' It looks to me that it was from deleted docs being completely removed from the index. Thanks On Tue, Aug 6, 2013 at 11:45

Re: 'Optimizing' Solr Index Size

2013-08-06 Thread Brendan Grainger
Well, I guess I can answer one of my questions which I didn't exactly explicitly state, which is: how do I force solr to merge segments to a given maximum. I forgot about doing this: curl ' http://localhost:8983/solr/update?optimize=true&maxSegments=10&waitFlush=false ' which reduced the number o

'Optimizing' Solr Index Size

2013-08-06 Thread Brendan Grainger
Hi All, First of all, what I was actually trying to do is actually get a little space back. So if there is a better way to do this by adjusting the MergePolicy or something else please let me know. My index is currently 200Gb. In the past (Solr 1.4) we've found that optimizing the index will doubl