Re: Optimizing cores in SolrCloud

2013-11-14 Thread Walter Underwood
Earlier, you said that optimize is the only way that deleted documents are expunged. That is false. They are expunged when the segment they are in is merged. A forced merge (optimize) merges all segments, so will expunge all deleted document. But those documents will be expunged by merges eventu

Re: Optimizing cores in SolrCloud

2013-11-14 Thread michael.boom
Thanks Erick! That's a really interesting idea, i'll try it! Another question would be, when does the merging actually happens? Is it triggered or conditioned by something? Currently I have a core with ~13M maxDocs and ~3M deleted docs, and although I see a lot of merges in SPM, deleted documents

Re: Optimizing cores in SolrCloud

2013-11-14 Thread Erick Erickson
I'm going to answer with something completely different First, though, optimization happens in the background, so it shouldn't have too big an impact on query performance outside of I/O contention. There also "shouldn't" be any problem with one shard being optimized and one not. Second, have