Hello Chris, Thank you for the response, I am following up on the e-mail chain for Scott.
I guess we can try using a commit with expungeDeletes=true, but does not really address the underlying problem. If we hadn't issued the "optimize" in the past, thereby creating the 2 big segments, my understanding is that Solr would have had (many more) smaller segments, with deleted docs distributed across them. And in all likelihood, the behind the scenes execution of the tiered merge policy would have cleaned the deleted docs as segments merged, reclaiming the space. But now that we have the two big segments, is there a way for Solr to reclaim this space as part of its merge operation, or do we have to manually (either via optimize or expunge deletes) remove the deleted docs until we eat up all the docs in those big segments (i.e. as they are purged with our purge logic)? We are running Solr 4.2.1 with TieredMergePolicy maxMergeAtOnce=10 and segmentsPerTier=10 Thanks, Gun --- Senior Software Engineer Carbon Black, Inc. gun.ak...@carbonblack.com On Thu, Oct 24, 2013 at 6:11 PM, Chris Hostetter <hossman_luc...@fucit.org>wrote: > > I didn't dig into the details of your mail too much, but a few things > jumped out at me... > > : - At some time in the past, a manual force merge / optimize with > : maxSegments=2 was run to troubleshoot high disk i/o and remove "too many > > Have you tried a simple commit using expungeDeletes=true? It should be a > little less intensive then a optimizing. (under the covers it does > IndexWriter.forceMergeDeletes()) > > > : - Merge policies are all at Solr 4 defaults. Index size is currently ~50M > : maxDocs, ~35M numDocs, 276GB. > > "Solr 4 defaults" is way to vague to be meaningful: 4.0? 4.1? ... 4.4? > > Do you mean you are using the example configs that came with that version > of Solr, or do you mean you have no mergePolicy configured and you are > getting the hardcoded defaults? .. either way it's important to specify > exactly which version of Solr are you running and exactly what does your > entire <indexConfig/> section looks like since both the example configs > and the hardcoded default behavior when configs aren't specified have > evolved since 4.0-ALPHA. > > > > -Hoss >
