Re: Deleted documents and expungeDeletes

2016-04-01 Thread David Santamauro
The docs on reclaimDeletesWeight say: "Controls how aggressively merges that reclaim more deletions are favored. Higher values favor selecting merges that reclaim deletions." I can't imagine you would notice anything after only a few commits. I have many shards that size or larger and what I

Re: Deleted documents and expungeDeletes

2016-04-01 Thread Jostein Elvaker Haande
On 30 March 2016 at 17:46, Erick Erickson wrote: > through a clever bit of reflection, you can set the > reclaimDeletesWeight variable from solrconfig by including something > like > 5 (going from memory > here, you'll get an error on startup if I've messed it up.) I added the following to my

Re: Deleted documents and expungeDeletes

2016-03-30 Thread Erick Erickson
through a clever bit of reflection, you can set the reclaimDeletesWeight variable from solrconfig by including something like 5 (going from memory here, you'll get an error on startup if I've messed it up.) That may help.. Best, Erick On Wed, Mar 30, 2016 at 6:15 AM, David Santamauro wrote

Re: Deleted documents and expungeDeletes

2016-03-30 Thread David Santamauro
On 03/30/2016 08:23 AM, Jostein Elvaker Haande wrote: On 30 March 2016 at 12:25, Markus Jelsma wrote: Hello - with TieredMergePolicy and default reclaimDeletesWeight of 2.0, and frequent updates, it is not uncommon to see a ratio of 25%. If you want deletes to be reclaimed more often, e.g.

Re: Deleted documents and expungeDeletes

2016-03-30 Thread Jostein Elvaker Haande
On 30 March 2016 at 12:25, Markus Jelsma wrote: > Hello - with TieredMergePolicy and default reclaimDeletesWeight of 2.0, and > frequent updates, it is not uncommon to see a ratio of 25%. If you want > deletes to be reclaimed more often, e.g. weight of 4.0, you will see very > frequent merging

Re: Deleted documents and expungeDeletes

2016-03-30 Thread Jostein Elvaker Haande
On 30 March 2016 at 02:49, Erick Erickson wrote: > Please specify "growing and growing", Until it gets to 15% or more of the > total > then I'd start to worry. And then only if it kept growing after that. I tested 'expungeDeletes' on four different cores, three of them were nearly identical in t

RE: Deleted documents and expungeDeletes

2016-03-30 Thread Markus Jelsma
spinning disks. Markus -Original message- > From:Erick Erickson > Sent: Wednesday 30th March 2016 2:50 > To: solr-user > Subject: Re: Deleted documents and expungeDeletes > > bq: where I see that the number of deleted documents just > keeps on growing and growing

Re: Deleted documents and expungeDeletes

2016-03-29 Thread Erick Erickson
bq: where I see that the number of deleted documents just keeps on growing and growing, but they never seem to be deleted This shouldn't be happening. The default TieredMergePolicy weights segments to be merged (which happens automatically) heavily as per the percentage of deleted docs. Here's a

Deleted documents and expungeDeletes

2016-03-29 Thread Jostein Elvaker Haande
Hello everyone, I apologise beforehand if this is a question that has been visited numerous times on this list, but after hours spent on Google and talking to SOLR savvy people on #solr @ Freenode I'm still a bit at a loss about SOLR and deleted documents. I have quite a few indexes in both produ