Re: Loss of "Optimise" button

2018-04-21 Thread Joe Doupnik
    Yet, that is a task which the main application, Solr, could and should undertake, rather than ask we human slaves to add sundry programs to tend it from afar.     Similarly, it would be useful for there to be feedback from Solr when adding material so that we don't overwhelm parts of the pip

Re: Loss of "Optimise" button

2018-04-21 Thread Erick Erickson
Yeah, trying to have something that satisfies all use cases is a bear. I know of one installation where the indexing rate was so huge that they couldn't afford to have any merging (80B docs/day) so in that situation any heuristics built into Solr would be wrong. Here's an alternate approach to hav

Re: Loss of "Optimise" button

2018-04-21 Thread Joe Doupnik
    A good find Erick, and one which brings into focus the real problem at hand. That overload case would happen if there were an Optimise button or if the curl equivalent command were issued, and is not a reason to avoid either/both.     So, what could be done to avoid such awkward difficultie

Re: Loss of "Optimise" button

2018-04-21 Thread Erick Erickson
Joe: Serendipity strikes, The thread titled "JVM Heap Memory Increase (SOLR CLOUD)" is a perfect example of why the optimize button is so "fraught". Best, Erick On Sat, Apr 21, 2018 at 9:43 AM, Erick Erickson wrote: > Joe: > > Thanks for moving the conversation over here that we were having on

Re: Loss of "Optimise" button

2018-04-21 Thread Joe Doupnik
On 21/04/2018 17:25, Doug Turnbull wrote: I haven’t tracked this change, but can you still optimize through the API? Here’s an example using update XML https://stackoverflow.com/questions/6954358/how-to-optimize-solr-index There are so many cases hitting “optimize” causes a huge segment merge

Re: Loss of "Optimise" button

2018-04-21 Thread Erick Erickson
Joe: Thanks for moving the conversation over here that we were having on the blog post. I think the wider audience will benefit from this going forward. bq: ...apparent inability to remove piles of deleted docs do note that deleted docs are removed during normal indexing when segments are merged

Re: Loss of "Optimise" button

2018-04-21 Thread Doug Turnbull
I haven’t tracked this change, but can you still optimize through the API? Here’s an example using update XML https://stackoverflow.com/questions/6954358/how-to-optimize-solr-index There are so many cases hitting “optimize” causes a huge segment merge that brings down a Solr cluster that I think

Loss of "Optimise" button

2018-04-21 Thread Joe Doupnik
    In Solr v7.3.0 the ability to removed "deleted" docs from a core by use of what until then was the Optmise button on the admin GUI has been changed in an ungood way. That is, in the V7.3.0 Changes list, item SOLR 7733 (quote remove "optmize from the UI, end quote). The result of that is an