Re: Period on-line index optimization

2018-11-28 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Erick, On 11/27/18 20:47, Erick Erickson wrote: > And do note one implication of the link Shawn gave you. Now that > you've optimized, you probably have one huge segment. It _will not_ > be merged unless and until it has < 2.5G "live" documents. So

Re: Period on-line index optimization

2018-11-27 Thread Erick Erickson
And do note one implication of the link Shawn gave you. Now that you've optimized, you probably have one huge segment. It _will not_ be merged unless and until it has < 2.5G "live" documents. So you may see your percentage of deleted documents get quite a bit larger than you've seen before merging

Re: Period on-line index optimization

2018-11-27 Thread Shawn Heisey
On 11/27/2018 10:04 AM, Christopher Schultz wrote: So, it's pretty much like GC promotion: the number of live objects is really the only things that matters? That's probably a better analogy than most anything else I could come up with. Lucene must completely reconstruct all of the index dat

Re: Period on-line index optimization

2018-11-27 Thread Walter Underwood
There is one case where optimize makes sense. You do a full reload of content rarely, maybe once per day or once per week. You use a master/slave cluster. Your index isn’t huge (say under 1 million docs). We have exactly that setup for our textbook search. We do not run optimize. Our median resp

Re: Period on-line index optimization

2018-11-27 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Walter, On 11/27/18 12:31, Walter Underwood wrote: > Optimize is just forcing a full merge. Solr does merges > automatically in the background. Understood. > It has been automatically doing merges for the months you’ve been > using it. Let it cont

Re: Period on-line index optimization

2018-11-27 Thread Walter Underwood
Optimize is just forcing a full merge. Solr does merges automatically in the background. It has been automatically doing merges for the months you’ve been using it. Let it continue. Don’t bother with optimize. It was a huge mistake to name that function “optimize”. Ultraseek had a button lable

Re: Period on-line index optimization

2018-11-27 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Shawn, On 11/27/18 11:01, Shawn Heisey wrote: > On 11/27/2018 7:47 AM, Christopher Schultz wrote: >> I've got a single-core Solr instance with something like 1M small >> documents in it. It contains user information for fast-lookups, >> and it gets

Re: Period on-line index optimization

2018-11-27 Thread Shawn Heisey
On 11/27/2018 7:47 AM, Christopher Schultz wrote: I've got a single-core Solr instance with something like 1M small documents in it. It contains user information for fast-lookups, and it gets updated any time relevant user-info changes. Here's the basic info from the Core Dashboard: I'm won