BTW, the place optimize seems best used is when the index isn't updated very often. I've seen a pattern where the index is updated once a night (or even less). In that situation, optimization makes more sense. But when an index is continually updated, it's mostly wasted effort.
Best, Erick On Thu, Apr 14, 2016 at 10:17 AM, Erick Erickson <erickerick...@gmail.com> wrote: > Unless you have somewhat unusual circumstances, I wouldn't optimize at > all, despite the name it really doesn't help all that much in _most_ > cases. > > If your percentage deleted docs doesn't exceed, say, 15-20% I wouldn't > bother. Most of what optimize does is reclaim resources from deleted > docs. This happens as part of general background merging anyway. > > There have been some reports of 10-15% query performance after > optimizing, but I would measure on your system before expending the > resources optimizing. > > Best, > Erick > > On Thu, Apr 14, 2016 at 9:56 AM, Novin Novin <toe.al...@gmail.com> wrote: >> Thanks Erick, for pointing out. You are right. I was optimizing every 10 >> mins. And I have change this to every day in night. >> On 14-Apr-2016 5:20 pm, "Erick Erickson" <erickerick...@gmail.com> wrote: >> >>> don't issue an optimize command... either you have a solrj client that >>> issues a client.optimize() command or you pressed the "optimize now" >>> in the admin UI. Solr doesn't do this by itself. >>> >>> Best, >>> Erick >>> >>> On Thu, Apr 14, 2016 at 8:30 AM, Novin Novin <toe.al...@gmail.com> wrote: >>> > How can I stop happening "DirectUpdateHandler2 Starting optimize... >>> Reading >>> > and rewriting the entire index! Use with care" >>> > >>> > Thanks >>> > novin >>> > >>> > On 14 April 2016 at 14:36, Shawn Heisey <apa...@elyograg.org> wrote: >>> > >>> >> On 4/14/2016 7:23 AM, Novin Novin wrote: >>> >> > Thanks for reply Shawn. >>> >> > >>> >> > Below is snippet of jetty.xml and jetty-https.xml >>> >> > >>> >> > jetty.xml:38: <Set name="idleTimeout" type="int"><Property >>> >> > name="solr.jetty.threads.idle.timeout" default="5000"/></Set> >>> >> > /// I presume this one I should increase, But I believe 5 second is >>> >> enough >>> >> > time for 250 docs to add to solr. >>> >> >>> >> 5 seconds might not be enough time. The *add* probably completes in >>> >> time, but the entire request might take longer, especially if you use >>> >> commit=true with the request. I would definitely NOT set this timeout >>> >> so low -- requests that take longer than 5 seconds are very likely going >>> >> to happen. >>> >> >>> >> > I'm also seeing "DirectUpdateHandler2 Starting optimize... Reading and >>> >> > rewriting the entire index! Use with care". Would this be causing >>> delay >>> >> > response from solr? >>> >> >>> >> Exactly how long an optimize takes is dependent on the size of your >>> >> index. Rewriting an index that's a few hundred megabytes may take 30 >>> >> seconds to a minute. Rewriting an index that's several gigabytes will >>> >> take a few minutes. Performance is typically lower during an optimize, >>> >> because the CPU and disks are very busy. >>> >> >>> >> Thanks, >>> >> Shawn >>> >> >>> >> >>>
