Please consider never running "optimize". That should be called "force merge".
wunder On Oct 24, 2012, at 3:28 AM, Dotan Cohen wrote: > On Tue, Oct 23, 2012 at 3:07 PM, Erick Erickson <erickerick...@gmail.com> > wrote: >> Maybe you've been looking at it but one thing that I didn't see on a fast >> scan was that maybe the commit bit is the problem. When you commit, >> eventually the segments will be merged and a new searcher will be opened >> (this is true even if you're NOT optimizing). So you're effectively >> committing >> every 1-2 seconds, creating many segments which get merged, but more >> importantly opening new searchers (which you are getting since you pasted >> the message: Overlapping onDeckSearchers=2). >> >> You could pinpoint this by NOT committing explicitly, just set your >> autocommit >> parameters (or specify commitWithin in your indexing program, which is >> preferred). Try setting it at a minute or so and see if your problem goes >> away >> perhaps? >> >> The NRT stuff happens on soft commits, so you have that option to have the >> documents immediately available for search. >> > > > Thanks, Erick. I'll play around with different configurations. So far > just removing the periodic optimize command worked wonders. I'll see > how much it helps or hurts to run that daily or more or less frequent. > > > -- > Dotan Cohen > > http://gibberish.co.il > http://what-is-what.com
