Re: Setting of ramBufferSizeMB

2016-01-14 Thread Erick Erickson
Yep. Here's Mike's classic video: http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html The third visualization down "TieredMergePolicy" is the default. Best, Erick On Wed, Jan 13, 2016 at 6:52 PM, Zheng Lin Edwin Yeo wrote: > Hi Erick, > > Thanks for your reply. > > So

Re: Setting of ramBufferSizeMB

2016-01-13 Thread Zheng Lin Edwin Yeo
Hi Erick, Thanks for your reply. So those small segments that I found is probably due to a commit happening during that time? I also found that those small segments are created during the last indexing. If I start another batch of indexing, those small segments will probably be get merge togethe

Re: Setting of ramBufferSizeMB

2016-01-13 Thread Erick Erickson
ramBufferSizeMB is a _limit_ that flushes the buffer when it is reached (actually, I think, it indexes a doc _then_ checks the size and if it's > the setting, flushes the buffer. So technically you can exceed the buffer size by your biggest doc's addition to the index). But I digress. This is a _l