thanks shawn that was really helpful On Sat, Aug 20, 2016 at 3:17 AM, Shawn Heisey <apa...@elyograg.org> wrote:
> On 8/16/2016 11:47 AM, kshitij tyagi wrote: > > I need to understand clearly that is there any relationship between solr > > merging and solr commit? > > > > If there is then what is it? > > > > Also i need to understand how both of these affect indexing speed on the > > core? > > Whenever a new segment is written, the merge policy is checked to see > whether a merge is needed. If it is needed, then the merge is scheduled. > > A commit operation can (and frequently does) write a new segment, but > that is not the only thing that can write (flush) new segments. When > the indexing RAM buffer fills up, a segment will be flushed, even > without a commit. > > When paired with the default NRT Directory implementation, soft commits > change the dynamics slightly, but not the way things generally operate. > Soft commits are capable of flushing the latest segment(s) to memory, > instead of the disk, but only if they are quite small. > > I would not expect commits to *directly* affect indexing speed unless > you are doing commits extremely frequently. Commits might indirectly > affect indexing speed if they trigger a large merge. > > Merging can cause issues with indexing speed, even if it's happening in > a different Solr core on the same machine. This is because the system > resources (I/O bandwidth, memory, CPU) required for a merge are also > required to write a new segment. Also, because flushing a new segment > is effectively the same operation as the writing part of a merge, if too > many merges are scheduled at once on a core, indexing on that core can > stop entirely until the number of scheduled merges drops. > > Merging can also cause issues with query speed, if there is not > sufficient memory available to the OS for effective disk caching. > > Thanks, > Shawn > >
