On 8/16/2016 11:47 AM, kshitij tyagi wrote: > I need to understand clearly that is there any relationship between solr > merging and solr commit? > > If there is then what is it? > > Also i need to understand how both of these affect indexing speed on the > core?
Whenever a new segment is written, the merge policy is checked to see whether a merge is needed. If it is needed, then the merge is scheduled. A commit operation can (and frequently does) write a new segment, but that is not the only thing that can write (flush) new segments. When the indexing RAM buffer fills up, a segment will be flushed, even without a commit. When paired with the default NRT Directory implementation, soft commits change the dynamics slightly, but not the way things generally operate. Soft commits are capable of flushing the latest segment(s) to memory, instead of the disk, but only if they are quite small. I would not expect commits to *directly* affect indexing speed unless you are doing commits extremely frequently. Commits might indirectly affect indexing speed if they trigger a large merge. Merging can cause issues with indexing speed, even if it's happening in a different Solr core on the same machine. This is because the system resources (I/O bandwidth, memory, CPU) required for a merge are also required to write a new segment. Also, because flushing a new segment is effectively the same operation as the writing part of a merge, if too many merges are scheduled at once on a core, indexing on that core can stop entirely until the number of scheduled merges drops. Merging can also cause issues with query speed, if there is not sufficient memory available to the OS for effective disk caching. Thanks, Shawn
