Probably more than you want to know about commits, hard and soft: https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
Best, Erick On Thu, Mar 10, 2016 at 3:40 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On 3/10/2016 4:06 PM, Steven White wrote: >> Last question on this topic (maybe), wouldn't a commit at the very end take >> too long on a 1 billion items? Wouldn't a commit every, lets say 10,000 >> items be more efficient? > > The behavior that I have witnessed suggests that commit speed on a > well-tuned index depends more on the autowarm config than anything > else. The total size of the index might make a difference, but I > suspect that the slow commit times I've seen on large shards are just > from the autowarming -- each warming query takes longer if the index is > large. > > If you have the autoCommit config I recommended, the "last" commit > should be very fast, because those auto commits will flush data to disk > as you index, and the final manual commit should only need to deal with > data that has not yet been flushed. > > More info than you wanted (TL;DR): Even if you don't do the autoCommit, > you'll find that indexing tons of data without any commit at all *will* > cause older segments to be flushed to disk ... but the transaction logs > won't be rotated, and that's a whole separate problem. > > Thanks, > Shawn >
