Sorry, I guess I don't understand the details of replication enough. So slave tries to replicate. It pulls down the new index files. It tries to do a commit but fails. But "the next commit that does succeed will have all the updates." Since it's a slave, it doesn't get any commits of it's own. But then some amount of time later, it does another replication pull. There are at this time maybe no _new_ changes since the last failed replication pull. Does this trigger a commit that will get those previous changes actually added to the slave?
In the meantime, between commits.. are those potentially large pulled new index files sitting around somewhere but not replacing the old slave index files, doubling disk space for those files? Thanks for any clarification. Jonathan ________________________________________ From: [email protected] [[email protected]] On Behalf Of Yonik Seeley [[email protected]] Sent: Monday, December 13, 2010 10:41 PM To: [email protected] Subject: Re: OutOfMemory GC: GC overhead limit exceeded - Why isn't WeakHashMap getting collected? On Mon, Dec 13, 2010 at 9:27 PM, Jonathan Rochkind <[email protected]> wrote: > Yonik, how will maxWarmingSearchers in this scenario effect replication? If > a slave is pulling down new indexes so quickly that the warming searchers > would ordinarily pile up, but maxWarmingSearchers is set to 1.... what > happens? Like any other commits, this will limit the number of searchers warming in the background to 1. If a commit is called, and that tries to open a new searcher while another is already warming, it will fail. The next commit that does succeed will have all the updates though. Today, this maxWarmingSearchers check is done after the writer has closed and before a new searcher is opened... so calling commit too often won't affect searching, but it will currently affect indexing speed (since the IndexWriter is constantly being closed/flushed). -Yonik http://www.lucidimagination.com
