Thanks for the quick reply, Shawn. I am less worried about long transaction-log replays after crashes because in normal life (long uptimes, rare orderly restarts) this should never happen, and even if it does, uses will not be affected.
I am more worried about potential increase in heap usage due to transaction-logs, and possible affect on indexer stability, and am considering increasing the JVM heap - do you think this worry is realistic? Thanks, Michael On Wed, Jun 12, 2013 at 12:58 AM, Shawn Heisey <elyog...@elyograg.org>wrote: > On 6/11/2013 3:45 PM, Michael Tsadikov wrote: > >> We're upgrading our Solr 3.1 distributed masters/slaves setup to 4.3. >> >> In 3.1 we used autoCommit every hour on masters, each commit is replicated >> to slaves, and all searches are done on slaves. 1hr visibility is ok - we >> don't need NRT. >> >> In 4.3 we enable transaction logs, for durability in case of crashes. >> >> However, we are wondering how to best setup autoCommit/autoSoftCommit on >> masters to preserve the old behavior. It seems that setting autoCommit to >> 1hr (openSearcher=true) without any autoSoftCommit preserves our previous >> setup - is this correct? Wil the transaction log make masters use much >> more >> heap due to 1hr periods between commits? This can be a problem for us >> because we put many master cores on one solr JVM >> > > If you want to completely preserve your previous setup, then you've got it > correct. Depending on how much you index over the course of that hour, you > might want to go a different way. > > If you are doing hard commits once an hour, you'll end up with at least 2 > hours of indexing in your transaction log. If a few hours of indexing > represents a lot of data, then it can take a really long time to replay > those transactions when Solr restarts. If it's not a lot of data, then you > shouldn't have any trouble. > > http://wiki.apache.org/solr/**SolrPerformanceProblems#Slow_**startup<http://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup> > > If a few hours of indexing data *IS* a lot, then autoCommit every five > minutes with openSearcher=false and autoSoftCommit once an hour would > preserve your current visibility characteristics without giving you really > huge transaction logs. > > Thanks, > Shawn > >
