I think it really depends - if you are gong for very fast visibility, your going to spend a bunch of time warming, and then just throw it out before it even gets much if any reuse. For very fast visibility turnaround, I suspect you don't want to do any warming. I think it depends on many things.
One of the tradeoffs off using a very fast soft commit is that Sol's std caches will not be nearly as useful. - Mark On Jan 10, 2013, at 11:24 AM, Upayavira <u...@odoko.co.uk> wrote: > That's great Mark. Thx. One final question... all the stuff to do with > autowarming and static warming of caches - I presume all of that > configuration is still relevant (if less so) as you still need to warm > caches on a soft commit, even if those caches are much smaller than they > would be otherwise? > > Thanks! Upayavira > > On Thu, Jan 10, 2013, at 04:18 PM, Mark Miller wrote: >> There is no need to open a Searcher because you are controlling >> visibility through the faster 'soft' commit. That will reopen the reader >> from the IndexWriter. Because of that, there is no reason to do a heavy, >> non NRT Searcher reopen on hard commits. Essentially, the hard commit >> becomes simply about periodically flushing the tlog and the soft commit >> completely controls visibility. >> >> - Mark >> >> On Jan 10, 2013, at 9:41 AM, Upayavira <u...@odoko.co.uk> wrote: >> >>> And you don't need to open a searcher (openSearcher=false) because >>> you've got caches built up already alongside the in-memory NRT segment >>> which you can continue to use once the hard commit has happened? Is that >>> correct? >>> >>> (sorry for hijacking the thread - hopefully it is somewhat relevant) >>> >>> Upayavira >>> >>> On Thu, Jan 10, 2013, at 02:18 PM, Mark Miller wrote: >>>> Setup hard auto commit with openSeacher=false. I would do it at least >>>> once a minute. Don't worry about the commit being out of sync on the >>>> different nodes - you will be using soft commits for visibility. The hard >>>> commits will just be about relieving the pressure on the tlog. >>>> >>>> - Mark >>>> >>>> On Jan 10, 2013, at 6:43 AM, gadde <gadde....@gmail.com> wrote: >>>> >>>>> we have a SolrCloud with 3 nodes. we add documents to leader node and use >>>>> commitwithin(100secs) option in SolrJ to add documents. AutoSoftCommit in >>>>> SolrConfig is 1000ms. >>>>> >>>>> Transaction logs on replicas grew bigger than the index and we ran out of >>>>> disk space in few days. Leader's tlogs are very small in few hundred MBs. >>>>> >>>>> The following post suggest hard commit is required for "relieving the >>>>> memory >>>>> pressure of the transactionlog" >>>>> http://lucene.472066.n3.nabble.com/SolrCloud-is-softcommit-cluster-wide-for-the-collection-td4021584.html#a4021631 >>>>> >>>>> what is the best way to do a hard commit on this setup in SolrCloud? >>>>> >>>>> a. Through autoCommit in SolrConfig? which would cause hard commit on all >>>>> the nodes at different times b. Trigger hard commit on leader while >>>>> updating >>>>> through SolrJ? >>>>> >>>>> Thanks >>>>> Shyam >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://lucene.472066.n3.nabble.com/SolrCloud-large-transaction-logs-tp4032160.html >>>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>> >>
