Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-26 Thread Erick Erickson
right, autoCommit (in solrconfig.xml) will 1> close the current Lucene segments and open a new one 2> close the tlog and start a new one. Those actions are independent of whether openSearcher=true or false. if (and only if) openSearcher=true, then the commits will be immediately visible to a query

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-26 Thread Vincenzo D'Amore
Thanks Erick for your willingness and patience, if I understood well when autoCommit with openSearcher=true at first commit (soft or hard) all new documents will be automatically available for search. But when openSearcher=false, the commit will flush recent index changes to stable storage, but do

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-25 Thread Erick Erickson
The design is that the latest successfully flushed tlog file is kept for "peer sync" in SolrCloud mode. When a replica comes up, there's a chance that it's not very many docs behind. So, if possible, some of the docs are taken from the leader's tlog and replayed to the follower that's just been sta

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-25 Thread Vincenzo D'Amore
Hi Erick, I have tried indexing code I have few times, this is the behaviour I have tried out: When an indexing process starts, even if one or more tlog file exists, a new tlog file is created and all the new documents are stored there. When indexing process ends and does an hard commit, older ol

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-25 Thread Erick Erickson
OK, assuming you're not doing any commits at all until the very end, then the tlog contains all the docs for the _entire_ run. The article really doesn't care whether the commits come from the solrconfig.xml or SolrJ client or curl. The tlog simply is not truncated until a hard commit happens, no m

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-25 Thread Vincenzo D'Amore
Hi Erick, thanks for your support. Reading the post I realised that my scenario does not apply the autoCommit configuration, now we don't have autoCommit in our solrconfig.xml. We need docs are searchable only after the indexing process, and all the documents are committed only at end of index pr

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-24 Thread Erick Erickson
Vincenzo: Here's perhaps more than you want to know about hard commits, soft commits and transaction logs: http://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ Best, Erick On Sun, May 24, 2015 at 12:04 AM, Vincenzo D'Amore wrote: > Thanks Shawn for your

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-24 Thread Vincenzo D'Amore
Thanks Shawn for your prompt support. Best regards, Vincenzo On Sun, May 24, 2015 at 6:45 AM, Shawn Heisey wrote: > On 5/23/2015 9:41 PM, Vincenzo D'Amore wrote: > > Thanks Shawn, > > > > may be this is a silly question, but I looked around and didn't find an > > answer... > > Well, could I upd

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-23 Thread Shawn Heisey
On 5/23/2015 9:41 PM, Vincenzo D'Amore wrote: > Thanks Shawn, > > may be this is a silly question, but I looked around and didn't find an > answer... > Well, could I update solrconfig.xml for the collection while the instances > are running or should I restart the cluster/reload the cores? You ca

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-23 Thread Vincenzo D'Amore
Thanks Shawn, may be this is a silly question, but I looked around and didn't find an answer... Well, could I update solrconfig.xml for the collection while the instances are running or should I restart the cluster/reload the cores? On Sun, May 24, 2015 at 5:07 AM, Shawn Heisey wrote: > On 5/23

Re: SolrCloud 4.8 - Transaction log size over 1GB

2015-05-23 Thread Shawn Heisey
On 5/23/2015 8:56 PM, Vincenzo D'Amore wrote: > looking at tlog size I see there are many collection that have keep more > than 1GB of space. > Tlog are growing and the code that adds new documents never does an hard > commit. > > The question is must I fix the code that update the collections or

SolrCloud 4.8 - Transaction log size over 1GB

2015-05-23 Thread Vincenzo D'Amore
Hi, looking at tlog size I see there are many collection that have keep more than 1GB of space. Tlog are growing and the code that adds new documents never does an hard commit. The question is must I fix the code that update the collections or can I do an hard commit externally using collection a

Re: transaction log size

2014-04-02 Thread Erick Erickson
> > > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/transaction-log-size-tp4128354p4128547.html > Sent from the Solr - User mailing list archive at Nabble.com.

Re: transaction log size

2014-04-02 Thread Gurfan
-in-sorlcloud/ How can we reduce Tlog size at its lowest, so that our system restart up time will less. Thanks, --Gurfan -- View this message in context: http://lucene.472066.n3.nabble.com/transaction-log-size-tp4128354p4128547.html Sent from the Solr - User mailing list archive at

Re: transaction log size

2014-04-01 Thread Shawn Heisey
ng autoSoftCommit would generate the > small size of transaction log". Transaction log size is purely controlled by hard commits (autoCommit), soft commits have no influence at all. > To test the aforesaid statement we executed some Run: > > Document Size: ~2KB. > > 1st Run: &g

transaction log size

2014-04-01 Thread Gurfan
l > Solr-config.xml > transactionLog/master/* > transactionLog/slave/* May you please give us some pointer so that we can control on Transaction log(Tlog) generation. Thanks, --Gurfan -- View this message in context: http://lucene.472066.n3.nabble.com/transaction-log-size-tp4128354.ht