SolrCloud will update Zookeeper on state changes (node goes to recovery, comes back up etc) or for leader election and during collection API commands. It doesn't correlate directly with indexing but is correlated with how frequently you call commit.
On Wed, Mar 19, 2014 at 5:46 AM, Shawn Heisey <s...@elyograg.org> wrote: > On 3/18/2014 5:46 PM, Chris W wrote: >> >> I am running a 3 node zookeeper 3.4.5 Quorum. I am running into issues >> with Zookeeper transaction logs >> >> [myid:2] - ERROR [main:QuorumPeer@453] - Unable to load database on disk >> java.io.IOException: Unreasonable length = 1048587 >> at >> org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:100) >> at >> org.apache.zookeeper.server.persistence.Util.readTxnBytes(Util.java:233) >> at >> >> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:602) >> at >> >> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:157) >> at >> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223) >> >> To unblock temporarily, i deleted the most recent txn log. How do i tell >> zookeeper to not grow the transaction log beyond x MegaBytes? >> >> How often does the transaction log get updated? Does zk transactions log >> grow everytime we index data into a new collection? > > > Zookeeper is a separate project at Apache. ZK file management is discussed > in the ZK documentation. > > http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance > > There is a bug filed on Zookeeper for the issue you are seeing, with a > fairly simple workaround. It is fixed in the 3.4.6 version, which was > released last week. I will see whether we can get ZK upgraded to 3.4.6 in > the Solr 4.8.0 release. I don't think we want to risk doing that upgrade in > 4.7.1, but I could be wrong. > > https://issues.apache.org/jira/browse/ZOOKEEPER-1513 > http://zookeeper.apache.org/releases.html > > I am actually not sure how often SolrCloud updates Zookeeper. It happens > whenever the collections API is called for sure, and it may happen anytime > you index data as well. > > Thanks, > Shawn > -- Regards, Shalin Shekhar Mangar.
