On 3/18/2014 5:46 PM, Chris W wrote:
I am running a 3 node zookeeper 3.4.5 Quorum. I am running into issues
with Zookeeper transaction logs
[myid:2] - ERROR [main:QuorumPeer@453] - Unable to load database on disk
java.io.IOException: Unreasonable length = 1048587
at
org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:100)
at org.apache.zookeeper.server.persistence.Util.readTxnBytes(Util.java:233)
at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:602)
at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:157)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
To unblock temporarily, i deleted the most recent txn log. How do i tell
zookeeper to not grow the transaction log beyond x MegaBytes?
How often does the transaction log get updated? Does zk transactions log
grow everytime we index data into a new collection?
Zookeeper is a separate project at Apache. ZK file management is
discussed in the ZK documentation.
http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
There is a bug filed on Zookeeper for the issue you are seeing, with a
fairly simple workaround. It is fixed in the 3.4.6 version, which was
released last week. I will see whether we can get ZK upgraded to 3.4.6
in the Solr 4.8.0 release. I don't think we want to risk doing that
upgrade in 4.7.1, but I could be wrong.
https://issues.apache.org/jira/browse/ZOOKEEPER-1513
http://zookeeper.apache.org/releases.html
I am actually not sure how often SolrCloud updates Zookeeper. It
happens whenever the collections API is called for sure, and it may
happen anytime you index data as well.
Thanks,
Shawn
