And I can now confirm that yes, ZooKeeper blows up when I attempted to add all the UIMA and content extraction jars to my conf/ directory in ZooKeeper! A couple small jars did upload, and then it started sending back "java.io.IOException: Broken pipe" errors.
So any thoughts on the best way to manage Jars that seem like they should be part of your config? Small jars I think will work, and maybe I just need to tweak my <lib/> definitions in my solrconfig.xml to look for all the places that Jars may exist, even though on my local box it's different then on my integration Solr box. Just seems a bit messy ;-) Eric On Aug 14, 2012, at 4:40 PM, Jack Krupansky wrote: > Dear Eric The Brave, > > As per the wiki:"znodes are limited to the amount of data that they can have. > ZooKeeper was designed to store coordination data: status information, > configuration, location information, etc. This kind of meta-information is > usually measured in kilobytes, if not bytes. ZooKeeper has a built-in sanity > check of 1M, to prevent it from being used as a large data store, but in > general it is used to store much smaller pieces of data." > > See: > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ProjectDescription > > Also:"jute.maxbuffer: (Java system property: jute.maxbuffer) > This option can only be set as a Java system property. There is no zookeeper > prefix on it. It specifies the maximum size of the data that can be stored in > a znode. The default is 0xfffff, or just under 1M. If this option is changed, > the system property must be set on all servers and clients otherwise problems > will arise. This is really a sanity check. ZooKeeper is designed to store > data on the order of kilobytes in size." > > See: > http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html > > -- Jack Krupansky > > -----Original Message----- From: Eric Pugh > Sent: Tuesday, August 14, 2012 4:11 PM > To: solr-user@lucene.apache.org > Subject: Custom Jars for a config in the Solr Cloud world.. > > I've got a Solr instance with a number of cores that are each configured by > upload the configuration information to ZooKeeper. The newest index needs > the UIMA jars. Normally I would put them in the core's /lib directory, but > since I am only accessing my server via ZooKeeper, I don't have that > directory as an option. > > I know I could manually upload the jars onto the server, and then put some > sort of path to them, but I'm hoping to manage all uploading of core specific > configurations (and jars) via ZooKeeper. I'm wondering if I am missing > something in this new ZooKeeper enabled world....? Just for fun, I'm going > to try and put the ~ 2 MB worth of Jars inside my /conf/ directory and then > upload through ZooKeeper to see what happens. > > Eric > > > ----------------------------------------------------- > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com > Co-Author: Apache Solr 3 Enterprise Search Server available from > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > > > > > > > > > > ----------------------------------------------------- Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com Co-Author: Apache Solr 3 Enterprise Search Server available from http://www.packtpub.com/apache-solr-3-enterprise-search-server/book This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
