Hey Dipti - Can you give the exact startup cmds you are using for each of the instances? I have got Example C going, so I'll have to try and dig into whatever you are seeing.
- mark On Jan 27, 2012, at 12:53 PM, Dipti Srivastava wrote: > Hi Mark, > Did you get a chance to look into the issues with running the embedded > Zookeeper ensemble, as per Example C, from the > http://wiki.apache.org/solr/SolrCloud2 > > Hi All, > Did anyone else run multiple shards with embedded zk ensemble successfully? > If so would like some tips on any issues that you came across. > > Regards, > Dipti > > From: diptis <dipti.srivast...@apollogrp.edu> > Date: Fri, 23 Dec 2011 10:32:52 -0700 > To: "markrmil...@gmail.com" <markrmil...@gmail.com> > Subject: Re: Release build or code for SolrCloud > > Hi Mark, > There is some issue with specifying localhost vs actual host names for zk. > When I changed my script to specify the actual hostname (which should be > local by default) the first, 2nd and 3rd instances came up, that have the > embedded zk running. Now, I am getting the same exception for the 4th AMI > which in NOT part of the zookeeper ensemble. I want to zk only on 3 of the 4 > instances. > > java -Dbootstrap_confdir=./solr/conf –DzkRun="<ami-1>9983>" > -DzkHost=<ami-1>:9983,<ami-2>:9983,<ami-3>:9983 -DnumShards=2 -jar > start.jar > > Dipti > > From: Mark Miller <markrmil...@gmail.com> > Reply-To: "markrmil...@gmail.com" <markrmil...@gmail.com> > Date: Fri, 23 Dec 2011 09:34:52 -0700 > To: diptis <dipti.srivast...@apollogrp.edu> > Subject: Re: Release build or code for SolrCloud > > I'm having trouble getting a quorum up using the built in SolrZkServer as > well - so i have not been able to replicate this - I'll have to keep digging. > Not sure if it's due to a ZooKeeper update or what yet. > > 2011/12/21 Dipti Srivastava <dipti.srivast...@apollogrp.edu> >> Hi Mark, >> Thanks! So now I am deploying a 4 node cluster on AMI's and the main >> instance that bootstraps the config to the zookeeper does not come up I >> get an exception as follows. My solrcloud.sh looks like >> >> #!/usr/bin/env bash >> >> cd .. >> >> rm -r -f example/solr/zoo_data >> rm -f example/example.log >> >> cd example >> #java -DzkRun -DnumShards=2 -DSTOP.PORT=7983 -DSTOP.KEY=key -jar start.jar >> 1>example.log 2>&1 & >> java -Dbootstrap_confdir=./solr/conf -DzkRun >> -DzkHost=<ami-1>:9983,<ami-2>:9983,<ami-3>:9983 -DnumShards=2 -jar >> start.jar >> >> >> >> >> And when I RUN it---- >> >> --CLOUD--[ec2-user@ cloud-dev]$ ./solrcloud.sh >> 2011-12-22 02:18:23.352:INFO::Logging to STDERR via >> org.mortbay.log.StdErrLog >> 2011-12-22 02:18:23.510:INFO::jetty-6.1-SNAPSHOT >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader >> locateSolrHome >> INFO: JNDI not configured for solr (NoInitialContextEx) >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader >> locateSolrHome >> INFO: solr home defaulted to 'solr/' (could not find system property or >> JNDI) >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader <init> >> INFO: Solr home set to 'solr/' >> Dec 22, 2011 2:18:23 AM org.apache.solr.servlet.SolrDispatchFilter init >> INFO: SolrDispatchFilter.init() >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader >> locateSolrHome >> INFO: JNDI not configured for solr (NoInitialContextEx) >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader >> locateSolrHome >> INFO: solr home defaulted to 'solr/' (could not find system property or >> JNDI) >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.CoreContainer$Initializer >> initialize >> INFO: looking for solr.xml: /home/ec2-user/solrcloud/example/solr/solr.xml >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.CoreContainer <init> >> INFO: New CoreContainer 1406140084 >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader >> locateSolrHome >> INFO: JNDI not configured for solr (NoInitialContextEx) >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader >> locateSolrHome >> INFO: solr home defaulted to 'solr/' (could not find system property or >> JNDI) >> Dec 22, 2011 2:18:23 AM org.apache.solr.core.SolrResourceLoader <init> >> INFO: Solr home set to 'solr/' >> Dec 22, 2011 2:18:24 AM org.apache.solr.cloud.SolrZkServerProps >> getProperties >> INFO: Reading configuration from: solr/zoo.cfg >> Dec 22, 2011 2:18:24 AM org.apache.solr.cloud.SolrZkServerProps >> parseProperties >> INFO: Defaulting to majority quorums >> Dec 22, 2011 2:18:24 AM org.apache.solr.servlet.SolrDispatchFilter init >> SEVERE: Could not start Solr. Check solr/home property and the logs >> java.lang.IllegalArgumentException: port out of range:-1 >> at java.net.InetSocketAddress.<init>(InetSocketAddress.java:83) >> at java.net.InetSocketAddress.<init>(InetSocketAddress.java:63) >> at >> org.apache.solr.cloud.SolrZkServerProps.setClientPort(SolrZkServer.java:310 >> ) >> at >> org.apache.solr.cloud.SolrZkServerProps.getMySeverId(SolrZkServer.java:273) >> at >> org.apache.solr.cloud.SolrZkServerProps.parseProperties(SolrZkServer.java:4 >> 50) >> at >> org.apache.solr.cloud.SolrZkServer.parseConfig(SolrZkServer.java:85) >> at >> org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:147) >> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:329) >> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:282) >> at >> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.jav >> a:231) >> at >> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:93) >> at >> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713 >> ) >> at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) >> at >> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282 >> ) >> at >> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) >> at >> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java: >> 152) >> at >> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCo >> llection.java:156) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java: >> 152) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) >> at org.mortbay.jetty.Server.doStart(Server.java:224) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:3 >> 9) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp >> l.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.mortbay.start.Main.invokeMain(Main.java:194) >> at org.mortbay.start.Main.start(Main.java:534) >> at org.mortbay.start.Main.start(Main.java:441) >> at org.mortbay.start.Main.main(Main.java:119) >> Dec 22, 2011 2:18:24 AM org.apache.solr.common.SolrException log >> SEVERE: java.lang.IllegalArgumentException: port out of range:-1 >> at java.net.InetSocketAddress.<init>(InetSocketAddress.java:83) >> at java.net.InetSocketAddress.<init>(InetSocketAddress.java:63) >> at >> org.apache.solr.cloud.SolrZkServerProps.setClientPort(SolrZkServer.java:310 >> ) >> at >> org.apache.solr.cloud.SolrZkServerProps.getMySeverId(SolrZkServer.java:273) >> at >> org.apache.solr.cloud.SolrZkServerProps.parseProperties(SolrZkServer.java:4 >> 50) >> at >> org.apache.solr.cloud.SolrZkServer.parseConfig(SolrZkServer.java:85) >> at >> org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:147) >> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:329) >> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:282) >> at >> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.jav >> a:231) >> at >> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:93) >> at >> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713 >> ) >> at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) >> at >> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282 >> ) >> at >> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) >> at >> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java: >> 152) >> at >> org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCo >> llection.java:156) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java: >> 152) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) >> at org.mortbay.jetty.Server.doStart(Server.java:224) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:3 >> 9) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp >> l.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.mortbay.start.Main.invokeMain(Main.java:194) >> at org.mortbay.start.Main.start(Main.java:534) >> at org.mortbay.start.Main.start(Main.java:441) >> at org.mortbay.start.Main.main(Main.java:119) >> >> Dec 22, 2011 2:18:24 AM org.apache.solr.servlet.SolrDispatchFilter init >> INFO: SolrDispatchFilter.init() done >> 2011-12-22 02:18:24.186:INFO::Started SocketConnector@0.0.0.0:8983 >> >> >> >> >> On 12/21/11 1:26 PM, "Mark Miller" <markrmil...@gmail.com> wrote: >> >> >Hey Dipti - that error is normal - the script fires up a tmp zookeeper >> >server to upload the conf files to. It then shuts that server down, >> >which unfortunately logs this exception. Then the first Solr instance will >> >run a zookeeper server. Uploading the configs ahead of time allows us to >> >be >> >sure the configs are certainly in zookeeper before the other servers start >> >to come up. If you where doing it by hand, you could just pass the conf >> >dir >> >to the first Solr you started to upload the confs - then wait a second and >> >start the other instances. Its done this other way in the script instead >> >to >> >eliminate any races. >> > >> >On Wed, Dec 21, 2011 at 2:35 PM, Dipti Srivastava < >> >dipti.srivast...@apollogrp.edu> wrote: >> > >> >> Ok, so the issue was that I had only copied the cloud_dev, example and >> >> dist directories and that¹s why some of the libraries were missing. I >> >> copied the build, lib as well and got around the issue. Now, I am >> >>getting >> >> this error when I run the script to start 6 nodes cluster. >> >> >> >> INFO: makePath: /configs/conf1/velocity/jquery.autocomplete.js >> >> Dec 21, 2011 7:24:49 PM org.apache.solr.common.cloud.SolrZkClient >> >>makePath >> >> INFO: makePath: /configs/conf1/velocity/query.vm >> >> Dec 21, 2011 7:24:49 PM org.apache.solr.common.cloud.SolrZkClient >> >>makePath >> >> INFO: makePath: /configs/conf1/velocity/hit.vm >> >> Dec 21, 2011 7:24:49 PM org.apache.zookeeper.server.ZooKeeperServerMain >> >> runFromConfig >> >> WARNING: Server interrupted >> >> java.lang.InterruptedException >> >> at java.lang.Object.wait(Native Method) >> >> at java.lang.Thread.join(Thread.java:1186) >> >> at java.lang.Thread.join(Thread.java:1239) >> >> at >> >> >> >>org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactor >> >>y. >> >> java:318) >> >> at >> >> >> >>org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperSe >> >>rv >> >> erMain.java:113) >> >> at >> >>org.apache.solr.cloud.SolrZkServer$1.run(SolrZkServer.java:116) >> >> --CLOUD--[ec2-user@ cloud-dev]$ ps -ef | grep zk >> >> ec2-user 23796 1 21 19:24 pts/0 00:00:05 java -DzkRun >> >>-DnumShards=2 >> >> -DSTOP.PORT=7983 -DSTOP.KEY=key -jar start.jar >> >> ec2-user 23797 1 18 19:24 pts/0 00:00:04 java -Djetty.port=7574 >> >> -DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6574 -DSTOP.KEY=key >> >>-jar >> >> start.jar >> >> ec2-user 23798 1 19 19:24 pts/0 00:00:04 java -Djetty.port=7575 >> >> -DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6575 -DSTOP.KEY=key >> >>-jar >> >> start.jar >> >> ec2-user 23799 1 18 19:24 pts/0 00:00:04 java -Djetty.port=7576 >> >> -DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6576 -DSTOP.KEY=key >> >>-jar >> >> start.jar >> >> ec2-user 23800 1 19 19:24 pts/0 00:00:04 java -Djetty.port=7577 >> >> -DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6577 -DSTOP.KEY=key >> >>-jar >> >> start.jar >> >> ec2-user 23801 1 19 19:24 pts/0 00:00:04 java -Djetty.port=7578 >> >> -DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6578 -DSTOP.KEY=key >> >>-jar >> >> start.jar >> >> ec2-user 23998 22962 0 19:25 pts/0 00:00:00 grep zk >> >> --CLOUD--[ec2-user@ cloud-dev]$ >> >> >> >> >> >> >> >> Thanks! >> >> Dipti >> >> >> >> On 12/21/11 10:18 AM, "Dipti Srivastava" >> >><dipti.srivast...@apollogrp.edu> >> >> wrote: >> >> >> >> >Hi Mark, >> >> >I built the example and dist and ran the solrcloud.sh script. While >> >> >running I get the following error... Is this ok? It appears that some >> >>of >> >> >the instances got started though. >> >> > >> >> >--CLOUD--[ec2-user@ cloud_dev]$ ./solrcloud.sh >> >> >./solrcloud.sh: line 16: ant: command not found >> >> >Exception in thread "main" java.lang.NoClassDefFoundError: >> >> >org/apache/solr/cloud/ZkController >> >> >Caused by: java.lang.ClassNotFoundException: >> >> >org.apache.solr.cloud.ZkController >> >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> >> > at java.security.AccessController.doPrivileged(Native Method) >> >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> >> > at >> >>sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> >> >Could not find the main class: org.apache.solr.cloud.ZkController. >> >> >Program will exit. >> >> >--CLOUD--[ec2-user@ cloud_dev]$ ls >> >> >solrcloud.sh stop.sh >> >> >--CLOUD--[ec2-user@ cloud_dev]$ cd .. >> >> >--CLOUD--[ec2-user@ solrcloud]$ ls >> >> >cloud_dev example example2 example3 example4 example5 example6 >> >> >--CLOUD--[ec2-user@ solrcloud]$ ps -ef | grep solr >> >> >ec2-user 22690 22452 0 18:12 pts/0 00:00:00 grep solr >> >> >--CLOUD--[ec2-user@ solrcloud]$ ps -ef | grep jetty >> >> >ec2-user 22521 1 4 18:11 pts/0 00:00:02 java -Djetty.port=7574 >> >> >-DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6574 -DSTOP.KEY=key >> >>-jar >> >> >start.jar >> >> >ec2-user 22522 1 4 18:11 pts/0 00:00:02 java -Djetty.port=7575 >> >> >-DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6575 -DSTOP.KEY=key >> >>-jar >> >> >start.jar >> >> >ec2-user 22523 1 4 18:11 pts/0 00:00:02 java -Djetty.port=7576 >> >> >-DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6576 -DSTOP.KEY=key >> >>-jar >> >> >start.jar >> >> >ec2-user 22524 1 4 18:11 pts/0 00:00:02 java -Djetty.port=7577 >> >> >-DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6577 -DSTOP.KEY=key >> >>-jar >> >> >start.jar >> >> >ec2-user 22525 1 4 18:11 pts/0 00:00:02 java -Djetty.port=7578 >> >> >-DzkHost=localhost:9983 -DnumShards=2 -DSTOP.PORT=6578 -DSTOP.KEY=key >> >>-jar >> >> >start.jar >> >> >ec2-user 22692 22452 0 18:12 pts/0 00:00:00 grep jetty >> >> > >> >> >Thanks! >> >> > >> >> >Dipti >> >> > >> >> >On 12/20/11 5:32 PM, "Mark Miller" <markrmil...@gmail.com> wrote: >> >> > >> >> >>You might find the solr/cloud-dev/solrcloud.sh script informative. >> >>From a >> >> >>solrcloud branch checkout, you can run it and it will start up a 2 >> >>shard, >> >> >>6 node cluster with zookeeper running on a single node. stop.sh will >> >> >>shutdown the 6 nodes. Once you start up the nodes, you can start >> >>indexing >> >> >>and searching on any of them, or use the CloudSolrServer solrj >> >>client. It >> >> >>simply takes the ZooKeeper address and figures out the servers from >> >>there >> >> >>(you do currently still have to pass distrib=true to make requests hit >> >> >>the whole collection). >> >> >> >> >> >>There will be more help on getting started produced soon. Still some >> >>work >> >> >>to finish up first. >> >> >> >> >> >>- Mark >> >> >> >> >> >> > -- > - Mark > > http://www.lucidimagination.com > > > This message is private and confidential. If you have received it in error, > please notify the sender and remove it from your system. > - Mark Miller lucidimagination.com
