Some more information that may help developers find out the cause. INFO 2014-02-04 14:47:08,931 DistributedQueue.java (line 211) Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged ... (there were 393 of these "Watcher fired on path" lines in total) ...
Thanks, Ray INFO 2014-02-04 14:47:08,923 CollectionsHandler.java (line 295) Creating Collection : numShards=1&name=abc.edu.xyz&replicationFactor=2&action=CREATE INFO 2014-02-04 14:47:08,931 DistributedQueue.java (line 211) Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged INFO 2014-02-04 14:47:08,931 DistributedQueue.java (line 211) Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged ... (there were 393 of these "Watcher fired on path" lines in total) ... INFO 2014-02-04 14:47:08,955 DistributedQueue.java (line 211) Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged INFO 2014-02-04 14:48:08,944 DistributedQueue.java (line 211) Watcher fired on path: /overseer/collection-queue-work/qnr-0000000134 state: SyncConnected type NodeDeleted ERROR 2014-02-04 14:48:08,945 SolrException.java (line 108) org.apache.solr.common.SolrException: createcollection the collection time out:60s at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:204) at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:185) at org.apache.solr.handler.admin.CollectionsHandler.handleCreateAction(CollectionsHandler.java:319) On Tuesday, February 4, 2014 7:26 PM, Ray Cheng <rch...@rocketmail.com> wrote: Hi, > >Our SolrCloud with Solr 4.6.0 has two replicas and uses three external >ZooKeeper servers. We have created 20 collections successfully in the last two >weeks. Then, creating new collections started failing since yesterday. New >collections are created with curl like this one: > >curl >'http://localhost:8983/solr/admin/collections?action=CREATE&name=abcd.xyz&numShards=1&replicationFactor=2&collection.configName=oursolrconfig' > >telnet to port 8983 from one to the other Solr host was okay. >The largest ZooKeeper node is about 17K bytes. >curl http://localhost:8983/solr/admin/ping printed OK on either SolrCloud host. > >The load on these two SolrCloud host is very low. So, these two systems are >not busy at all. >After Updated/inserted a document in one SolrCloud server, the document >appeared on the other SolrCloud server correctly. >Thus far only creating new collections failed. (I haven't tried to delete >collections.) > >The create collection printed the following lines on the shell that I ran the >above curl command. Solr on the other Linux didn't print anything during this >time frame. If I run the same curl on the second SolrCloud host, it printed >the same error message and the first SolrCloud host didn't print anything. > >Any idea on the cause of this problem or the fix? Thanks in advance. > >$ curl >'http://localhost:8983/solr/admin/collections?action=CREATE&name=abcd.xyz&numShards=1&replicationFactor=2&collection.configName=oursolrconfig' ><?xml version="1.0" encoding="UTF-8"?> ><response> ><lst name="responseHeader"><int name="status">500</int><int >name="QTime">60016</int></lst><lst name="error"><str >name="msg">createcollection the collection time out:60s</str><str >name="trace">org.apache.solr.common.SolrException: createcollection the >collection time out:60s >at >org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:204) >at >org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:185) >at >org.apache.solr.handler.admin.CollectionsHandler.handleCreateAction(CollectionsHandler.java:319) >at >org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:134) >at >org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) >at >org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:662) >at >org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) >at >org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) >at >org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) >at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) >at >org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) >at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) >at >org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) >at >org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) >at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) >at >org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) >at >org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) >at >org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) >at >org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) >at >org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) >at >org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) >at org.eclipse.jetty.server.Server.handle(Server.java:368) >at >org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) >at >org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) >at >org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) >at >org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) >at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) >at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >at >org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) >at >org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) >at >org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) >at >org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) >at java.lang.Thread.run(Thread.java:744) ></str><int name="code">500</int></lst> ></response> > > >Ray > >
