Hi,
trying to experiment with overcharging on our Solr 4.7.2 cluster and called
SPLITSHARD command which after ~30 minutes of work failed with
curl
"http://solrhost:1234/solr/admin/collections?collection=acme&shard=shard1&action=SPLITSHARD";
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">500</int><int
name="QTime">836129</int></lst><lst name="success"><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">1499</int></lst><str
name="core">elmar_v3_shard1_1_replica1</str></lst><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">1507</int></lst><str
name="core">elmar_v3_shard1_0_replica1</str></lst><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">1002</int></lst></lst><lst><lst name="responseHeader"><int
name="status">0</int><int name="QTime">1001</int></lst></lst><lst><lst
name="responseHeader"><int name="status">0</int><int
name="QTime">713458</int></lst></lst><lst><lst name="responseHeader"><int
name="status">0</int><int name="QTime">1</int></lst><str
name="core">elmar_v3_shard1_0_replica1</str><str
name="status">EMPTY_BUFFER</str></lst><lst><lst name="responseHeader"><int
name="status">0</int><int name="QTime">1</int></lst><str
name="core">elmar_v3_shard1_1_replica1</str><str
name="status">EMPTY_BUFFER</str></lst></lst><str name="Operation splitshard
caused
exception:">java.util.EmptyStackException:java.util.EmptyStackException</str><lst
name="exception"><null name="msg"/><int name="rspCode">-1</int></lst><lst
name="error"><str name="trace">org.apache.solr.common.SolrException
at
org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:248)
at
org.apache.solr.handler.admin.CollectionsHandler.handleSplitShardAction(CollectionsHandler.java:484)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:165)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:720)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:265)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:205)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
</str><int name="code">500</int></lst>
</response>
It also seems that for 30 minutes it was very slow and not even half way done:
acme@solrhost:/var/lib/solr/cores$ du -h acme_shard1_replica1/
271M acme_shard1_replica1/data/tlog
5.4G acme_shard1_replica1/data/index
5.7G acme_shard1_replica1/data
5.7G acme_shard1_replica1/
acme@solrhost:/var/lib/solr/cores$ du -h acme_shard1_1_replica1/
4.0K acme_shard1_1_replica1/data/tlog
1004M acme_shard1_1_replica1/data/index
1004M acme_shard1_1_replica1/data
1004M acme_shard1_1_replica1/
acme@solrhost:/var/lib/solr/cores$ du -h acme_shard1_0_replica1/
4.0K acme_shard1_0_replica1/data/tlog
898M acme_shard1_0_replica1/data/index
898M acme_shard1_0_replica1/data
898M acme_shard1_0_replica1/
What is the recommended cleanup strategy?
Also is there a way for me run the command successfully on my end without
upgrading?
Cheers,
Oliver
