Never mind. I figured it out. It was due to a NPE on the missing
updateLog in solrconfig.xml. My solrconfig.xml is from an older Solr
release, which doesn't have certain required sections, etc. After adding
them to solrconfig.xml per this official doc, everything started to
work. It'd be great if null checks were there to produce informative
error on SolrCore.java, so as to make it easier to find the root cause.
http://wiki.apache.org/solr/SolrCloud#Required_Config
Regards,
HaiXin
On 09/16/2013 06:44 PM, HaiXin Tie wrote:
Hi Solr experts,
I am using Solr 4.4 with ZK 3.4.5, trying to split "shard1" of a
collection named "body". There is only one core on one machine for
this collection. When I call SPLITSHARD to split this collection, Solr
is able to create two sub-shards, but failed with a NPE in
SolrCore.java while publishing the new shards. It seems that either
the updateHandler or its updateLog is null, though they work fine in
the original shard:
SolrCore.java
if (cc != null && cc.isZooKeeperAware() &&
Slice.CONSTRUCTION.equals(cd.getCloudDescriptor().getShardState())) {
// set update log to buffer before publishing the core
862: getUpdateHandler().getUpdateLog().bufferUpdates();
cd.getCloudDescriptor().setShardState(null);
cd.getCloudDescriptor().setShardRange(null);
}
Here are the details. Any pointers to aid debugging this issue is
greatly appreciated!
# curl request/response to split the shard:
curl -s
"http://localhost:8983/solr/admin/collections?action=SPLITSHARD&collection=body&shard=shard1";
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">500</int><int
name="QTime">2688</int></lst><lst
name="failure"><str>org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
CREATEing SolrCore 'body_shard1_0_replica1': Unable to create core:
body_shard1_0_replica1 Caused by: null</str></lst><str name="Operation
splitshard caused
exception:">org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
SPLTSHARD failed to create subshard leaders</str><lst
name="exception"><str name="msg">SPLTSHARD failed to create subshard
leaders</str><int name="rspCode">500</int></lst><lst name="error"><str
name="msg">SPLTSHARD failed to create subshard leaders</str><str
name="trace">org.apache.solr.common.SolrException: SPLTSHARD failed to
create subshard leaders
at
org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:171)
at
org.apache.solr.handler.admin.CollectionsHandler.handleSplitShardAction(CollectionsHandler.java:322)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:136)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:218)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
</str><int name="code">500</int></lst>
</response>
# Full solr log for the split shard call:
384779 [qtp334936591-10] INFO
org.apache.solr.handler.admin.CollectionsHandler ? Splitting shard :
shard=shard1&action=SPLITSHARD&collection=body
384791 [main-EventThread] INFO org.apache.solr.cloud.DistributedQueue
? Watcher fired on path: /overseer/collection-queue-work state:
SyncConnected type NodeChildrenChanged
384791 [main-EventThread] INFO org.apache.solr.cloud.DistributedQueue
? Watcher fired on path: /overseer/collection-queue-work state:
SyncConnected type NodeChildrenChanged
384797 [main-EventThread] INFO org.apache.solr.cloud.DistributedQueue
? Watcher fired on path: /overseer/collection-queue-work state:
SyncConnected type NodeChildrenChanged
384798
[Overseer-90358802380161262-10.100.131.76:8983_solr-n_0000000012]
INFO org.apache.solr.cloud.OverseerCollectionProcessor ? Overseer
Collection Processor: Get the message
id:/overseer/collection-queue-work/qn-0000000040 message:{
"operation":"splitshard",
"shard":"shard1",
"collection":"body"}
384799
[Overseer-90358802380161262-10.100.131.76:8983_solr-n_0000000012]
INFO org.apache.solr.cloud.OverseerCollectionProcessor ? Split shard
invoked
384800
[Overseer-90358802380161262-10.100.131.76:8983_solr-n_0000000012]
INFO org.apache.solr.cloud.OverseerCollectionProcessor ? Creating
shard body_shard1_0_replica1 as part of slice shard1_0 of collection
body on 10.100.131.76:8983_solr
384801
[Overseer-90358802380161262-10.100.131.76:8983_solr-n_0000000012]
INFO org.apache.solr.cloud.OverseerCollectionProcessor ? Creating
shard body_shard1_1_replica1 as part of slice shard1_1 of collection
body on 10.100.131.76:8983_solr
384916 [qtp334936591-13] INFO org.apache.solr.cloud.ZkController ?
publishing core=body_shard1_1_replica1 state=down
384916 [qtp334936591-12] INFO org.apache.solr.cloud.ZkController ?
publishing core=body_shard1_0_replica1 state=down
384917 [qtp334936591-12] INFO org.apache.solr.cloud.ZkController ?
numShards not found on descriptor - reading it from system property
384917 [qtp334936591-13] INFO org.apache.solr.cloud.ZkController ?
numShards not found on descriptor - reading it from system property
384926 [qtp334936591-12] INFO org.apache.solr.cloud.ZkController ?
waiting to find shard id in clusterstate for body_shard1_0_replica1
384930 [qtp334936591-13] INFO org.apache.solr.cloud.ZkController ?
waiting to find shard id in clusterstate for body_shard1_1_replica1
386369 [Thread-13] INFO org.apache.solr.common.cloud.ZkStateReader ?
Updating cloud state from ZooKeeper...
386372 [Thread-13] INFO org.apache.solr.cloud.Overseer ? Update
state numShards=null message={
"operation":"state",
"core_node_name":"10.100.131.76:8983_solr_body_shard1_0_replica1",
"numShards":null,
"shard":"shard1_0",
"roles":null,
"state":"down",
"shard_state":"construction",
"core":"body_shard1_0_replica1",
"collection":"body",
"shard_range":"80000000-ffffffff",
"node_name":"10.100.131.76:8983_solr",
"base_url":"http://10.100.131.76:8983/solr"}
386392 [Thread-13] INFO org.apache.solr.cloud.Overseer ? Update
state numShards=null message={
"operation":"state",
"core_node_name":"10.100.131.76:8983_solr_body_shard1_1_replica1",
"numShards":null,
"shard":"shard1_1",
"roles":null,
"state":"down",
"shard_state":"construction",
"core":"body_shard1_1_replica1",
"collection":"body",
"shard_range":"0-7fffffff",
"node_name":"10.100.131.76:8983_solr",
"base_url":"http://10.100.131.76:8983/solr"}
386481 [main-EventThread] INFO
org.apache.solr.common.cloud.ZkStateReader ? A cluster state change:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
386930 [qtp334936591-12] INFO org.apache.solr.core.CoreContainer ?
Creating SolrCore 'body_shard1_0_replica1' using instanceDir:
mind/body_shard1_0_replica1
386931 [qtp334936591-12] INFO org.apache.solr.cloud.ZkController ?
Check for collection zkNode:body
386932 [qtp334936591-12] INFO org.apache.solr.cloud.ZkController ?
Collection zkNode exists
386932 [qtp334936591-12] INFO org.apache.solr.cloud.ZkController ?
Load collection config from:/collections/body
386934 [qtp334936591-13] INFO org.apache.solr.core.CoreContainer ?
Creating SolrCore 'body_shard1_1_replica1' using instanceDir:
mind/body_shard1_1_replica1
386934 [qtp334936591-13] INFO org.apache.solr.cloud.ZkController ?
Check for collection zkNode:body
386935 [qtp334936591-12] INFO org.apache.solr.core.SolrResourceLoader
? new SolrResourceLoader for directory: 'mind/body_shard1_0_replica1/'
386936 [qtp334936591-13] INFO org.apache.solr.cloud.ZkController ?
Collection zkNode exists
386936 [qtp334936591-13] INFO org.apache.solr.cloud.ZkController ?
Load collection config from:/collections/body
386939 [qtp334936591-13] INFO org.apache.solr.core.SolrResourceLoader
? new SolrResourceLoader for directory: 'mind/body_shard1_1_replica1/'
386971 [qtp334936591-12] INFO org.apache.solr.core.SolrConfig ?
Adding specified lib dirs to ClassLoader
386971 [qtp334936591-12] WARN org.apache.solr.core.SolrResourceLoader
? Can't find (or read) directory to add to classloader: ../lib/
(resolved as:
/TivoData/solr-4.4.0/example/mind/body_shard1_0_replica1/../lib).
386974 [qtp334936591-13] INFO org.apache.solr.core.SolrConfig ?
Adding specified lib dirs to ClassLoader
386974 [qtp334936591-13] WARN org.apache.solr.core.SolrResourceLoader
? Can't find (or read) directory to add to classloader: ../lib/
(resolved as:
/TivoData/solr-4.4.0/example/mind/body_shard1_1_replica1/../lib).
387016 [qtp334936591-12] INFO org.apache.solr.core.SolrConfig ? Using
Lucene MatchVersion: LUCENE_44
387023 [qtp334936591-13] INFO org.apache.solr.core.SolrConfig ? Using
Lucene MatchVersion: LUCENE_44
387072 [qtp334936591-12] INFO org.apache.solr.core.Config ? Loaded
SolrConfig: solrconfig.xml
387075 [qtp334936591-12] INFO org.apache.solr.schema.IndexSchema ?
Reading Solr Schema from schema.xml
387078 [qtp334936591-13] INFO org.apache.solr.core.Config ? Loaded
SolrConfig: solrconfig.xml
387080 [qtp334936591-13] INFO org.apache.solr.schema.IndexSchema ?
Reading Solr Schema from schema.xml
387095 [qtp334936591-12] INFO org.apache.solr.schema.IndexSchema ?
[body_shard1_0_replica1] Schema name=body
387099 [qtp334936591-13] INFO org.apache.solr.schema.IndexSchema ?
[body_shard1_1_replica1] Schema name=body
387195 [qtp334936591-12] INFO org.apache.solr.schema.IndexSchema ?
default search field in schema is bodyId
387196 [qtp334936591-13] INFO org.apache.solr.schema.IndexSchema ?
default search field in schema is bodyId
387197 [qtp334936591-12] INFO org.apache.solr.schema.IndexSchema ?
query parser default operator is OR
387197 [qtp334936591-13] INFO org.apache.solr.schema.IndexSchema ?
query parser default operator is OR
387198 [qtp334936591-13] INFO org.apache.solr.schema.IndexSchema ?
unique key field: bodyId
387198 [qtp334936591-12] INFO org.apache.solr.schema.IndexSchema ?
unique key field: bodyId
387218 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ?
solr.NRTCachingDirectoryFactory
387219 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ?
[body_shard1_1_replica1] Opening new SolrCore at
mind/body_shard1_1_replica1/,
dataDir=/TivoData/solr-4.4.0/example/mind/body_shard1_1_replica1/data/
387219 [qtp334936591-13] INFO org.apache.solr.core.JmxMonitoredMap ?
No JMX servers found, not exposing Solr information with JMX.
387219 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ?
[body_shard1_1_replica1] Added SolrEventListener for newSearcher:
org.apache.solr.core.QuerySenderListener{queries=[{q=solr,start=0,rows=10},
{q=rocks,start=0,rows=10}, {q=static newSearcher warming query from
solrconfig.xml}]}
387219 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ?
[body_shard1_1_replica1] Added SolrEventListener for firstSearcher:
org.apache.solr.core.QuerySenderListener{queries=[]}
387220 [qtp334936591-13] INFO
org.apache.solr.core.CachingDirectoryFactory ? return new directory
for /TivoData/solr-4.4.0/example/mind/body_shard1_1_replica1/data
387220 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ? New
index directory detected: old=null
new=/TivoData/solr-4.4.0/example/mind/body_shard1_1_replica1/data/index/
387220 [qtp334936591-13] WARN org.apache.solr.core.SolrCore ?
[body_shard1_1_replica1] Solr index directory
'/TivoData/solr-4.4.0/example/mind/body_shard1_1_replica1/data/index'
doesn't exist. Creating new index...
387221 [qtp334936591-13] INFO
org.apache.solr.core.CachingDirectoryFactory ? return new directory
for /TivoData/solr-4.4.0/example/mind/body_shard1_1_replica1/data/index
387225 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ?
solr.NRTCachingDirectoryFactory
387226 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ?
[body_shard1_0_replica1] Opening new SolrCore at
mind/body_shard1_0_replica1/,
dataDir=/TivoData/solr-4.4.0/example/mind/body_shard1_0_replica1/data/
387227 [qtp334936591-12] INFO org.apache.solr.core.JmxMonitoredMap ?
No JMX servers found, not exposing Solr information with JMX.
387227 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ?
[body_shard1_0_replica1] Added SolrEventListener for newSearcher:
org.apache.solr.core.QuerySenderListener{queries=[{q=solr,start=0,rows=10},
{q=rocks,start=0,rows=10}, {q=static newSearcher warming query from
solrconfig.xml}]}
387228 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ?
[body_shard1_0_replica1] Added SolrEventListener for firstSearcher:
org.apache.solr.core.QuerySenderListener{queries=[]}
387229 [qtp334936591-12] INFO
org.apache.solr.core.CachingDirectoryFactory ? return new directory
for /TivoData/solr-4.4.0/example/mind/body_shard1_0_replica1/data
387229 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ? New
index directory detected: old=null
new=/TivoData/solr-4.4.0/example/mind/body_shard1_0_replica1/data/index/
387230 [qtp334936591-12] WARN org.apache.solr.core.SolrCore ?
[body_shard1_0_replica1] Solr index directory
'/TivoData/solr-4.4.0/example/mind/body_shard1_0_replica1/data/index'
doesn't exist. Creating new index...
387230 [qtp334936591-12] INFO
org.apache.solr.core.CachingDirectoryFactory ? return new directory
for /TivoData/solr-4.4.0/example/mind/body_shard1_0_replica1/data/index
387346 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ?
SolrDeletionPolicy.onCommit: commits: num=1
commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/TivoData/solr-4.4.0/example/mind/body_shard1_1_replica1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@3bea817f;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
387346 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ?
SolrDeletionPolicy.onCommit: commits: num=1
commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/TivoData/solr-4.4.0/example/mind/body_shard1_0_replica1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@20dbd794;
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
387346 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ? newest
commit generation = 1
387346 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ? newest
commit generation = 1
387348 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ? created
xslt: solr.XSLTResponseWriter
387348 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ? created
xslt: solr.XSLTResponseWriter
387348 [qtp334936591-12] INFO
org.apache.solr.response.XSLTResponseWriter ? xsltCacheLifetimeSeconds=5
387348 [qtp334936591-13] INFO
org.apache.solr.response.XSLTResponseWriter ? xsltCacheLifetimeSeconds=5
387350 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created standard: solr.StandardRequestHandler
387350 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
adding lazy requestHandler: solr.SpellCheckerRequestHandler
387350 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created standard: solr.StandardRequestHandler
387350 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created spellchecker: solr.SpellCheckerRequestHandler
387351 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
adding lazy requestHandler: solr.SpellCheckerRequestHandler
387351 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /mlt: solr.MoreLikeThisHandler
387351 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created spellchecker: solr.SpellCheckerRequestHandler
387352 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /dataimport: org.apache.solr.handler.dataimport.DataImportHandler
387352 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /mlt: solr.MoreLikeThisHandler
387352 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /search: org.apache.solr.handler.component.SearchHandler
387353 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
adding lazy requestHandler:
org.apache.solr.handler.component.SearchHandler
387353 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /dataimport: org.apache.solr.handler.dataimport.DataImportHandler
387353 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /elevate: org.apache.solr.handler.component.SearchHandler
387354 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /search: org.apache.solr.handler.component.SearchHandler
387354 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
adding lazy requestHandler:
org.apache.solr.handler.component.SearchHandler
387355 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /elevate: org.apache.solr.handler.component.SearchHandler
387354 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /update: solr.UpdateRequestHandler
387355 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /update: solr.UpdateRequestHandler
387355 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /admin/: org.apache.solr.handler.admin.AdminHandlers
387356 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /admin/: org.apache.solr.handler.admin.AdminHandlers
387356 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /admin/ping: solr.PingRequestHandler
387356 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /admin/ping: solr.PingRequestHandler
387357 [qtp334936591-12] INFO org.apache.solr.core.RequestHandlers ?
created /debug/dump: solr.DumpRequestHandler
387357 [qtp334936591-13] INFO org.apache.solr.core.RequestHandlers ?
created /debug/dump: solr.DumpRequestHandler
387358 [qtp334936591-12] INFO
org.apache.solr.handler.loader.XMLLoader ? xsltCacheLifetimeSeconds=60
387358 [qtp334936591-13] INFO
org.apache.solr.handler.loader.XMLLoader ? xsltCacheLifetimeSeconds=60
387362 [qtp334936591-12] INFO
org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@6745934e
main
387363 [qtp334936591-13] INFO
org.apache.solr.search.SolrIndexSearcher ? Opening Searcher@23bca486
main
387363 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ? Hard
AutoCommit: disabled
387363 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ? Hard
AutoCommit: disabled
387363 [qtp334936591-12] INFO org.apache.solr.core.SolrCore ? Soft
AutoCommit: disabled
387364 [qtp334936591-13] INFO org.apache.solr.core.SolrCore ? Soft
AutoCommit: disabled
387364 [qtp334936591-12] INFO org.apache.solr.cloud.ZkController ?
Load collection config from:/collections/body
387364 [qtp334936591-13] INFO org.apache.solr.cloud.ZkController ?
Load collection config from:/collections/body
387370 [searcherExecutor-36-thread-1] INFO
org.apache.solr.core.SolrCore ? QuerySenderListener sending requests
to Searcher@6745934e main{StandardDirectoryReader(segments_1:1)}
387371 [searcherExecutor-36-thread-1] INFO
org.apache.solr.core.SolrCore ? QuerySenderListener done.
387371 [searcherExecutor-35-thread-1] INFO
org.apache.solr.core.SolrCore ? QuerySenderListener sending requests
to Searcher@23bca486 main{StandardDirectoryReader(segments_1:1)}
387372 [qtp334936591-13] ERROR org.apache.solr.core.CoreContainer ?
Unable to create core: body_shard1_1_replica1
java.lang.NullPointerException
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:862)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:629)
at
org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:270)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:655)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:523)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:145)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:209)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
387372 [searcherExecutor-35-thread-1] INFO
org.apache.solr.core.SolrCore ? QuerySenderListener done.
387372 [qtp334936591-12] ERROR org.apache.solr.core.CoreContainer ?
Unable to create core: body_shard1_0_replica1
java.lang.NullPointerException
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:862)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:629)
at
org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:270)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:655)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:523)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:145)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:209)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
387371 [searcherExecutor-36-thread-1] INFO
org.apache.solr.core.SolrCore ? [body_shard1_0_replica1] Registered
new searcher Searcher@6745934e
main{StandardDirectoryReader(segments_1:1)}
387375 [searcherExecutor-35-thread-1] INFO
org.apache.solr.core.SolrCore ? [body_shard1_1_replica1] Registered
new searcher Searcher@23bca486
main{StandardDirectoryReader(segments_1:1)}
387389 [qtp334936591-13] ERROR org.apache.solr.core.SolrCore ?
org.apache.solr.common.SolrException: Error CREATEing SolrCore
'body_shard1_1_replica1': Unable to create core:
body_shard1_1_replica1 Caused by: null
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:554)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:145)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:209)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.common.SolrException: Unable to create
core: body_shard1_1_replica1
at
org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1150)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:666)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:523)
... 30 more
Caused by: java.lang.NullPointerException
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:862)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:629)
at
org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:270)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:655)
... 31 more
387389 [qtp334936591-13] INFO
org.apache.solr.servlet.SolrDispatchFilter ? [admin] webapp=null
path=/admin/cores
params={shard.range=0-7fffffff&shard.state=construction&shard=shard1_1&name=body_shard1_1_replica1&action=CREATE&collection=body&wt=javabin&qt=/admin/cores&version=2}
status=400 QTime=2474
387392 [qtp334936591-12] ERROR org.apache.solr.core.SolrCore ?
org.apache.solr.common.SolrException: Error CREATEing SolrCore
'body_shard1_0_replica1': Unable to create core:
body_shard1_0_replica1 Caused by: null
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:554)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:145)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:209)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.common.SolrException: Unable to create
core: body_shard1_0_replica1
at
org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1150)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:666)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:523)
... 30 more
Caused by: java.lang.NullPointerException
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:862)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:629)
at
org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:270)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:655)
... 31 more
387393 [qtp334936591-12] INFO
org.apache.solr.servlet.SolrDispatchFilter ? [admin] webapp=null
path=/admin/cores
params={shard.range=80000000-ffffffff&shard.state=construction&shard=shard1_0&name=body_shard1_0_replica1&action=CREATE&collection=body&wt=javabin&qt=/admin/cores&version=2}
status=400 QTime=2478
387405
[Overseer-90358802380161262-10.100.131.76:8983_solr-n_0000000012]
ERROR org.apache.solr.cloud.OverseerCollectionProcessor ? Error from
shard: 10.100.131.76:8983/solr
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Error CREATEing SolrCore 'body_shard1_0_replica1': Unable to create
core: body_shard1_0_replica1 Caused by: null
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:156)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
387406
[Overseer-90358802380161262-10.100.131.76:8983_solr-n_0000000012]
ERROR org.apache.solr.cloud.OverseerCollectionProcessor ? Collection
splitshard of splitshard failed:org.apache.solr.common.SolrException:
SPLTSHARD failed to create subshard leaders
at
org.apache.solr.cloud.OverseerCollectionProcessor.collectShardResponses(OverseerCollectionProcessor.java:647)
at
org.apache.solr.cloud.OverseerCollectionProcessor.splitShard(OverseerCollectionProcessor.java:422)
at
org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:187)
at
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:127)
at java.lang.Thread.run(Thread.java:662)
Caused by:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:
Error CREATEing SolrCore 'body_shard1_0_replica1': Unable to create
core: body_shard1_0_replica1 Caused by: null
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:156)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
... 1 more
387450 [main-EventThread] INFO org.apache.solr.cloud.DistributedQueue
? Watcher fired on path:
/overseer/collection-queue-work/qnr-0000000040 state: SyncConnected
type NodeDataChanged
387465
[Overseer-90358802380161262-10.100.131.76:8983_solr-n_0000000012]
INFO org.apache.solr.cloud.OverseerCollectionProcessor ? Overseer
Collection Processor: Message
id:/overseer/collection-queue-work/qn-0000000040 complete,
response:{failure={null=org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
CREATEing SolrCore 'body_shard1_0_replica1': Unable to create core:
body_shard1_0_replica1 Caused by: null},Operation splitshard caused
exception:=org.apache.solr.common.SolrException: SPLTSHARD failed to
create subshard leaders,exception={msg=SPLTSHARD failed to create
subshard leaders,rspCode=500}}
387466 [qtp334936591-10] INFO
org.apache.solr.servlet.SolrDispatchFilter ? [admin] webapp=null
path=/admin/collections
params={shard=shard1&action=SPLITSHARD&collection=body} status=500
QTime=2688
387467 [qtp334936591-10] ERROR
org.apache.solr.servlet.SolrDispatchFilter ?
null:org.apache.solr.common.SolrException: SPLTSHARD failed to create
subshard leaders
at
org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:171)
at
org.apache.solr.handler.admin.CollectionsHandler.handleSplitShardAction(CollectionsHandler.java:322)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:136)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:218)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:662)
388009 [Thread-13] INFO org.apache.solr.common.cloud.ZkStateReader ?
Updating cloud state from ZooKeeper...
388066 [main-EventThread] INFO
org.apache.solr.common.cloud.ZkStateReader ? A cluster state change:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
738810 [Finalizer] ERROR org.apache.solr.core.SolrCore ? REFCOUNT
ERROR: unreferenced org.apache.solr.core.SolrCore@529801f4
(body_shard1_0_replica1) has a reference count of 1
738810 [Finalizer] ERROR org.apache.solr.util.ConcurrentLRUCache ?
ConcurrentLRUCache was not destroyed prior to finalize(), indicates a
bug -- POSSIBLE RESOURCE LEAK!!!
738835 [Finalizer] ERROR org.apache.solr.core.SolrCore ? REFCOUNT
ERROR: unreferenced org.apache.solr.core.SolrCore@25927275
(body_shard1_1_replica1) has a reference count of 1
738836 [Finalizer] ERROR org.apache.solr.util.ConcurrentLRUCache ?
ConcurrentLRUCache was not destroyed prior to finalize(), indicates a
bug -- POSSIBLE RESOURCE LEAK!!!
941518 [qtp334936591-14] ERROR org.apache.solr.core.SolrCore ?
org.apache.solr.common.SolrException: no servers hosting shard:
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:149)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
________________________________
This email and any attachments may contain confidential and privileged material
for the sole use of the intended recipient. Any review, copying, or
distribution of this email (or any attachments) by others is prohibited. If you
are not the intended recipient, please contact the sender immediately and
permanently delete this email and any attachments. No employee or agent of TiVo
Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by
email. Binding agreements with TiVo Inc. may only be made by a signed written
agreement.
