Chris, I wasn't able to reproduce the error with a stock install of the HelioSearch Distribution for Solr (HDS). HDS runs under Tomcat.
Looking at the stack trace this looks like the problem: Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock: java.nio.channels.OverlappingFileLockException at org.apache.lucene.store.Lock.obtain(Lock.java:89) at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710) at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) It looks like the new collection is using the same data dir as an existing collection. This startup param might be the issue: export JAVA_OPTS="$JAVA_OPTS -Dsolr.data.dir=/ebs-data/solr/data at Joel Bernstein Search Engineer at Heliosearch On Wed, May 21, 2014 at 2:08 PM, christopher palm <cpa...@gmail.com> wrote: > Hello, > I am trying to use the collections API to add in a secondary collection > similar to what is described in this article > http://heliosearch.org/solrcloud-assigning-nodes-machines/ > It works fine on a jetty setup where bootstrap config dir is specified. > When I try to run this on a tomcat solr that has the example configs > already in zookeeper from an upconfig it fails > > > http://SERVERADDRESS:8080/solr/admin/collections?action=CREATE&name=realtimecollection&collection.configName=collection1&numShards=1&createNodeSet=SERVERADDRESS:8080_solr > > ": Unable to create core: realtimecollection_shard1_replica1 Caused by: > null" > > Any ideas why this fails? > > > From the logs it seems to be trying to create this in the same folder as > the solr 4.8 war. > > Below are the logs I see and the startup config arguments I am passing into > tomcat/SOLR. > > Thanks, > Chris > > export SOLR_HOME=/ebs-data/bin/solr > export JAVA_OPTS="$JAVA_OPTS -DzkHost=ZOOKEEPERCLUSTERADDRESS/app/test" > export JAVA_OPTS="$JAVA_OPTS -DnumShards=2" > export JAVA_OPTS="$JAVA_OPTS -Dcollection.configName=collection1 " > export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/ebs-data/solr/conf " > export JAVA_OPTS="$JAVA_OPTS -Dsolr.data.dir=/ebs-data/solr/data " > > > – A cluster state change: WatchedEvent state:SyncConnected > type:NodeDataChanged path:/clusterstate.json, has occurred - updating... > (live nodes size: 1) > 176300 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000] > INFO org.apache.solr.cloud.OverseerCollectionProcessor – Creating > SolrCores for new collection, shardNames [shard1] , replicationFactor : 1 > 176300 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000] > INFO org.apache.solr.cloud.OverseerCollectionProcessor – Creating shard > realtimecollection_shard1_replica1 as part of slice shard1 of collection > realtimecollection on SERVERADDRESS:8080_solr > 178443 [http-bio-8080-exec-13] INFO > org.apache.solr.handler.admin.CoreAdminHandler – core create command > > numShards=1&shard=shard1&collection.configName=collection1&name=realtimecollection_shard1_replica1&action=CREATE&collection=realtimecollection&wt=javabin&qt=/admin/cores&version=2 > 178540 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController – > publishing core=realtimecollection_shard1_replica1 state=down > collection=realtimecollection > 178568 [localhost-startStop-1-EventThread] INFO > org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path: > /overseer/queue state: SyncConnected type NodeChildrenChanged > 178569 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController – > waiting to find shard id in clusterstate for > realtimecollection_shard1_replica1 > 178641 [Thread-2] INFO org.apache.solr.common.cloud.ZkStateReader – > Updating cloud state from ZooKeeper... > 178667 [Thread-2] INFO org.apache.solr.cloud.Overseer – Update state > numShards=1 message={ > "operation":"state", > "state":"down", > "base_url":"http://SERVERADDRESS:8080/solr";, > "core":"realtimecollection_shard1_replica1", > "roles":null, > "node_name":"SERVERADDRESS:8080_solr", > "shard":"shard1", > "collection":"realtimecollection", > "numShards":"1", > > > "core_node_name":"SERVERADDRESS:8080_solr_realtimecollection_shard1_replica1"} > 178771 [localhost-startStop-1-EventThread] INFO > org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path: > /overseer/queue state: SyncConnected type NodeChildrenChanged > 178798 [localhost-startStop-1-EventThread] INFO > org.apache.solr.common.cloud.ZkStateReader – A cluster state change: > WatchedEvent state:SyncConnected type:NodeDataChanged > path:/clusterstate.json, has occurred - updating... (live nodes size: 1) > 179572 [http-bio-8080-exec-13] INFO > org.apache.solr.core.SolrXMLCoresLocator – Persisted core descriptions to > /etc/solr/solr.xml > 179573 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController – > Check for collection zkNode:realtimecollection > 179600 [http-bio-8080-exec-13] INFO org.apache.solr.cloud.ZkController – > Collection zkNode exists > 179600 [http-bio-8080-exec-13] INFO > org.apache.solr.common.cloud.ZkStateReader – Load collection config > from:/collections/realtimecollection > 179654 [http-bio-8080-exec-13] INFO > org.apache.solr.common.cloud.ZkStateReader – > path=/collections/realtimecollection configName=collection1 specified > config exists in ZooKeeper > 179655 [http-bio-8080-exec-13] INFO > org.apache.solr.core.SolrResourceLoader – new SolrResourceLoader for > directory: '/etc/solr/realtimecollection_shard1_replica1/' > 179900 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrConfig – > Adding specified lib dirs to ClassLoader > 179901 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../contrib/extraction/lib (resolved as: > > /etc/solr/realtimecollection_shard1_replica1/../../../contrib/extraction/lib). > 179902 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../dist/ (resolved as: > /etc/solr/realtimecollection_shard1_replica1/../../../dist). > 179902 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../contrib/clustering/lib/ (resolved as: > > /etc/solr/realtimecollection_shard1_replica1/../../../contrib/clustering/lib). > 179903 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../dist/ (resolved as: > /etc/solr/realtimecollection_shard1_replica1/../../../dist). > 179903 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../contrib/langid/lib/ (resolved as: > /etc/solr/realtimecollection_shard1_replica1/../../../contrib/langid/lib). > 179904 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../dist/ (resolved as: > /etc/solr/realtimecollection_shard1_replica1/../../../dist). > 179904 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../contrib/velocity/lib (resolved as: > > /etc/solr/realtimecollection_shard1_replica1/../../../contrib/velocity/lib). > 179904 [http-bio-8080-exec-13] WARN > org.apache.solr.core.SolrResourceLoader – Can't find (or read) directory > to add to classloader: ../../../dist/ (resolved as: > /etc/solr/realtimecollection_shard1_replica1/../../../dist). > 180011 [http-bio-8080-exec-13] INFO org.apache.solr.update.SolrIndexConfig > – IndexWriter infoStream solr logging is enabled > 180038 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrConfig – > Using Lucene MatchVersion: LUCENE_48 > 180185 [http-bio-8080-exec-13] INFO org.apache.solr.core.Config – Loaded > SolrConfig: solrconfig.xml > 180285 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema – > Reading Solr Schema from schema.xml > 180524 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema – > [realtimecollection_shard1_replica1] Schema name=example > 182144 [http-bio-8080-exec-13] INFO org.apache.solr.schema.IndexSchema – > unique key field: id > 182147 [http-bio-8080-exec-13] INFO org.apache.solr.rest.RestManager – > Registered ManagedResource impl > org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path > /schema/analysis/stopwords/english > 182148 [http-bio-8080-exec-13] INFO org.apache.solr.rest.RestManager – > Registered ManagedResource impl > > org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager > for path /schema/analysis/synonyms/english > 183114 [http-bio-8080-exec-13] INFO > org.apache.solr.schema.FileExchangeRateProvider – Reloading exchange > rates from file currency.xml > 183175 [http-bio-8080-exec-13] INFO > org.apache.solr.schema.FileExchangeRateProvider – Reloading exchange > rates from file currency.xml > 185157 [http-bio-8080-exec-13] INFO org.apache.solr.core.CoreContainer – > Creating SolrCore 'realtimecollection_shard1_replica1' using configuration > from collection realtimecollection > 185158 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > solr.NRTCachingDirectoryFactory > 185158 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > [realtimecollection_shard1_replica1] Opening new SolrCore at > /etc/solr/realtimecollection_shard1_replica1/, dataDir=/ebs-data/solr/data/ > 185159 [http-bio-8080-exec-13] INFO org.apache.solr.core.JmxMonitoredMap > – JMX monitoring is enabled. Adding Solr mbeans to JMX Server: > com.sun.jmx.mbeanserver.JmxMBeanServer@1de10458 > 185160 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > [realtimecollection_shard1_replica1] Added SolrEventListener for > newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]} > 185160 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > [realtimecollection_shard1_replica1] Added SolrEventListener for > firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static > firstSearcher warming in solrconfig.xml}]} > 185161 [http-bio-8080-exec-13] INFO > org.apache.solr.core.CachingDirectoryFactory – return new directory for > /ebs-data/solr/data > 185162 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – New > index directory detected: old=null new=/ebs-data/solr/data/index/ > 185163 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > created json: solr.JSONResponseWriter > 185163 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > adding lazy queryResponseWriter: solr.VelocityResponseWriter > 185164 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > created velocity: solr.VelocityResponseWriter > 185164 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > created xslt: solr.XSLTResponseWriter > 185164 [http-bio-8080-exec-13] INFO > org.apache.solr.response.XSLTResponseWriter – xsltCacheLifetimeSeconds=5 > 185170 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – no > updateRequestProcessorChain defined as default, creating implicit default > 185171 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /select: solr.SearchHandler > 185173 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /query: solr.SearchHandler > 185174 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /get: solr.RealTimeGetHandler > 185175 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /browse: solr.SearchHandler > 185177 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /update: solr.UpdateRequestHandler > 185178 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /update/json: solr.UpdateRequestHandler > 185179 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /update/csv: solr.UpdateRequestHandler > 185179 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.extraction.ExtractingRequestHandler > 185180 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /update/extract: solr.extraction.ExtractingRequestHandler > 185180 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.FieldAnalysisRequestHandler > 185181 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /analysis/field: solr.FieldAnalysisRequestHandler > 185181 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.DocumentAnalysisRequestHandler > 185182 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /analysis/document: solr.DocumentAnalysisRequestHandler > 185183 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /admin/: solr.admin.AdminHandlers > 185184 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /admin/ping: solr.PingRequestHandler > 185185 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /debug/dump: solr.DumpRequestHandler > 185186 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /replication: solr.ReplicationHandler > 185187 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.SearchHandler > 185187 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /spell: solr.SearchHandler > 185188 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.SearchHandler > 185188 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /suggest: solr.SearchHandler > 185189 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.SearchHandler > 185213 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /tvrh: solr.SearchHandler > 185214 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.SearchHandler > 185214 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /terms: solr.SearchHandler > 185215 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – adding lazy requestHandler: solr.SearchHandler > 185215 [http-bio-8080-exec-13] INFO org.apache.solr.core.RequestHandlers > – created /elevate: solr.SearchHandler > 185217 [http-bio-8080-exec-13] INFO > org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60 > 185218 [http-bio-8080-exec-13] INFO > org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60 > 185218 [http-bio-8080-exec-13] INFO > org.apache.solr.handler.loader.XMLLoader – xsltCacheLifetimeSeconds=60 > 185223 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – Hard > AutoCommit: if uncommited for 15000ms; > 185224 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – Soft > AutoCommit: disabled > 185225 [http-bio-8080-exec-13] INFO > org.apache.solr.core.CachingDirectoryFactory – return new directory for > /ebs-data/solr/data/index > 186227 [http-bio-8080-exec-13] INFO > org.apache.solr.core.CachingDirectoryFactory – looking to close > /ebs-data/solr/data/index > [CachedDir<<refCount=0;path=/ebs-data/solr/data/index;done=true>>] > 186228 [http-bio-8080-exec-13] INFO > org.apache.solr.core.CachingDirectoryFactory – Closing directory: > /ebs-data/solr/data/index > 186236 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > [realtimecollection_shard1_replica1] CLOSING SolrCore > org.apache.solr.core.SolrCore@810a57c > 186243 [http-bio-8080-exec-13] INFO org.apache.solr.update.UpdateHandler > – closing DirectUpdateHandler2{commits=0,autocommit > maxTime=15000ms,autocommits=0,soft > > autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0} > 186243 [http-bio-8080-exec-13] INFO org.apache.solr.update.SolrCoreState > – Closing SolrCoreState > 186244 [http-bio-8080-exec-13] INFO > org.apache.solr.update.DefaultSolrCoreState – SolrCoreState ref count has > reached 0 - closing IndexWriter > 186244 [http-bio-8080-exec-13] INFO > org.apache.solr.update.DefaultSolrCoreState – closing IndexWriter with > IndexWriterCloser > 186245 [http-bio-8080-exec-13] INFO org.apache.solr.core.SolrCore – > [realtimecollection_shard1_replica1] Closing main searcher on request. > 186246 [http-bio-8080-exec-13] INFO > org.apache.solr.core.CachingDirectoryFactory – Closing > NRTCachingDirectoryFactory - 1 directories currently being tracked > 186247 [http-bio-8080-exec-13] INFO > org.apache.solr.core.CachingDirectoryFactory – looking to close > /ebs-data/solr/data > [CachedDir<<refCount=0;path=/ebs-data/solr/data;done=false>>] > 186248 [http-bio-8080-exec-13] INFO > org.apache.solr.core.CachingDirectoryFactory – Closing directory: > /ebs-data/solr/data > 186252 [http-bio-8080-exec-13] ERROR org.apache.solr.core.CoreContainer – > Unable to create core: realtimecollection_shard1_replica1 > org.apache.solr.common.SolrException: Error opening new searcher > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) > at > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:571) > at > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:200) > at > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186) > at > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) > at > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > at > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) > at > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) > at > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) > at > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) > at > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) > at > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) > at > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550) > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838) > ... 24 more > Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain > timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock: > java.nio.channels.OverlappingFileLockException > at org.apache.lucene.store.Lock.obtain(Lock.java:89) > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710) > at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77) > at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) > at > > org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267) > at > > org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110) > at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1513) > ... 26 more > Caused by: java.nio.channels.OverlappingFileLockException > at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) > at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) > at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1017) > at java.nio.channels.FileChannel.tryLock(FileChannel.java:1154) > at > org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148) > at org.apache.lucene.store.Lock.obtain(Lock.java:100) > ... 32 more > 186334 [localhost-startStop-1-EventThread] INFO > org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path: > /overseer/queue state: SyncConnected type NodeChildrenChanged > 186336 [http-bio-8080-exec-13] ERROR org.apache.solr.core.SolrCore – > org.apache.solr.common.SolrException: Error CREATEing SolrCore > 'realtimecollection_shard1_replica1': Unable to create core: > realtimecollection_shard1_replica1 Caused by: null > at > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:608) > at > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:200) > at > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186) > at > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:726) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:258) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) > at > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > at > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > at > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) > at > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) > at > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) > at > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) > at > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) > at > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) > at > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > Caused by: org.apache.solr.common.SolrException: Unable to create core: > realtimecollection_shard1_replica1 > at > org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568) > at > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:571) > ... 21 more > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) > ... 22 more > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550) > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838) > ... 24 more > Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain > timed out: NativeFSLock@/ebs-data/solr/data/index/write.lock: > java.nio.channels.OverlappingFileLockException > at org.apache.lucene.store.Lock.obtain(Lock.java:89) > at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:710) > at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77) > at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) > at > > org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267) > at > > org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110) > at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1513) > ... 26 more > Caused by: java.nio.channels.OverlappingFileLockException > at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) > at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) > at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1017) > at java.nio.channels.FileChannel.tryLock(FileChannel.java:1154) > at > org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148) > at org.apache.lucene.store.Lock.obtain(Lock.java:100) > ... 32 more > > 186337 [http-bio-8080-exec-13] INFO > org.apache.solr.servlet.SolrDispatchFilter – [admin] webapp=null > path=/admin/cores > > params={numShards=1&shard=shard1&collection.configName=collection1&name=realtimecollection_shard1_replica1&action=CREATE&collection=realtimecollection&wt=javabin&qt=/admin/cores&version=2} > status=400 QTime=7895 > 186487 [Thread-2] INFO org.apache.solr.common.cloud.ZkStateReader – > Updating cloud state from ZooKeeper... > 186554 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000] > ERROR org.apache.solr.cloud.OverseerCollectionProcessor – Error from > shard: http://SERVERADDRESS:8080/solr > org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error > CREATEing SolrCore 'realtimecollection_shard1_replica1': Unable to create > core: realtimecollection_shard1_replica1 Caused by: null > at > > org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554) > at > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210) > at > > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206) > at > > org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:157) > at > > org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > 186555 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000] > INFO org.apache.solr.cloud.OverseerCollectionProcessor – Finished create > command on all shards for collection: realtimecollection > 186698 [localhost-startStop-1-EventThread] INFO > org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path: > /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type > NodeDataChanged > 186787 [localhost-startStop-1-EventThread] INFO > org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path: > /overseer/collection-queue-work state: SyncConnected type > NodeChildrenChanged > 186792 [http-bio-8080-exec-9] INFO > org.apache.solr.servlet.SolrDispatchFilter – [admin] webapp=null > path=/admin/collections > > params={numShards=1&collection.configName=collection1&createNodeSet=SERVERADDRESS:8080_solr&name=realtimecollection&action=CREATE} > status=0 QTime=11206 > 186795 [Overseer-91593013188428078-SERVERADDRESS:8080_solr-n_0000000000] > INFO org.apache.solr.cloud.OverseerCollectionProcessor – Overseer > Collection Processor: Message > id:/overseer/collection-queue-work/qn-0000000000 complete, > > response:{failure={null=org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error > CREATEing SolrCore 'realtimecollection_shard1_replica1': Unable to create > core: realtimecollection_shard1_replica1 Caused by: null}} > 186842 [localhost-startStop-1-EventThread] INFO > org.apache.solr.cloud.DistributedQueue – LatchChildWatcher fired on path: > /overseer/queue state: SyncConnected type NodeChildrenChanged > 186908 [localhost-startStop-1-EventThread] INFO > org.apache.solr.common.cloud.ZkStateReader – A cluster state change: > WatchedEvent state:SyncConnected type:NodeDataChanged > path:/clusterstate.json, has occurred - updating... (live nodes size: 1) >
