Re: Update solr schema.xml in real time for Solr 4.10.1
Hi Zheng, As you said **there's no physical schema.xml** but I have. I am using sampletechproductsconfig configuration where I have found schema.xml. In that, I am managing my schema.xml and then I upload that it into zookeeper and reload the collection. On 3/14/15, Zheng Lin Edwin Yeo wrote: > Hi Erick, > > The "real time update of the schema" means we can just do an update using > REST-API curl instead of manually editing the schema.xml and restart the > Solr server. In Solr 5.0, if Solr is loading the schema from the resource > named in 'managedSchemaResourceName', instead of schema.xml, I can just > update it from the REST-API curl. > > For earlier version of Solr, the default setting is > ClassicIndexSchemaFactory, which is read from schema.xml. So besides > getting Solr to load the schema from the resource named in > 'managedSchemaResourceName', rather than from schema.xml, is there other > settings required? > > Zheng Lin > > On 12 March 2015 at 23:26, Erick Erickson wrote: > >> Actually I ran across a neat IntelliJ plugin that you could install >> and directly edit ZK files. And I'm pretty sure there are stand-alone >> programs that do this, but they are all outside Solr. >> >> I'm not sure what "real time update of the schema" is for, would you >> (Zheng) explain further? Collections _must_ be reloaded for schema >> changes to take effect so I'm not quite sure what you're referring to. >> >> Nitin: >> The usual process is to have the master config be local, change the >> local version then upload it to ZK with the upconfig option in zkCli, >> then reload your collection. >> >> Best, >> Erick >> >> On Thu, Mar 12, 2015 at 6:04 AM, Shawn Heisey >> wrote: >> > On 3/12/2015 2:00 AM, Zheng Lin Edwin Yeo wrote: >> >> I understand that in Solr 5.0, they provide a REST API to do real-time >> >> update of the schema using Curl. However, I could not do that for my >> >> eariler version of Solr 4.10.1. >> >> >> >> Would like to check, is this function available for the earlier >> >> version >> of >> >> Solr, and is the curl syntax the same as Solr 5.0? >> > >> > Providing a way to simply edit the config files directly is a potential >> > security issue. We briefly had a way to edit those configs right in >> > the >> > admin UI, but Redhat reported this capability as a security problem, so >> > we removed it. I don't remember whether there is a way to re-enable >> > this functionality. >> > >> > The Schema REST API is available in 4.10. It was also present in 4.9. >> > Currently you can only *add* to the schema, you cannot edit what's >> > already there. >> > >> > Thanks, >> > Shawn >> > >> >
Re: Update solr schema.xml in real time for Solr 4.10.1
Hi Nitin, What I experienced is when I create a new collection, there's no physical schema in that collection. But there is schema.xml in some of the example folder. You can create your own schema.xml in your own collection, but in order to use it, you have to change the schemaFactory class to ClassicIndexSchemaFactory in solrconfig.xml. As by default, the schemaFactory class is set to ManagedIndexSchemaFactory in Solr 5.0. Zheng Lin On 14 March 2015 at 15:22, Nitin Solanki wrote: > Hi Zheng, > As you said **there's no physical schema.xml** but I > have. I am using sampletechproductsconfig configuration where I have > found schema.xml. In that, I am managing my schema.xml and then I > upload that it into zookeeper and reload the collection. > > > > On 3/14/15, Zheng Lin Edwin Yeo wrote: > > Hi Erick, > > > > The "real time update of the schema" means we can just do an update using > > REST-API curl instead of manually editing the schema.xml and restart the > > Solr server. In Solr 5.0, if Solr is loading the schema from the resource > > named in 'managedSchemaResourceName', instead of schema.xml, I can just > > update it from the REST-API curl. > > > > For earlier version of Solr, the default setting is > > ClassicIndexSchemaFactory, which is read from schema.xml. So besides > > getting Solr to load the schema from the resource named in > > 'managedSchemaResourceName', rather than from schema.xml, is there other > > settings required? > > > > Zheng Lin > > > > On 12 March 2015 at 23:26, Erick Erickson > wrote: > > > >> Actually I ran across a neat IntelliJ plugin that you could install > >> and directly edit ZK files. And I'm pretty sure there are stand-alone > >> programs that do this, but they are all outside Solr. > >> > >> I'm not sure what "real time update of the schema" is for, would you > >> (Zheng) explain further? Collections _must_ be reloaded for schema > >> changes to take effect so I'm not quite sure what you're referring to. > >> > >> Nitin: > >> The usual process is to have the master config be local, change the > >> local version then upload it to ZK with the upconfig option in zkCli, > >> then reload your collection. > >> > >> Best, > >> Erick > >> > >> On Thu, Mar 12, 2015 at 6:04 AM, Shawn Heisey > >> wrote: > >> > On 3/12/2015 2:00 AM, Zheng Lin Edwin Yeo wrote: > >> >> I understand that in Solr 5.0, they provide a REST API to do > real-time > >> >> update of the schema using Curl. However, I could not do that for my > >> >> eariler version of Solr 4.10.1. > >> >> > >> >> Would like to check, is this function available for the earlier > >> >> version > >> of > >> >> Solr, and is the curl syntax the same as Solr 5.0? > >> > > >> > Providing a way to simply edit the config files directly is a > potential > >> > security issue. We briefly had a way to edit those configs right in > >> > the > >> > admin UI, but Redhat reported this capability as a security problem, > so > >> > we removed it. I don't remember whether there is a way to re-enable > >> > this functionality. > >> > > >> > The Schema REST API is available in 4.10. It was also present in 4.9. > >> > Currently you can only *add* to the schema, you cannot edit what's > >> > already there. > >> > > >> > Thanks, > >> > Shawn > >> > > >> > > >
Re: Update solr schema.xml in real time for Solr 4.10.1
Ok.. Got Zheng... Thanks a Lot.. On Sat, Mar 14, 2015 at 1:02 PM, Zheng Lin Edwin Yeo wrote: > Hi Nitin, > > What I experienced is when I create a new collection, there's no physical > schema in that collection. But there is schema.xml in some of the example > folder. You can create your own schema.xml in your own collection, but in > order to use it, you have to change the schemaFactory class > to ClassicIndexSchemaFactory in solrconfig.xml. As by default, the > schemaFactory class is set to ManagedIndexSchemaFactory in Solr 5.0. > > Zheng Lin > > > On 14 March 2015 at 15:22, Nitin Solanki wrote: > > > Hi Zheng, > > As you said **there's no physical schema.xml** but I > > have. I am using sampletechproductsconfig configuration where I have > > found schema.xml. In that, I am managing my schema.xml and then I > > upload that it into zookeeper and reload the collection. > > > > > > > > On 3/14/15, Zheng Lin Edwin Yeo wrote: > > > Hi Erick, > > > > > > The "real time update of the schema" means we can just do an update > using > > > REST-API curl instead of manually editing the schema.xml and restart > the > > > Solr server. In Solr 5.0, if Solr is loading the schema from the > resource > > > named in 'managedSchemaResourceName', instead of schema.xml, I can just > > > update it from the REST-API curl. > > > > > > For earlier version of Solr, the default setting is > > > ClassicIndexSchemaFactory, which is read from schema.xml. So besides > > > getting Solr to load the schema from the resource named in > > > 'managedSchemaResourceName', rather than from schema.xml, is there > other > > > settings required? > > > > > > Zheng Lin > > > > > > On 12 March 2015 at 23:26, Erick Erickson > > wrote: > > > > > >> Actually I ran across a neat IntelliJ plugin that you could install > > >> and directly edit ZK files. And I'm pretty sure there are stand-alone > > >> programs that do this, but they are all outside Solr. > > >> > > >> I'm not sure what "real time update of the schema" is for, would you > > >> (Zheng) explain further? Collections _must_ be reloaded for schema > > >> changes to take effect so I'm not quite sure what you're referring to. > > >> > > >> Nitin: > > >> The usual process is to have the master config be local, change the > > >> local version then upload it to ZK with the upconfig option in zkCli, > > >> then reload your collection. > > >> > > >> Best, > > >> Erick > > >> > > >> On Thu, Mar 12, 2015 at 6:04 AM, Shawn Heisey > > >> wrote: > > >> > On 3/12/2015 2:00 AM, Zheng Lin Edwin Yeo wrote: > > >> >> I understand that in Solr 5.0, they provide a REST API to do > > real-time > > >> >> update of the schema using Curl. However, I could not do that for > my > > >> >> eariler version of Solr 4.10.1. > > >> >> > > >> >> Would like to check, is this function available for the earlier > > >> >> version > > >> of > > >> >> Solr, and is the curl syntax the same as Solr 5.0? > > >> > > > >> > Providing a way to simply edit the config files directly is a > > potential > > >> > security issue. We briefly had a way to edit those configs right in > > >> > the > > >> > admin UI, but Redhat reported this capability as a security problem, > > so > > >> > we removed it. I don't remember whether there is a way to re-enable > > >> > this functionality. > > >> > > > >> > The Schema REST API is available in 4.10. It was also present in > 4.9. > > >> > Currently you can only *add* to the schema, you cannot edit what's > > >> > already there. > > >> > > > >> > Thanks, > > >> > Shawn > > >> > > > >> > > > > > >
Solr How to sorting suggestions by sales
We are trying to use solr on our website as a search engine , but we have a problem , we can not sort the suggestions by the number of sales. I tried the components facet, terms, FreeTextLookupFactory and the spellcheck component, but in none of the above components are able to get the results that I want. The most important thing that I would understand its if we can sort the suggestion for a weight chosen by us. The version of solr we are using is the 5.0. schema.xml: solrconfig.xml FreeTextLookupFactory: suggest_product_free FreeTextLookupFactory DocumentDictionaryFactory name_complete suggest_product_free n_sales true text_shingle 3 explicit json true true 10 suggest spellcheck: text_shingle autocomplete org.apache.solr.spelling.suggest.Suggester org.apache.solr.spelling.suggest.fst.WFSTLookupFactory name_complete true 0.005 ./suggester_autocomplete true 10 true autocomplete spellcheck any ideas how I can make to order the suggestion for the weight? Thanks to all and have a nice day. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-How-to-sorting-suggestions-by-sales-tp4193009.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Update solr schema.xml in real time for Solr 4.10.1
On 3/14/2015 1:32 AM, Zheng Lin Edwin Yeo wrote: > What I experienced is when I create a new collection, there's no physical > schema in that collection. But there is schema.xml in some of the example > folder. You can create your own schema.xml in your own collection, but in > order to use it, you have to change the schemaFactory class > to ClassicIndexSchemaFactory in solrconfig.xml. As by default, the > schemaFactory class is set to ManagedIndexSchemaFactory in Solr 5.0. If you are running SolrCloud, then the situation can be slightly different than what you have described. In SolrCloud, the active schema (whether it is schema.xml or managed-schema) is in Zookeeper, not on the disk. If you're NOT using the managed schema with SolrCloud (and probably even if you are), you can upload a new config set to zookeeper at any time using a zookeeper client, then reload the collection. There is a custom commandline zookeeper client included with Solr, handled by a script called zkCli. With that, you can change the config, schema, and anything else that would normally be in the conf directory, even if you do not have direct access to the filesystem. It's not quite correct to say that the default in 5.0 is a managed schema. It is default if you choose to use the data-driven example, but the techproducts example is the one that is the most like the examples in previous versions, and it does not have a managed schema. Thanks, Shawn
Using the collections API to create a new collection
We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud deployed in Tomcat, not something like the Cloudera distribution (I note that because I can't seem to find solrctl and other things referenced in the Cloudera tutorials). I'm trying to create a new Solr collection like this: /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection Then I found this error message in the logs: org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper:newCollection2 at org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) at org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Unknown Source) Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore 'newCollection2_shard1_replica1': Unable to create core: newCollection2_shard1_replica1 at org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:495) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at org.apache.tomcat.util.threads.TaskT
Re: Whole RAM consumed while Indexing.
First, the soft commit interval is very short. Very, very, very, very
short. 300ms is
just short of insane unless it's a typo ;).
Here's a long background:
https://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
But the short form is that you're opening searchers every 300 ms. The
hard commit is better,
but every 3 seconds is still far too short IMO. I'd start with soft
commits of 6 and hard
commits of 6 (60 seconds), meaning that you're going to have to
wait 1 minute for
docs to show up unless you explicitly commit.
You're throwing away all the caches configured in solrconfig.xml more
than 3 times a second,
executing autowarming, etc, etc, etc
Changing these to longer intervals might cure the problem, but if not
then, as Hoss would
say, "details matter". I suspect you're also seeing "overlapping
searchers" warning messages
in your log, and it;s _possible_ that what's happening is that you're
just exceeding the
max warming searchers and never opening a new searcher with the
newly-indexed documents.
But that's a total shot in the dark.
How are you looking for docs (and not finding them)? Does the numDocs number in
the solr admin screen change?
Best,
Erick
On Thu, Mar 12, 2015 at 10:27 PM, Nitin Solanki wrote:
> Hi Alexandre,
>
>
> *Hard Commit* is :
>
>
>${solr.autoCommit.maxTime:3000}
>false
>
>
> *Soft Commit* is :
>
>
> ${solr.autoSoftCommit.maxTime:300}
>
>
> And I am committing 2 documents each time.
> Is it good config for committing?
> Or I am good something wrong ?
>
>
> On Fri, Mar 13, 2015 at 8:52 AM, Alexandre Rafalovitch
> wrote:
>
>> What's your commit strategy? Explicit commits? Soft commits/hard
>> commits (in solrconfig.xml)?
>>
>> Regards,
>>Alex.
>>
>> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
>> http://www.solr-start.com/
>>
>>
>> On 12 March 2015 at 23:19, Nitin Solanki wrote:
>> > Hello,
>> > I have written a python script to do 2 documents indexing
>> > each time on Solr. I have 28 GB RAM with 8 CPU.
>> > When I started indexing, at that time 15 GB RAM was freed. While
>> indexing,
>> > all RAM is consumed but **not** a single document is indexed. Why so?
>> > And it through *HTTPError: HTTP Error 503: Service Unavailable* in python
>> > script.
>> > I think it is due to heavy load on Zookeeper by which all nodes went
>> down.
>> > I am not sure about that. Any help please..
>> > Or anything else is happening..
>> > And how to overcome this issue.
>> > Please assist me towards right path.
>> > Thanks..
>> >
>> > Warm Regards,
>> > Nitin Solanki
>>
Re: how to store _text field
Right, your schema.xml file will define, perhaps, some "dynamic fields". First insure that stored="true" is specified. If you change this, you have to re-index the docs. Second, insure that your "fl" parameter with the field is specified on the requests, something like q=*:*&fl=eoe_txt. Third, insure that you are actually sending content to that field when you index docs. If none of this helps, show us the definition from schema.xml and a sample input document and a query that illustrate the problem please. Best, Erick On Fri, Mar 13, 2015 at 1:20 AM, Mirko Torrisi wrote: > Hi Alexandre, > > I need to visualize the content of _txt. For some reasons, actual it is not > showed in the results (the "response"). > I guess that it doesn't happen because it isn't stored (for some default > setting that I'd like to change). > > Thanks for your help, > > Mirko > > > On 13/03/15 00:27, Alexandre Rafalovitch wrote: >> >> Wait, step back. This is confusing. What's your real problem you are >> trying to solve? >> >> Regards, >> Alex. >> >> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: >> http://www.solr-start.com/ >> >> >> On 12 March 2015 at 19:50, Mirko Torrisi >> wrote: >>> >>> Hi folks, >>> >>> I googled and tried without success so I ask you: how can I modify the >>> setting of a field to store it ? >>> >>> It is interesting to note that I did not add _text field so I guess it is >>> a >>> default one. Maybe it is normal that it is not showed on the result but >>> actually this is my real problem. It could be grand also to copy it in a >>> new >>> field but I do not know how to do it with the last Solr (5) and the new >>> kind >>> of schema. I know that I have to use curl but I do not know how to use it >>> to >>> copy a field. >>> >>> Thank you in advance! >>> Cheers, >>> >>> Mirko > >
Re: Using the collections API to create a new collection
I bet you did not push the configuration to Zookeeper before creating the collection. If you look in your admin UI, the Cloud link and the "tree" version, you'll find a "configsets" directory that'll show you what you _have_ put in ZK, and I'll bet you find nothing like a config set (containing schema.xml etc) named what you specified for someExistingCollection. It's not the _collection_ that should be existing, it should be the configset. It's often a bit confusing because if the configName is not specified, the default is to look for a config set of the same name as the collection being created. Best, Erick On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen wrote: > We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud > deployed in Tomcat, not something like the Cloudera distribution (I note > that because I can't seem to find solrctl and other things referenced in > the Cloudera tutorials). > > I'm trying to create a new Solr collection like this: > > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > > Then I found this error message in the logs: > > org.apache.solr.common.cloud.ZooKeeperException: Specified config does not > exist in ZooKeeper:newCollection2 > at > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > at > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > Source) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Unknown Source) > > Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore > 'newCollection2_shard1_replica1': Unable to create core: > newCollection2_shard1_replica1 > at > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:495) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > at > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > at
Re: Using the collections API to create a new collection
I looked in the tree view and I have only a node called "configs." Nothing called "configsets." That's a serious problem, right? So if I'm reading this correctly, I should be able to create a configset based on an existing collection and load it into zookeeper once I find the right location to put it on our system. Sound right? https://cwiki.apache.org/confluence/display/solr/Config+Sets Thanks, Mike On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson wrote: > I bet you did not push the configuration to Zookeeper before creating > the collection. > If you look in your admin UI, the Cloud link and the "tree" version, > you'll find > a "configsets" directory that'll show you what you _have_ put in ZK, and > I'll > bet you find nothing like a config set (containing schema.xml etc) named > what > you specified for someExistingCollection. It's not the _collection_ > that should be > existing, it should be the configset. > > It's often a bit confusing because if the configName is not specified, > the default > is to look for a config set of the same name as the collection being > created. > > Best, > Erick > > On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen > wrote: > > We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud > > deployed in Tomcat, not something like the Cloudera distribution (I note > > that because I can't seem to find solrctl and other things referenced in > > the Cloudera tutorials). > > > > I'm trying to create a new Solr collection like this: > > > > > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > > > > Then I found this error message in the logs: > > > > org.apache.solr.common.cloud.ZooKeeperException: Specified config does > not > > exist in ZooKeeper:newCollection2 > > at > > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > > at > > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > > at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > > at > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > > at > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > > at > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > > at > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > > at > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > > at > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > > at > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > > at > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > > at > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) > > at > > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) > > at > > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > > at > > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > Source) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > > Source) > > at > > > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > > at java.lang.Thread.run(Unknown Source) > > > > Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log > > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore > > 'newCollection2_shard1_replica1': Unable to create core: > > newCollection2_shard1_replica1 > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:495) > > at > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > >
Re: Using the collections API to create a new collection
I added this to my solr.xml and restarted, but it didn't do anything even
though the path is valid and /opt/configsets contains a folder called base
with a conf folder and valid schema and solrconfig...
${configSetBaseDir:/opt/configsets}
Any ideas? Is there a way to force an update into zookeeper? Or should I
just purge the zookeeper data?
On Sat, Mar 14, 2015 at 3:02 PM, Mike Thomsen
wrote:
> I looked in the tree view and I have only a node called "configs." Nothing
> called "configsets." That's a serious problem, right? So if I'm reading
> this correctly, I should be able to create a configset based on an existing
> collection and load it into zookeeper once I find the right location to put
> it on our system. Sound right?
>
> https://cwiki.apache.org/confluence/display/solr/Config+Sets
>
> Thanks,
>
> Mike
>
> On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson
> wrote:
>
>> I bet you did not push the configuration to Zookeeper before creating
>> the collection.
>> If you look in your admin UI, the Cloud link and the "tree" version,
>> you'll find
>> a "configsets" directory that'll show you what you _have_ put in ZK, and
>> I'll
>> bet you find nothing like a config set (containing schema.xml etc) named
>> what
>> you specified for someExistingCollection. It's not the _collection_
>> that should be
>> existing, it should be the configset.
>>
>> It's often a bit confusing because if the configName is not specified,
>> the default
>> is to look for a config set of the same name as the collection being
>> created.
>>
>> Best,
>> Erick
>>
>> On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen
>> wrote:
>> > We're running SolrCloud 4.5.0. It's just a standard version of SolrCloud
>> > deployed in Tomcat, not something like the Cloudera distribution (I note
>> > that because I can't seem to find solrctl and other things referenced in
>> > the Cloudera tutorials).
>> >
>> > I'm trying to create a new Solr collection like this:
>> >
>> >
>> /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection
>> >
>> > Then I found this error message in the logs:
>> >
>> > org.apache.solr.common.cloud.ZooKeeperException: Specified config does
>> not
>> > exist in ZooKeeper:newCollection2
>> > at
>> > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742)
>> > at
>> > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244)
>> > at
>> org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
>> > at
>> >
>> org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465)
>> > at
>> >
>> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146)
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
>> > at
>> >
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
>> > at
>> >
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
>> > at
>> >
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
>> > at
>> >
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
>> > at
>> >
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>> > at
>> >
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
>> > at
>> >
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>> > at
>> >
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
>> > at
>> >
>> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
>> > at
>> >
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
>> > at
>> >
>> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
>> > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
>> Source)
>> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> > Source)
>> > at
>> >
>> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>> > at java.lang.Thread.run(Unknown Source)
>> >
>> > Mar 14, 2015 1:21:07 PM org.apache.solr.common.SolrException log
>> > SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCor
Re: Using the collections API to create a new collection
Hi Mike, Here's what you want to do: 1. Create or use an existing config set. 2. Upload it to ZooKeeper ( https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities) 3. Use the config name when you create the collection. This would link the config set in zk with your collection. I think it would make a lot of sense for you to go through the getting started with SolrCloud section in the Solr Reference Guide for 4.5. On Sat, Mar 14, 2015 at 12:02 PM, Mike Thomsen wrote: > I looked in the tree view and I have only a node called "configs." Nothing > called "configsets." That's a serious problem, right? So if I'm reading > this correctly, I should be able to create a configset based on an existing > collection and load it into zookeeper once I find the right location to put > it on our system. Sound right? > > https://cwiki.apache.org/confluence/display/solr/Config+Sets > > Thanks, > > Mike > > On Sat, Mar 14, 2015 at 2:27 PM, Erick Erickson > wrote: > > > I bet you did not push the configuration to Zookeeper before creating > > the collection. > > If you look in your admin UI, the Cloud link and the "tree" version, > > you'll find > > a "configsets" directory that'll show you what you _have_ put in ZK, and > > I'll > > bet you find nothing like a config set (containing schema.xml etc) named > > what > > you specified for someExistingCollection. It's not the _collection_ > > that should be > > existing, it should be the configset. > > > > It's often a bit confusing because if the configName is not specified, > > the default > > is to look for a config set of the same name as the collection being > > created. > > > > Best, > > Erick > > > > On Sat, Mar 14, 2015 at 10:26 AM, Mike Thomsen > > wrote: > > > We're running SolrCloud 4.5.0. It's just a standard version of > SolrCloud > > > deployed in Tomcat, not something like the Cloudera distribution (I > note > > > that because I can't seem to find solrctl and other things referenced > in > > > the Cloudera tutorials). > > > > > > I'm trying to create a new Solr collection like this: > > > > > > > > > /admin/collections?action=CREATE&name=newCollection&numShards=1&collection.configName=someExistingCollection > > > > > > Then I found this error message in the logs: > > > > > > org.apache.solr.common.cloud.ZooKeeperException: Specified config does > > not > > > exist in ZooKeeper:newCollection2 > > > at > > > > org.apache.solr.cloud.ZkController.readConfigName(ZkController.java:742) > > > at > > > org.apache.solr.core.ZkContainer.createFromZk(ZkContainer.java:244) > > > at > > org.apache.solr.core.CoreContainer.create(CoreContainer.java:557) > > > at > > > > > > org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:465) > > > at > > > > > > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:146) > > > at > > > > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:655) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:246) > > > at > > > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195) > > > at > > > > > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > > > at > > > > > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > > > at > > > > > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > > > at > > > > > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > > > at > > > > > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) > > > at > > > > > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > > > at > > > > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > > > at > > > > > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > > > at > > > > > > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) > > > at > > > > > > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074) > > > at > > > > > > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > > > at > > > > > > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > > Source) > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > > > Source) > > > at > > > > > > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(T
