On May 18, 2012, at 3:06 AM, Per Steffensen wrote: > First of all, sorry about the subject of this discussion. It should have been > something like "Adding config to SolrCloud without starting a Solr server" > > Mark Miller skrev: >> k >> On May 16, 2012, at 5:35 AM, Per Steffensen wrote: >> >> >>> Hi >>> >>> We want to create a Solr config in ZK during installation of our product, >>> but we dont want to create any shards in that phase. We will create shards >>> from our application when it starts up and also automatically maintain the >>> set of shards from our application (which uses SolrCloud). The only way we >>> know to create a Solr config in ZK is to spin up a Solr with system >>> properties zkHost, bootstrap_confdir and collection.configName. Is there >>> another, more API-ish, way of creating a Solr config in ZK? >>> >>> Regards, Per Steffensen >>> >> >> I've started some work on this, but I have not finished. >> >> There is a main method in ZkController that has some initial code. Currently >> it just lets you upload a specifically named config set directory - I would >> also like to add the same multi core config set upload option we have on >> startup - where it reads solr.xml, finds all the config dirs and uploads >> them, and links each collection to a config set named after it. >> > Yeah ok, I just want the config created - no shards/slices/collections.
That's all that is created. >> Technically, you could use any tool to set this up - there are a variety of >> options in the zk world - you just have to place the config files under the >> right node. > I would really want to do it through Solr. This is the correct way, I think. > So that, when you change your "strategy" e.g. location or format of configs > in ZK, I will automatically inherit that. We have to commit to some level of back compat support with our ZK layout regardless. We expect to expose it. >> There is one other tricky part though - the collection has to be set to the >> right config set name. This is specified on the collection node in >> ZooKeeper. When creating a new collection, you can specify this as a param. >> If none is set and there is only one config set, that one config set is >> used. However, some link must be made, and it is not done automatically with >> your initial collections in solr.xml unless there is only one config set. >> > I know about that, and will use Solr to create collections. I just want the > config established in ZK before that, and not create the config "during the > process of creating a collection". Yeah, but doing what you want is tricky because of that point. >> So now I'm thinking perhaps we should default the config set name to the >> collection name. Then if you simply use the collection name initially when >> you upload the set, no new linking is needed. If you don't like that, you >> can explicitly override what config set to use. Convention would be to name >> your config sets after your collection name, but extra work would allow you >> to do whatever you want. >> > I want several collections to use the same config, so I would have to do that > extra work. I'm not sure I have a great solution yet then. How are you creating your initial collections? If you are creating them on the fly with solrj (a collections api coming soon by the way), then you can simply give the collection set name to use when you do. If you are creating them in solr.xml so that they exist on startup, and some have to share config sets, I think we need to add something else. Perhaps a hint property you could add to each core in solr.xml that caused a link to be made when the core is first started? Since the config sets will be uploaded first, we need some way of indicating to each collection which set to end up using. >> You can find an example of the ZkContoller main method being used in >> solr/cloud-dev scripts. The one caveat is that we pass an extra param to >> solrhome and briefly run a ZkServer within the ZkController#main method >> since we don't have an external ensemble. Normally this would not make sense >> and you would want to leave that out. I need to clean this all up (the >> ZkController params) and document it on the wiki as soon as I make these >> couple tweaks though. >> > Ok I've actually made the changes that I said I would. So now, this would be pretty easy if each collection had it's own config set. Let's work out how to make your case a little easier as well. > > Thanks, Mark >> - Mark Miller >> lucidimagination.com >> > > Regards, Per Steffensen >> >> >> >> >> >> >> >> >> >> >> >> > - Mark Miller lucidimagination.com
