FYI - this should be no problem now. You can upload config and make config -> collection links before starting Solr - using the ZkCLI cmd tool or just modifying zk yourself.
On Fri, May 18, 2012 at 10:12 AM, Mark Miller <markrmil...@gmail.com> wrote: > > On May 18, 2012, at 3:06 AM, Per Steffensen wrote: > >> First of all, sorry about the subject of this discussion. It should have >> been something like "Adding config to SolrCloud without starting a Solr >> server" >> >> Mark Miller skrev: >>> k >>> On May 16, 2012, at 5:35 AM, Per Steffensen wrote: >>> >>> >>>> Hi >>>> >>>> We want to create a Solr config in ZK during installation of our product, >>>> but we dont want to create any shards in that phase. We will create shards >>>> from our application when it starts up and also automatically maintain the >>>> set of shards from our application (which uses SolrCloud). The only way we >>>> know to create a Solr config in ZK is to spin up a Solr with system >>>> properties zkHost, bootstrap_confdir and collection.configName. Is there >>>> another, more API-ish, way of creating a Solr config in ZK? >>>> >>>> Regards, Per Steffensen >>>> >>> >>> I've started some work on this, but I have not finished. >>> >>> There is a main method in ZkController that has some initial code. >>> Currently it just lets you upload a specifically named config set directory >>> - I would also like to add the same multi core config set upload option we >>> have on startup - where it reads solr.xml, finds all the config dirs and >>> uploads them, and links each collection to a config set named after it. >>> >> Yeah ok, I just want the config created - no shards/slices/collections. > > That's all that is created. > >>> Technically, you could use any tool to set this up - there are a variety of >>> options in the zk world - you just have to place the config files under the >>> right node. >> I would really want to do it through Solr. This is the correct way, I think. >> So that, when you change your "strategy" e.g. location or format of configs >> in ZK, I will automatically inherit that. > > We have to commit to some level of back compat support with our ZK layout > regardless. We expect to expose it. > >>> There is one other tricky part though - the collection has to be set to the >>> right config set name. This is specified on the collection node in >>> ZooKeeper. When creating a new collection, you can specify this as a param. >>> If none is set and there is only one config set, that one config set is >>> used. However, some link must be made, and it is not done automatically >>> with your initial collections in solr.xml unless there is only one config >>> set. >>> >> I know about that, and will use Solr to create collections. I just want the >> config established in ZK before that, and not create the config "during the >> process of creating a collection". > > Yeah, but doing what you want is tricky because of that point. > >>> So now I'm thinking perhaps we should default the config set name to the >>> collection name. Then if you simply use the collection name initially when >>> you upload the set, no new linking is needed. If you don't like that, you >>> can explicitly override what config set to use. Convention would be to name >>> your config sets after your collection name, but extra work would allow you >>> to do whatever you want. >>> >> I want several collections to use the same config, so I would have to do >> that extra work. > > I'm not sure I have a great solution yet then. How are you creating your > initial collections? > > If you are creating them on the fly with solrj (a collections api coming soon > by the way), then you can simply give the collection set name to use when you > do. > > If you are creating them in solr.xml so that they exist on startup, and some > have to share config sets, I think we need to add something else. Perhaps a > hint property you could add to each core in solr.xml that caused a link to be > made when the core is first started? Since the config sets will be uploaded > first, we need some way of indicating to each collection which set to end up > using. > >>> You can find an example of the ZkContoller main method being used in >>> solr/cloud-dev scripts. The one caveat is that we pass an extra param to >>> solrhome and briefly run a ZkServer within the ZkController#main method >>> since we don't have an external ensemble. Normally this would not make >>> sense and you would want to leave that out. I need to clean this all up >>> (the ZkController params) and document it on the wiki as soon as I make >>> these couple tweaks though. >>> >> Ok > > I've actually made the changes that I said I would. So now, this would be > pretty easy if each collection had it's own config set. Let's work out how to > make your case a little easier as well. > >> >> Thanks, Mark >>> - Mark Miller >>> lucidimagination.com >>> >> >> Regards, Per Steffensen >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> > > - Mark Miller > lucidimagination.com > > > > > > > > > > > -- - Mark
