Let's back up a second. Configsets are what _used_ to be in the conf directory for each core on a local drive, it's just that they're now kept up on Zookeeper. Otherwise, you'd have to put them on each instance in SolrCloud, and bringing up a new replica on a new machine would look a lot like adding a core with the old core admin API.
So instead, configurations are kept on zookeeper. A config set consists of, essentially, a named old-style "conf" directory. There's no a-priori limit to the number of config sets you can have. Look in the admin UI, Cloud>>tree>>configs and you'll see each name you've pushed to ZK. If you explore that tree, you'll see a lot of old familiar faces, schema.xml, solrconfig.xml, etc. So now we come to associating configs with collections. You've probably done one of the examples where some things happen under the covers, including explicitly pushing the configset to Zookeeper. Currently, there's no option in the bin/solr script to push a config, although I know there's a JIRA to do that. So, to put a new config set up you currently need to use the zkCli.sh script see: https://cwiki.apache.org/confluence/display/solr/Command+Line+Utilities, the "upconfig" command. That pushes the configset up to ZK and gives it a name. Now, you create a collection and it needs a configset stored in ZK. It's a little tricky in that if you do _not_ explicitly specify a configest (using the collection.configName parameter to the collections API CREATE command), then by default it'll look for a configset with the same name as the collection. If it doesn't find one, _and_ there is one and only one configset, then it'll use that one (personally I find that confusing, but that's the way it works). See: https://cwiki.apache.org/confluence/display/solr/Collections+API If you have two or more configsets in ZK, then either the configset name has to be identical to the collection name (if you don't specify collection.configName), _or_ you specify collection.configName at create time. NOTE: there are _no_ config files on the local disk! When a replica of a collection loads, it "knows" what collection it's part of and pulls the corresponding configset from ZK. So typically the process is this. > you create the config set by editing all the usual suspects, schema.xml, > solrconfig.xml, DIH config etc. > you put those configuration files into some version control system (you are > using one, right?) > you push the configs to Zookeeper > you create the collection > you figure out you need to change the configs so you > check the code out of your version control > edit them > put the current version back into version control > push the configs up to zookeeper, overwriting the ones already there with that name > reload the collection or bounce all the servers. As each replica in the collection comes up, it downloads the latest configs from Zookeeper to memory (not to disk) and uses them. Seems like a long drawn-out process, but pretty soon it's automatic. And really, the only extra step is the push to Zookeeper, the rest is just like old-style cores with the exception that you don't have to manually push all the configs to all the machines hosting cores. Notice that I have mostly avoided talking about "cores" here. Although it's true that a replica in a collection is just another core, it's "special" in that it has certain very specific properties set. I _strongly_ advise you stop thinking about old-style Solr cores and instead thing about collections and replicas. And above all, do _not_ use the admin core API to try to create members of a collection (cores), use the collections API to ADDREPLICA/DELETEREPLICA instead. Loading/unloading cores is less "fraught", but I try to avoid that too and use Best, Erick On Sat, Sep 19, 2015 at 9:08 PM, Ravi Solr <ravis...@gmail.com> wrote: > Thanks Erick, I will report back once the reindex is finished. Oh, your > answer reminded me of another question - Regarding configsets the > documentation says > > "On a multicore Solr instance, you may find that you want to share > configuration between a number of different cores." > > Can the same be used to push disparate mutually exclusive configs ? I ask > this as I have 4 mutually exclusive apps each with a 4 single core index on > a single machine which I am trying to convert to SolrCloud with single > shard approach. Just being lazy and trying to find a way to update and link > configs to zookeeper ;-) > > Thanks > > Rvai Kiran Bhaskar > > On Sat, Sep 19, 2015 at 6:54 PM, Erick Erickson <erickerick...@gmail.com> > wrote: > >> Just pushing up the entire configset would be easiest, but the >> Zookeeper command line tools allow you to push up a single >> file if you want. >> >> Yeah, it puzzles me too that the import worked yesterday, not really >> sure what happened, the file shouldn't just disappear.... >> >> Erick >> >> On Sat, Sep 19, 2015 at 2:46 PM, Ravi Solr <ravis...@gmail.com> wrote: >> > Thank you for the prompt response Erick. I did a full-import yesterday, >> you >> > are correct that I did not push dataimport.properties to ZK, should it >> have >> > not worked even for a full import ?. You may be right about 'clean' >> option, >> > I will reindex again today. BTW how do we push a single file to a >> specific >> > config name in zookeeper ? >> > >> > >> > Thanks, >> > >> > Ravi Kiran Bhaskar >> > >> > >> > On Sat, Sep 19, 2015 at 1:48 PM, Erick Erickson <erickerick...@gmail.com >> > >> > wrote: >> > >> >> Could not read DIH properties from >> >> /configs/sitesearchcore/dataimport.properties >> >> >> >> This looks like somehow you didn't push this file up to Zookeeper. You >> >> can check what files are there in the admin UI. How you indexed >> >> yesterday is a mystery though, unless somehow this file was removed >> >> from ZK. >> >> >> >> As for why you lost all the docs, my suspicion is that you have the >> >> clean param set up for delta import.... >> >> >> >> FWIW, >> >> Erick >> >> >> >> On Sat, Sep 19, 2015 at 10:36 AM, Ravi Solr <ravis...@gmail.com> wrote: >> >> > I am facing a weird problem. As part of upgrade from 4.7.2 >> (Master-Slave) >> >> > to 5.3.0 (Solrcloud) I re-indexed 1.5 million records via DIH using >> >> > SolrEntityProcessor yesterday, all of them indexed properly. Today >> >> morning >> >> > I just ran the DIH again with delta import and I lost all docs...what >> am >> >> I >> >> > missing ? Did anybody face similar issue ? >> >> > >> >> > Here are the errors in the logs >> >> > >> >> > 9/19/2015, 2:41:17 AM ERROR null SolrCore Previous SolrRequestInfo was >> >> not >> >> > closed! >> >> > req=waitSearcher=true&distrib.from= >> >> >> http://10.128.159.32:8983/solr/sitesearchcore/&update.distrib=FROMLEADER&openSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false >> >> > 9/19/2015, >> >> > 2:41:17 AM ERROR null SolrCore prev == info : false 9/19/2015, >> 2:41:17 AM >> >> > WARN null ZKPropertiesWriter Could not read DIH properties from >> >> > /configs/sitesearchcore/dataimport.properties :class >> >> > org.apache.zookeeper.KeeperException$NoNodeException >> >> > >> >> > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode >> >> > = NoNode for /configs/sitesearchcore/dataimport.properties >> >> > at >> >> org.apache.zookeeper.KeeperException.create(KeeperException.java:111) >> >> > at >> >> org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >> >> > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) >> >> > at >> >> org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:349) >> >> > at >> >> >> org.apache.solr.handler.dataimport.ZKPropertiesWriter.readIndexerProperties(ZKPropertiesWriter.java:91) >> >> > at >> >> >> org.apache.solr.handler.dataimport.ZKPropertiesWriter.persist(ZKPropertiesWriter.java:65) >> >> > at >> >> >> org.apache.solr.handler.dataimport.DocBuilder.finish(DocBuilder.java:307) >> >> > at >> >> >> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:253) >> >> > at >> >> >> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416) >> >> > at >> >> >> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480) >> >> > at >> >> >> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461) >> >> > >> >> > 9/19/2015, 11:16:43 AM ERROR null SolrCore Previous SolrRequestInfo >> was >> >> not >> >> > closed! >> >> > req=waitSearcher=true&distrib.from= >> >> >> http://10.128.159.32:8983/solr/sitesearchcore/&update.distrib=FROMLEADER&openSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false >> >> > 9/19/2015, >> >> > 11:16:43 AM ERROR null SolrCore prev == info : false >> >> > >> >> > >> >> > >> >> > Thanks >> >> > >> >> > Ravi Kiran Bhaskar >> >> >>
